[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:23:25.776536  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:34055
I20260709 12:23:25.778251  5137 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:23:25.779417  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:25.797470  5143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:25.797503  5144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:25.800562  5137 server_base.cc:1061] running on GCE node
W20260709 12:23:25.798146  5146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.003206  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:27.003510  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:27.003697  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599807003677 us; error 0 us; skew 500 ppm
I20260709 12:23:27.004623  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:27.011543  5137 webserver.cc:533] Webserver started at http://127.5.4.126:34457/ using document root <none> and password file <none>
I20260709 12:23:27.012534  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:27.012773  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:27.013271  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:27.020572  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "089e202e6d55477594676d6916a9424b"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.029996  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.002s	sys 0.008s
I20260709 12:23:27.036515  5152 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.037984  5137 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:23:27.038357  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "089e202e6d55477594676d6916a9424b"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.038724  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:27.093493  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:27.095227  5137 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:23:27.095692  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:27.178906  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:34055
I20260709 12:23:27.179004  5203 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:34055 every 8 connection(s)
I20260709 12:23:27.187700  5204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:27.212925  5204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b: Bootstrap starting.
I20260709 12:23:27.219875  5204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:27.222261  5204 log.cc:826] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b: Log is configured to *not* fsync() on all Append() calls
I20260709 12:23:27.227756  5204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b: No bootstrap required, opened a new log
I20260709 12:23:27.250159  5204 raft_consensus.cc:359] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089e202e6d55477594676d6916a9424b" member_type: VOTER }
I20260709 12:23:27.250831  5204 raft_consensus.cc:385] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:27.251091  5204 raft_consensus.cc:740] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 089e202e6d55477594676d6916a9424b, State: Initialized, Role: FOLLOWER
I20260709 12:23:27.252070  5204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089e202e6d55477594676d6916a9424b" member_type: VOTER }
I20260709 12:23:27.252570  5204 raft_consensus.cc:399] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:27.252827  5204 raft_consensus.cc:493] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:27.253140  5204 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:27.257746  5204 raft_consensus.cc:515] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089e202e6d55477594676d6916a9424b" member_type: VOTER }
I20260709 12:23:27.258617  5204 leader_election.cc:304] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 089e202e6d55477594676d6916a9424b; no voters: 
I20260709 12:23:27.260617  5204 leader_election.cc:290] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:27.261083  5207 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:27.264067  5207 raft_consensus.cc:697] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 1 LEADER]: Becoming Leader. State: Replica: 089e202e6d55477594676d6916a9424b, State: Running, Role: LEADER
I20260709 12:23:27.265172  5207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089e202e6d55477594676d6916a9424b" member_type: VOTER }
I20260709 12:23:27.266422  5204 sys_catalog.cc:565] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:27.279011  5209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 089e202e6d55477594676d6916a9424b. Latest consensus state: current_term: 1 leader_uuid: "089e202e6d55477594676d6916a9424b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089e202e6d55477594676d6916a9424b" member_type: VOTER } }
I20260709 12:23:27.278923  5208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "089e202e6d55477594676d6916a9424b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "089e202e6d55477594676d6916a9424b" member_type: VOTER } }
I20260709 12:23:27.279673  5209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:27.279868  5208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:27.282585  5215 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:27.297849  5215 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:27.303610  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:27.318476  5215 catalog_manager.cc:1383] Generated new cluster ID: 0fc381e1f0114e338062a1adc582942e
I20260709 12:23:27.318868  5215 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:27.341766  5215 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:27.343890  5215 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:27.357928  5215 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b: Generated new TSK 0
I20260709 12:23:27.359016  5215 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:27.371366  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:27.382541  5226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:27.384989  5227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.389706  5137 server_base.cc:1061] running on GCE node
W20260709 12:23:27.390749  5229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.392107  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:27.392458  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:27.392693  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599807392674 us; error 0 us; skew 500 ppm
I20260709 12:23:27.393668  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:27.398989  5137 webserver.cc:533] Webserver started at http://127.5.4.65:37125/ using document root <none> and password file <none>
I20260709 12:23:27.399827  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:27.400146  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:27.400563  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:27.402426  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "b7b9869a31434137a0c6006721b9d82e"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.410827  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:23:27.417404  5234 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.418622  5137 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:23:27.419064  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "b7b9869a31434137a0c6006721b9d82e"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.419570  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:27.442842  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:27.445065  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:27.448557  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:27.455242  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:27.455612  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.455976  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:27.456259  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.571599  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:39519
I20260709 12:23:27.571673  5296 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:39519 every 8 connection(s)
I20260709 12:23:27.581216  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:27.597709  5301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:27.598789  5302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.609681  5137 server_base.cc:1061] running on GCE node
W20260709 12:23:27.610883  5304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.611991  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:27.612280  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:27.612496  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599807612477 us; error 0 us; skew 500 ppm
I20260709 12:23:27.613106  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:27.616442  5137 webserver.cc:533] Webserver started at http://127.5.4.66:43225/ using document root <none> and password file <none>
I20260709 12:23:27.617113  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:27.617348  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:27.617697  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:27.618973  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.619925  5297 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34055
I20260709 12:23:27.620424  5297 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:27.621973  5297 heartbeater.cc:507] Master 127.5.4.126:34055 requested a full tablet report, sending...
I20260709 12:23:27.625814  5169 ts_manager.cc:194] Registered new tserver with Master: b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519)
I20260709 12:23:27.626309  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:27.628823  5169 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38490
I20260709 12:23:27.632077  5309 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.633177  5137 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:27.633512  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.633834  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:27.654623  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:27.655938  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:27.658097  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:27.661799  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:27.662060  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.662323  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:27.662495  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.764817  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:34061
I20260709 12:23:27.765112  5373 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:34061 every 8 connection(s)
I20260709 12:23:27.771704  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:27.782233  5377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:27.784803  5378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.788249  5374 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34055
I20260709 12:23:27.788700  5374 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:27.789745  5374 heartbeater.cc:507] Master 127.5.4.126:34055 requested a full tablet report, sending...
I20260709 12:23:27.791690  5137 server_base.cc:1061] running on GCE node
W20260709 12:23:27.792479  5381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:27.792553  5169 ts_manager.cc:194] Registered new tserver with Master: 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:27.793771  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:27.794104  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:27.794188  5169 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38492
I20260709 12:23:27.794376  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599807794356 us; error 0 us; skew 500 ppm
I20260709 12:23:27.795212  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:27.798218  5137 webserver.cc:533] Webserver started at http://127.5.4.67:36463/ using document root <none> and password file <none>
I20260709 12:23:27.798856  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:27.799154  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:27.799490  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:27.800992  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.806953  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:23:27.811251  5385 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.812240  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:23:27.812633  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
format_stamp: "Formatted at 2026-07-09 12:23:27 on dist-test-slave-lg8f"
I20260709 12:23:27.812995  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:27.840171  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:27.842190  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:27.845932  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:27.851487  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:27.851739  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.851994  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:27.852164  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:27.969424  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:44361
I20260709 12:23:27.969714  5448 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:44361 every 8 connection(s)
I20260709 12:23:27.990083  5449 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34055
I20260709 12:23:27.990610  5449 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:27.991595  5449 heartbeater.cc:507] Master 127.5.4.126:34055 requested a full tablet report, sending...
I20260709 12:23:27.993952  5169 ts_manager.cc:194] Registered new tserver with Master: 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:27.994408  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016337146s
I20260709 12:23:27.995924  5169 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38502
I20260709 12:23:28.052623  5169 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38510:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:28.057047  5169 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:28.177659  5337 tablet_service.cc:1511] Processing CreateTablet for tablet 2a96746f816946229f2dafb62041c3c7 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:28.178364  5338 tablet_service.cc:1511] Processing CreateTablet for tablet 39ce31bc32f54d39b4a1aadcf5c6673e (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:28.183686  5334 tablet_service.cc:1511] Processing CreateTablet for tablet 8b4dffe16edb489faa69eae70a4576dc (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:28.182010  5336 tablet_service.cc:1511] Processing CreateTablet for tablet 32b862bb13d94efe9bd649d8d7c1c48d (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:28.188519  5337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a96746f816946229f2dafb62041c3c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.188243  5333 tablet_service.cc:1511] Processing CreateTablet for tablet f4738de3cbeb44bc98d70ada18ae878b (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:28.180269  5335 tablet_service.cc:1511] Processing CreateTablet for tablet d863001bd204499e86120d2e106d5fb6 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:28.190507  5336 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b862bb13d94efe9bd649d8d7c1c48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.193934  5333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4738de3cbeb44bc98d70ada18ae878b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.197329  5335 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d863001bd204499e86120d2e106d5fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.193851  5332 tablet_service.cc:1511] Processing CreateTablet for tablet d05c0136f5484f84b954faf3595fb8e0 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:28.199280  5338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ce31bc32f54d39b4a1aadcf5c6673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.202298  5332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d05c0136f5484f84b954faf3595fb8e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.207767  5334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4dffe16edb489faa69eae70a4576dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.227597  5331 tablet_service.cc:1511] Processing CreateTablet for tablet 595873b824354ecbad935eeec61cd3a9 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:28.230649  5331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595873b824354ecbad935eeec61cd3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.249826  5330 tablet_service.cc:1511] Processing CreateTablet for tablet 2c1a48bb38974df9a4dc6b35d239966b (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:28.251397  5330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c1a48bb38974df9a4dc6b35d239966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.270798  5261 tablet_service.cc:1511] Processing CreateTablet for tablet 2a96746f816946229f2dafb62041c3c7 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:28.277346  5261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a96746f816946229f2dafb62041c3c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.277063  5262 tablet_service.cc:1511] Processing CreateTablet for tablet 39ce31bc32f54d39b4a1aadcf5c6673e (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:28.285640  5260 tablet_service.cc:1511] Processing CreateTablet for tablet 32b862bb13d94efe9bd649d8d7c1c48d (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:28.281399  5259 tablet_service.cc:1511] Processing CreateTablet for tablet d863001bd204499e86120d2e106d5fb6 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:28.301990  5412 tablet_service.cc:1511] Processing CreateTablet for tablet 2a96746f816946229f2dafb62041c3c7 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:23:28.311618  5407 tablet_service.cc:1511] Processing CreateTablet for tablet d05c0136f5484f84b954faf3595fb8e0 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:28.304708  5262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ce31bc32f54d39b4a1aadcf5c6673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.299263  5410 tablet_service.cc:1511] Processing CreateTablet for tablet d863001bd204499e86120d2e106d5fb6 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:23:28.308220  5413 tablet_service.cc:1511] Processing CreateTablet for tablet 39ce31bc32f54d39b4a1aadcf5c6673e (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:23:28.322230  5412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a96746f816946229f2dafb62041c3c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.305598  5255 tablet_service.cc:1511] Processing CreateTablet for tablet 595873b824354ecbad935eeec61cd3a9 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:28.305809  5257 tablet_service.cc:1511] Processing CreateTablet for tablet f4738de3cbeb44bc98d70ada18ae878b (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:28.337911  5259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d863001bd204499e86120d2e106d5fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.331844  5256 tablet_service.cc:1511] Processing CreateTablet for tablet d05c0136f5484f84b954faf3595fb8e0 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:23:28.338968  5407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d05c0136f5484f84b954faf3595fb8e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.343183  5466 tablet_bootstrap.cc:492] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:28.339938  5258 tablet_service.cc:1511] Processing CreateTablet for tablet 8b4dffe16edb489faa69eae70a4576dc (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:28.340188  5408 tablet_service.cc:1511] Processing CreateTablet for tablet f4738de3cbeb44bc98d70ada18ae878b (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:23:28.339993  5254 tablet_service.cc:1511] Processing CreateTablet for tablet 2c1a48bb38974df9a4dc6b35d239966b (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 12:23:28.353353  5218 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:28.346349  5411 tablet_service.cc:1511] Processing CreateTablet for tablet 32b862bb13d94efe9bd649d8d7c1c48d (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:23:28.344620  5409 tablet_service.cc:1511] Processing CreateTablet for tablet 8b4dffe16edb489faa69eae70a4576dc (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:23:28.355319  5406 tablet_service.cc:1511] Processing CreateTablet for tablet 595873b824354ecbad935eeec61cd3a9 (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:23:28.344638  5405 tablet_service.cc:1511] Processing CreateTablet for tablet 2c1a48bb38974df9a4dc6b35d239966b (DEFAULT_TABLE table=test-workload [id=ab84c300079f4a789e61018446cc313c]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:23:28.356913  5255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595873b824354ecbad935eeec61cd3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.359526  5258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4dffe16edb489faa69eae70a4576dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.362787  5256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d05c0136f5484f84b954faf3595fb8e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.368854  5260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b862bb13d94efe9bd649d8d7c1c48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.375238  5410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d863001bd204499e86120d2e106d5fb6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.387045  5413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39ce31bc32f54d39b4a1aadcf5c6673e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.388280  5257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4738de3cbeb44bc98d70ada18ae878b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.402199  5406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 595873b824354ecbad935eeec61cd3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.407013  5411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32b862bb13d94efe9bd649d8d7c1c48d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.416221  5405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c1a48bb38974df9a4dc6b35d239966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.424180  5466 tablet_bootstrap.cc:654] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.418748  5408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4738de3cbeb44bc98d70ada18ae878b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.425791  5254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c1a48bb38974df9a4dc6b35d239966b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.433568  5409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b4dffe16edb489faa69eae70a4576dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:28.452108  5467 tablet_bootstrap.cc:492] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.531219  5467 tablet_bootstrap.cc:654] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.542204  5466 tablet_bootstrap.cc:492] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:28.542768  5466 ts_tablet_manager.cc:1403] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.210s	user 0.028s	sys 0.008s
I20260709 12:23:28.548090  5466 raft_consensus.cc:359] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.549355  5466 raft_consensus.cc:385] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.549793  5466 raft_consensus.cc:740] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.550870  5469 tablet_bootstrap.cc:492] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:28.558254  5466 consensus_queue.cc:260] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.571805  5374 heartbeater.cc:499] Master 127.5.4.126:34055 was elected leader, sending a full tablet report...
I20260709 12:23:28.570567  5467 tablet_bootstrap.cc:492] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:28.573458  5467 ts_tablet_manager.cc:1403] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.123s	user 0.014s	sys 0.007s
I20260709 12:23:28.579843  5466 ts_tablet_manager.cc:1434] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.037s	user 0.012s	sys 0.000s
I20260709 12:23:28.581135  5466 tablet_bootstrap.cc:492] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:28.581125  5467 raft_consensus.cc:359] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.582044  5467 raft_consensus.cc:385] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.583710  5469 tablet_bootstrap.cc:654] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.582381  5467 raft_consensus.cc:740] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.586865  5467 consensus_queue.cc:260] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.599416  5466 tablet_bootstrap.cc:654] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.609652  5471 raft_consensus.cc:493] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.610323  5471 raft_consensus.cc:515] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.618454  5471 leader_election.cc:290] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:28.621973  5467 ts_tablet_manager.cc:1434] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.048s	user 0.003s	sys 0.023s
I20260709 12:23:28.621850  5449 heartbeater.cc:499] Master 127.5.4.126:34055 was elected leader, sending a full tablet report...
I20260709 12:23:28.623132  5467 tablet_bootstrap.cc:492] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.624420  5473 raft_consensus.cc:493] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.624993  5473 raft_consensus.cc:515] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.630434  5467 tablet_bootstrap.cc:654] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.632892  5297 heartbeater.cc:499] Master 127.5.4.126:34055 was elected leader, sending a full tablet report...
I20260709 12:23:28.637787  5469 tablet_bootstrap.cc:492] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:28.646689  5469 ts_tablet_manager.cc:1403] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.108s	user 0.003s	sys 0.030s
I20260709 12:23:28.647194  5473 leader_election.cc:290] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:28.663146  5467 tablet_bootstrap.cc:492] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:28.664752  5467 ts_tablet_manager.cc:1403] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.016s
I20260709 12:23:28.658677  5469 raft_consensus.cc:359] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.666217  5469 raft_consensus.cc:385] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.667328  5469 raft_consensus.cc:740] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.674784  5467 raft_consensus.cc:359] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.675657  5467 raft_consensus.cc:385] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.676010  5467 raft_consensus.cc:740] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.676834  5467 consensus_queue.cc:260] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.680595  5469 consensus_queue.cc:260] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.686452  5467 ts_tablet_manager.cc:1434] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:23:28.687744  5466 tablet_bootstrap.cc:492] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:28.694298  5467 tablet_bootstrap.cc:492] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.693902  5466 ts_tablet_manager.cc:1403] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.113s	user 0.019s	sys 0.006s
I20260709 12:23:28.697085  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:28.701813  5272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
W20260709 12:23:28.704980  5313 leader_election.cc:343] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.705269  5469 ts_tablet_manager.cc:1434] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.058s	user 0.013s	sys 0.003s
W20260709 12:23:28.705858  5313 leader_election.cc:343] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.706281  5469 tablet_bootstrap.cc:492] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:28.706292  5313 leader_election.cc:304] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0; no voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:28.706940  5466 raft_consensus.cc:359] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.708384  5466 raft_consensus.cc:385] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.708604  5471 raft_consensus.cc:2749] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:28.708738  5466 raft_consensus.cc:740] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.710538  5466 consensus_queue.cc:260] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.722036  5481 raft_consensus.cc:493] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.724747  5481 raft_consensus.cc:515] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.727268  5466 ts_tablet_manager.cc:1434] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.000s
I20260709 12:23:28.728470  5272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:28.731806  5466 tablet_bootstrap.cc:492] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
W20260709 12:23:28.731119  5389 leader_election.cc:343] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:28.732892  5450 tablet.cc:2406] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:28.732405  5481 leader_election.cc:290] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:28.741771  5469 tablet_bootstrap.cc:654] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.746042  5467 tablet_bootstrap.cc:654] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.764449  5467 tablet_bootstrap.cc:492] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:28.765012  5467 ts_tablet_manager.cc:1403] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.007s
I20260709 12:23:28.765646  5471 raft_consensus.cc:493] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:28.766790  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:28.768877  5471 raft_consensus.cc:515] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.772220  5471 leader_election.cc:290] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:28.775434  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:28.775714  5272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:28.777963  5466 tablet_bootstrap.cc:654] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:28.778039  5313 leader_election.cc:343] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.781174  5467 raft_consensus.cc:359] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
W20260709 12:23:28.781540  5313 leader_election.cc:343] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.782033  5467 raft_consensus.cc:385] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.782235  5313 leader_election.cc:304] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0; no voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:28.782514  5467 raft_consensus.cc:740] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.784358  5471 raft_consensus.cc:2749] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:28.785382  5389 leader_election.cc:343] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.786638  5389 leader_election.cc:304] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:28.787962  5473 raft_consensus.cc:2749] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:28.789582  5467 consensus_queue.cc:260] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.796264  5467 ts_tablet_manager.cc:1434] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 12:23:28.797626  5467 tablet_bootstrap.cc:492] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.808547  5469 tablet_bootstrap.cc:492] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:28.809162  5469 ts_tablet_manager.cc:1403] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.103s	user 0.016s	sys 0.024s
I20260709 12:23:28.809018  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:28.810976  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:28.812618  5423 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:28.815492  5238 leader_election.cc:304] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
W20260709 12:23:28.816574  5238 leader_election.cc:343] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.817051  5481 raft_consensus.cc:2804] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:28.817553  5481 raft_consensus.cc:493] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:28.818001  5481 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:28.820881  5467 tablet_bootstrap.cc:654] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.823390  5469 raft_consensus.cc:359] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.825423  5469 raft_consensus.cc:385] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.825736  5469 raft_consensus.cc:740] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.826545  5466 tablet_bootstrap.cc:492] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:28.827111  5466 ts_tablet_manager.cc:1403] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.096s	user 0.020s	sys 0.013s
I20260709 12:23:28.831112  5466 raft_consensus.cc:359] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.831954  5466 raft_consensus.cc:385] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.832278  5466 raft_consensus.cc:740] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.831231  5481 raft_consensus.cc:515] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.833145  5466 consensus_queue.cc:260] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.826540  5469 consensus_queue.cc:260] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.835700  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:28.837276  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:28.838016  5423 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:28.841784  5466 ts_tablet_manager.cc:1434] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:23:28.842945  5466 tablet_bootstrap.cc:492] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
W20260709 12:23:28.843320  5238 leader_election.cc:343] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:28.847985  5423 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:28.849453  5238 leader_election.cc:304] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:28.850160  5466 tablet_bootstrap.cc:654] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.834426  5481 leader_election.cc:290] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:28.856081  5481 raft_consensus.cc:2804] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:28.856627  5481 raft_consensus.cc:697] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:28.863262  5481 consensus_queue.cc:237] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:28.865310  5469 ts_tablet_manager.cc:1434] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.011s
I20260709 12:23:28.866680  5469 tablet_bootstrap.cc:492] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:28.870754  5467 tablet_bootstrap.cc:492] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:28.873482  5467 ts_tablet_manager.cc:1403] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.076s	user 0.013s	sys 0.018s
I20260709 12:23:28.878787  5467 raft_consensus.cc:359] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.879738  5467 raft_consensus.cc:385] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.879943  5469 tablet_bootstrap.cc:654] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.880092  5467 raft_consensus.cc:740] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.881230  5467 consensus_queue.cc:260] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.886260  5467 ts_tablet_manager.cc:1434] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:23:28.887272  5467 tablet_bootstrap.cc:492] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.892990  5466 tablet_bootstrap.cc:492] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:28.894223  5467 tablet_bootstrap.cc:654] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.894413  5466 ts_tablet_manager.cc:1403] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.009s
I20260709 12:23:28.898664  5466 raft_consensus.cc:359] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.899276  5466 raft_consensus.cc:385] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.899627  5466 raft_consensus.cc:740] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.900419  5466 consensus_queue.cc:260] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.901635  5469 tablet_bootstrap.cc:492] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:28.902230  5469 ts_tablet_manager.cc:1403] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.007s
I20260709 12:23:28.901257  5168 catalog_manager.cc:5697] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 0 to 1, leader changed from <none> to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 1 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:28.905390  5469 raft_consensus.cc:359] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.906117  5469 raft_consensus.cc:385] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.906415  5469 raft_consensus.cc:740] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.907215  5469 consensus_queue.cc:260] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.909658  5469 ts_tablet_manager.cc:1434] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:28.910100  5467 tablet_bootstrap.cc:492] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:28.910717  5469 tablet_bootstrap.cc:492] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:28.910704  5467 ts_tablet_manager.cc:1403] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.002s
I20260709 12:23:28.914840  5466 ts_tablet_manager.cc:1434] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:23:28.916077  5466 tablet_bootstrap.cc:492] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:28.918934  5469 tablet_bootstrap.cc:654] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.915024  5467 raft_consensus.cc:359] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.920286  5467 raft_consensus.cc:385] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.920624  5467 raft_consensus.cc:740] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.921662  5467 consensus_queue.cc:260] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.924220  5466 tablet_bootstrap.cc:654] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.924670  5467 ts_tablet_manager.cc:1434] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:23:28.932862  5467 tablet_bootstrap.cc:492] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.941727  5467 tablet_bootstrap.cc:654] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.945583  5469 tablet_bootstrap.cc:492] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:28.945614  5466 tablet_bootstrap.cc:492] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:28.946521  5466 ts_tablet_manager.cc:1403] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.001s
I20260709 12:23:28.949411  5467 tablet_bootstrap.cc:492] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:28.949795  5467 ts_tablet_manager.cc:1403] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.002s
I20260709 12:23:28.950505  5469 ts_tablet_manager.cc:1403] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.014s
I20260709 12:23:28.949784  5466 raft_consensus.cc:359] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.951879  5466 raft_consensus.cc:385] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.952237  5466 raft_consensus.cc:740] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.953189  5466 consensus_queue.cc:260] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.955731  5466 ts_tablet_manager.cc:1434] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:28.956790  5466 tablet_bootstrap.cc:492] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:28.954245  5467 raft_consensus.cc:359] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.957194  5467 raft_consensus.cc:385] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.957507  5467 raft_consensus.cc:740] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.958303  5467 consensus_queue.cc:260] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.960613  5467 ts_tablet_manager.cc:1434] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:28.963145  5469 raft_consensus.cc:359] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.963693  5467 tablet_bootstrap.cc:492] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:28.963925  5469 raft_consensus.cc:385] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:28.964257  5469 raft_consensus.cc:740] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:28.965047  5469 consensus_queue.cc:260] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:28.967525  5469 ts_tablet_manager.cc:1434] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:23:28.968568  5469 tablet_bootstrap.cc:492] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:28.971393  5467 tablet_bootstrap.cc:654] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.975672  5469 tablet_bootstrap.cc:654] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:28.978298  5466 tablet_bootstrap.cc:654] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.002514  5466 tablet_bootstrap.cc:492] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:29.003156  5466 ts_tablet_manager.cc:1403] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.047s	user 0.012s	sys 0.004s
I20260709 12:23:29.007100  5466 raft_consensus.cc:359] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.009125  5473 raft_consensus.cc:493] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.009825  5473 raft_consensus.cc:515] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.013652  5272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.014390  5473 leader_election.cc:290] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.008492  5466 raft_consensus.cc:385] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.014451  5272 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.015713  5466 raft_consensus.cc:740] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.015062  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.016834  5466 consensus_queue.cc:260] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.019547  5389 leader_election.cc:304] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.016891  5348 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.022516  5467 tablet_bootstrap.cc:492] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:29.022722  5473 raft_consensus.cc:2804] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.023023  5467 ts_tablet_manager.cc:1403] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.060s	user 0.024s	sys 0.032s
I20260709 12:23:29.023144  5473 raft_consensus.cc:493] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.023573  5473 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.026145  5467 raft_consensus.cc:359] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.026835  5467 raft_consensus.cc:385] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.027030  5467 raft_consensus.cc:740] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.027619  5467 consensus_queue.cc:260] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.031612  5469 tablet_bootstrap.cc:492] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:29.032100  5469 ts_tablet_manager.cc:1403] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.064s	user 0.010s	sys 0.036s
I20260709 12:23:29.032502  5473 raft_consensus.cc:515] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.034941  5469 raft_consensus.cc:359] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.035696  5469 raft_consensus.cc:385] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.035991  5469 raft_consensus.cc:740] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.036201  5272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.036818  5272 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.036727  5469 consensus_queue.cc:260] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.039085  5469 ts_tablet_manager.cc:1434] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:23:29.041263  5467 ts_tablet_manager.cc:1434] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:23:29.042301  5467 tablet_bootstrap.cc:492] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:29.048800  5471 raft_consensus.cc:493] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.048985  5504 raft_consensus.cc:493] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.049307  5471 raft_consensus.cc:515] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.049466  5504 raft_consensus.cc:515] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.033272  5466 ts_tablet_manager.cc:1434] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.001s
I20260709 12:23:29.051373  5471 leader_election.cc:290] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.051934  5467 tablet_bootstrap.cc:654] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.052736  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.053561  5271 raft_consensus.cc:2468] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 0.
I20260709 12:23:29.055032  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:29.055634  5423 raft_consensus.cc:2468] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 0.
I20260709 12:23:29.057277  5313 leader_election.cc:304] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.058043  5471 raft_consensus.cc:2804] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.058421  5471 raft_consensus.cc:493] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.058787  5471 raft_consensus.cc:3060] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.059650  5504 leader_election.cc:290] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.058794  5469 tablet_bootstrap.cc:492] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:29.062592  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.063203  5271 raft_consensus.cc:2468] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.064429  5389 leader_election.cc:304] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.065230  5504 raft_consensus.cc:2804] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.065616  5504 raft_consensus.cc:493] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.066612  5504 raft_consensus.cc:3060] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.069139  5467 tablet_bootstrap.cc:492] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:29.069590  5467 ts_tablet_manager.cc:1403] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.027s	user 0.003s	sys 0.011s
I20260709 12:23:29.065989  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.065876  5471 raft_consensus.cc:515] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.071062  5466 tablet_bootstrap.cc:492] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:29.072594  5469 tablet_bootstrap.cc:654] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.072414  5467 raft_consensus.cc:359] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.073105  5467 raft_consensus.cc:385] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.073400  5467 raft_consensus.cc:740] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.074146  5467 consensus_queue.cc:260] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.077092  5348 raft_consensus.cc:2393] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in current term 1: Already voted for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in this term.
I20260709 12:23:29.078881  5471 leader_election.cc:290] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.082108  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.083180  5504 raft_consensus.cc:515] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.085119  5471 raft_consensus.cc:493] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.085448  5423 raft_consensus.cc:2393] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in current term 1: Already voted for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in this term.
I20260709 12:23:29.085749  5471 raft_consensus.cc:515] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.086987  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.087599  5271 raft_consensus.cc:3060] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.088168  5467 ts_tablet_manager.cc:1434] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.011s
I20260709 12:23:29.089242  5467 tablet_bootstrap.cc:492] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: Bootstrap starting.
I20260709 12:23:29.090917  5471 leader_election.cc:290] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.092893  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:29.069585  5272 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 1.
I20260709 12:23:29.093930  5422 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 0.
I20260709 12:23:29.095604  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.096655  5270 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 0.
I20260709 12:23:29.097030  5473 leader_election.cc:290] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.100066  5467 tablet_bootstrap.cc:654] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.100752  5481 raft_consensus.cc:493] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.101807  5469 tablet_bootstrap.cc:492] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:29.101509  5481 raft_consensus.cc:515] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.099678  5313 leader_election.cc:304] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.103466  5471 raft_consensus.cc:2804] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.103878  5471 raft_consensus.cc:493] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.104272  5471 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.106469  5469 ts_tablet_manager.cc:1403] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.005s
I20260709 12:23:29.106218  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:29.108589  5422 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:29.109879  5238 leader_election.cc:304] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.104807  5481 leader_election.cc:290] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.106846  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.111253  5490 raft_consensus.cc:2804] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.112318  5348 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:29.112984  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.113991  5347 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.111662  5490 raft_consensus.cc:493] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.117246  5466 tablet_bootstrap.cc:654] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.117123  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.115680  5490 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.124852  5469 raft_consensus.cc:359] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.125643  5469 raft_consensus.cc:385] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.125944  5469 raft_consensus.cc:740] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.138680  5347 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 1.
I20260709 12:23:29.140277  5471 raft_consensus.cc:515] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.126749  5389 leader_election.cc:304] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.138708  5490 raft_consensus.cc:515] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.143329  5473 raft_consensus.cc:2804] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.143842  5473 raft_consensus.cc:697] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:29.144690  5467 tablet_bootstrap.cc:492] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: No bootstrap required, opened a new log
I20260709 12:23:29.145203  5467 ts_tablet_manager.cc:1403] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent bootstrapping tablet: real 0.056s	user 0.021s	sys 0.006s
I20260709 12:23:29.146013  5473 consensus_queue.cc:237] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.133782  5469 consensus_queue.cc:260] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.159353  5504 leader_election.cc:290] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.161458  5271 raft_consensus.cc:2468] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 1.
I20260709 12:23:29.164315  5466 tablet_bootstrap.cc:492] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:29.165817  5466 ts_tablet_manager.cc:1403] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.095s	user 0.009s	sys 0.014s
I20260709 12:23:29.166330  5469 ts_tablet_manager.cc:1434] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.057s	user 0.011s	sys 0.000s
I20260709 12:23:29.166270  5490 leader_election.cc:290] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.167744  5313 leader_election.cc:304] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 2fd208acf5b3467cb5bc36c3f25bfc86
I20260709 12:23:29.168228  5469 tablet_bootstrap.cc:492] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:29.167718  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.163628  5347 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b4dffe16edb489faa69eae70a4576dc" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.168848  5348 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.167718  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.171581  5347 raft_consensus.cc:2393] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in current term 1: Already voted for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in this term.
I20260709 12:23:29.165271  5471 leader_election.cc:290] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.176301  5422 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.176616  5471 raft_consensus.cc:2804] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.177165  5471 raft_consensus.cc:697] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:29.179543  5467 raft_consensus.cc:359] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.180248  5467 raft_consensus.cc:385] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.180495  5467 raft_consensus.cc:740] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.182021  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.182657  5423 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.183211  5467 consensus_queue.cc:260] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.183231  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.178193  5471 consensus_queue.cc:237] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.195590  5467 ts_tablet_manager.cc:1434] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: Time spent starting tablet: real 0.050s	user 0.005s	sys 0.004s
I20260709 12:23:29.202860  5466 raft_consensus.cc:359] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.203357  5389 leader_election.cc:304] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:29.203805  5466 raft_consensus.cc:385] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.200812  5271 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.204671  5167 catalog_manager.cc:5697] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:29.206300  5348 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:29.205809  5466 raft_consensus.cc:740] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.215371  5466 consensus_queue.cc:260] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.217336  5238 leader_election.cc:304] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.217840  5466 ts_tablet_manager.cc:1434] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.004s
I20260709 12:23:29.206873  5473 raft_consensus.cc:2749] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:29.207834  5422 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 1.
I20260709 12:23:29.220793  5313 leader_election.cc:304] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.221521  5466 tablet_bootstrap.cc:492] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:29.223466  5471 raft_consensus.cc:2804] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.224277  5471 raft_consensus.cc:697] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:29.225117  5471 consensus_queue.cc:237] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.208148  5469 tablet_bootstrap.cc:654] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.208726  5423 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:29.218277  5490 raft_consensus.cc:2804] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.228654  5490 raft_consensus.cc:697] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:29.229653  5490 consensus_queue.cc:237] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.231127  5271 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 1.
I20260709 12:23:29.227026  5168 catalog_manager.cc:5697] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.245206  5469 tablet_bootstrap.cc:492] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:29.245776  5469 ts_tablet_manager.cc:1403] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.078s	user 0.013s	sys 0.003s
I20260709 12:23:29.249155  5469 raft_consensus.cc:359] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.249925  5469 raft_consensus.cc:385] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.250245  5469 raft_consensus.cc:740] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.251295  5469 consensus_queue.cc:260] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.253530  5469 ts_tablet_manager.cc:1434] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:29.254400  5466 tablet_bootstrap.cc:654] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.254539  5469 tablet_bootstrap.cc:492] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:29.260617  5168 catalog_manager.cc:5697] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 0 to 1, leader changed from <none> to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 1 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.264518  5166 catalog_manager.cc:5697] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.266389  5466 tablet_bootstrap.cc:492] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:29.266959  5466 ts_tablet_manager.cc:1403] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.046s	user 0.014s	sys 0.004s
I20260709 12:23:29.271394  5466 raft_consensus.cc:359] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.272174  5469 tablet_bootstrap.cc:654] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.272320  5466 raft_consensus.cc:385] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.272933  5466 raft_consensus.cc:740] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.274016  5466 consensus_queue.cc:260] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.276841  5466 ts_tablet_manager.cc:1434] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:23:29.278137  5466 tablet_bootstrap.cc:492] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: Bootstrap starting.
I20260709 12:23:29.284605  5466 tablet_bootstrap.cc:654] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.286999  5473 raft_consensus.cc:493] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.287511  5473 raft_consensus.cc:515] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.289685  5473 leader_election.cc:290] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.290675  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.291302  5348 raft_consensus.cc:2468] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.292531  5389 leader_election.cc:304] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.293426  5473 raft_consensus.cc:2804] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.293816  5473 raft_consensus.cc:493] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.294145  5473 raft_consensus.cc:3060] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.290548  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.295035  5271 raft_consensus.cc:2468] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.300179  5473 raft_consensus.cc:515] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.304163  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.304598  5466 tablet_bootstrap.cc:492] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: No bootstrap required, opened a new log
I20260709 12:23:29.304824  5271 raft_consensus.cc:3060] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.305121  5466 ts_tablet_manager.cc:1403] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent bootstrapping tablet: real 0.027s	user 0.011s	sys 0.000s
I20260709 12:23:29.303565  5469 tablet_bootstrap.cc:492] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:29.308198  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.308862  5348 raft_consensus.cc:3060] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.309777  5466 raft_consensus.cc:359] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:29.310418  5466 raft_consensus.cc:385] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:29.310336  5238 consensus_peers.cc:597] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e -> Peer 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061): Couldn't send request to peer 1fd3a0f7a6784e5f93dd4385a263f6e0. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:29.310945  5466 raft_consensus.cc:740] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.311681  5466 consensus_queue.cc:260] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:29.308034  5473 leader_election.cc:290] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.314239  5466 ts_tablet_manager.cc:1434] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:29.315762  5348 raft_consensus.cc:2468] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 1.
I20260709 12:23:29.316743  5469 ts_tablet_manager.cc:1403] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.062s	user 0.021s	sys 0.007s
I20260709 12:23:29.316756  5389 leader_election.cc:304] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.317667  5473 raft_consensus.cc:2804] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.318297  5473 raft_consensus.cc:697] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:29.319090  5473 consensus_queue.cc:237] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.319558  5469 raft_consensus.cc:359] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.320153  5469 raft_consensus.cc:385] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.320406  5469 raft_consensus.cc:740] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.321034  5469 consensus_queue.cc:260] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.323683  5469 ts_tablet_manager.cc:1434] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:29.324838  5469 tablet_bootstrap.cc:492] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: Bootstrap starting.
I20260709 12:23:29.331888  5469 tablet_bootstrap.cc:654] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:29.336922  5469 tablet_bootstrap.cc:492] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: No bootstrap required, opened a new log
I20260709 12:23:29.337338  5469 ts_tablet_manager.cc:1403] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:23:29.338718  5169 catalog_manager.cc:5697] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:29.339823  5469 raft_consensus.cc:359] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.340610  5469 raft_consensus.cc:385] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:29.340893  5469 raft_consensus.cc:740] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Initialized, Role: FOLLOWER
I20260709 12:23:29.341487  5469 consensus_queue.cc:260] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.343779  5469 ts_tablet_manager.cc:1434] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:23:29.348470  5473 raft_consensus.cc:493] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.349102  5473 raft_consensus.cc:515] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.351204  5473 leader_election.cc:290] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.352612  5271 raft_consensus.cc:2468] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 1.
I20260709 12:23:29.353044  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.353756  5348 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.355311  5389 leader_election.cc:304] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.355924  5473 raft_consensus.cc:2804] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.356173  5473 raft_consensus.cc:493] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.356375  5473 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:29.357066  5218 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:29.357957  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.358680  5271 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 0.
I20260709 12:23:29.361555  5473 raft_consensus.cc:515] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.363337  5473 leader_election.cc:290] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.364360  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.364339  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.365023  5271 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.365070  5348 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.371604  5348 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 1.
I20260709 12:23:29.371608  5271 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 1.
I20260709 12:23:29.372912  5389 leader_election.cc:304] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:29.373610  5473 raft_consensus.cc:2804] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.374074  5473 raft_consensus.cc:697] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:29.374890  5473 consensus_queue.cc:237] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.382222  5490 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.382988  5169 catalog_manager.cc:5697] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:29.436129  5481 raft_consensus.cc:493] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.436640  5481 raft_consensus.cc:515] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.438324  5481 leader_election.cc:290] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.439615  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:29.439543  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.440418  5423 raft_consensus.cc:2468] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:29.440536  5348 raft_consensus.cc:2468] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:29.441421  5238 leader_election.cc:304] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.442147  5481 raft_consensus.cc:2804] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.442546  5481 raft_consensus.cc:493] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.442965  5481 raft_consensus.cc:3060] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.447525  5481 raft_consensus.cc:515] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.449399  5481 leader_election.cc:290] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.450295  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.450513  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.450868  5348 raft_consensus.cc:3060] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.451234  5423 raft_consensus.cc:3060] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.455358  5348 raft_consensus.cc:2468] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:29.456363  5238 leader_election.cc:304] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.457036  5481 raft_consensus.cc:2804] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.457474  5481 raft_consensus.cc:697] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:29.458289  5481 consensus_queue.cc:237] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.458451  5423 raft_consensus.cc:2468] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:29.466909  5169 catalog_manager.cc:5697] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 0 to 1, leader changed from <none> to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 1 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.511854  5481 raft_consensus.cc:493] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.512372  5481 raft_consensus.cc:515] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.514464  5481 leader_election.cc:290] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.515633  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" is_pre_election: true
I20260709 12:23:29.515956  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:29.516299  5348 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:29.516636  5423 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 0.
I20260709 12:23:29.517781  5238 leader_election.cc:304] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.518563  5481 raft_consensus.cc:2804] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:29.518991  5481 raft_consensus.cc:493] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:29.519400  5481 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.525720  5481 raft_consensus.cc:515] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.527427  5481 leader_election.cc:290] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.528476  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.529172  5348 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.529716  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.530248  5423 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.534902  5423 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:29.536044  5238 leader_election.cc:304] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.536780  5481 raft_consensus.cc:2804] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:29.536825  5348 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 1.
I20260709 12:23:29.537700  5481 raft_consensus.cc:697] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:29.538847  5481 consensus_queue.cc:237] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.548617  5169 catalog_manager.cc:5697] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 0 to 1, leader changed from <none> to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 1 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.553058  5473 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.570066  5506 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.619827  5471 raft_consensus.cc:493] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:29.620424  5471 raft_consensus.cc:515] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:29.622629  5471 leader_election.cc:290] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361), b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519)
I20260709 12:23:29.622830  5481 consensus_queue.cc:1048] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.623665  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" is_pre_election: true
I20260709 12:23:29.624310  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7b9869a31434137a0c6006721b9d82e" is_pre_election: true
I20260709 12:23:29.626863  5313 leader_election.cc:304] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0; no voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:29.627950  5422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
mode: GRACEFUL
new_leader_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:59914
I20260709 12:23:29.628240  5511 raft_consensus.cc:2749] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:29.628759  5422 raft_consensus.cc:606] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Received request to transfer leadership to TS b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:29.629473  5511 consensus_queue.cc:1048] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.638222  5471 consensus_queue.cc:1048] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.644204  5423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
mode: GRACEFUL
new_leader_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:59914
I20260709 12:23:29.644968  5423 raft_consensus.cc:606] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Received request to transfer leadership to TS b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:29.651070  5490 consensus_queue.cc:1048] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.703749  5271 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:33548
I20260709 12:23:29.704520  5271 raft_consensus.cc:606] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:29.711346  5423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59914
I20260709 12:23:29.711998  5423 raft_consensus.cc:606] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:29.713749  5471 consensus_queue.cc:1048] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.717329  5473 consensus_queue.cc:1048] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.722370  5270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
mode: GRACEFUL
new_leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:33548
I20260709 12:23:29.723122  5270 raft_consensus.cc:606] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Received request to transfer leadership to TS 2fd208acf5b3467cb5bc36c3f25bfc86
I20260709 12:23:29.725049  5348 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:29.750821  5506 consensus_queue.cc:1048] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.768139  5504 raft_consensus.cc:993] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:29.768303  5471 consensus_queue.cc:1048] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.768682  5504 raft_consensus.cc:1081] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:29.772754  5347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59796
I20260709 12:23:29.773442  5347 raft_consensus.cc:493] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:29.773903  5347 raft_consensus.cc:3060] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.780795  5347 raft_consensus.cc:515] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.783931  5347 leader_election.cc:290] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 2 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.783999  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.784821  5270 raft_consensus.cc:3060] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.785161  5423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05c0136f5484f84b954faf3595fb8e0" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.786301  5423 raft_consensus.cc:3055] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:29.786695  5423 raft_consensus.cc:740] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:29.787554  5423 consensus_queue.cc:260] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.788411  5506 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.788951  5423 raft_consensus.cc:3060] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.793658  5270 raft_consensus.cc:2468] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 2.
I20260709 12:23:29.795234  5313 leader_election.cc:304] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.796442  5471 raft_consensus.cc:2804] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:29.797092  5471 raft_consensus.cc:697] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:29.798070  5471 consensus_queue.cc:237] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.802034  5423 raft_consensus.cc:2468] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 2.
I20260709 12:23:29.819509  5168 catalog_manager.cc:5697] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 1 to 2, leader changed from 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.830010  5473 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:29.843724  5473 raft_consensus.cc:993] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: : Instructing follower b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:29.844213  5473 raft_consensus.cc:1081] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Signalling peer b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:29.846469  5271 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:33536
I20260709 12:23:29.847112  5271 raft_consensus.cc:493] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:29.847519  5271 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.851853  5481 consensus_queue.cc:1048] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.853895  5271 raft_consensus.cc:515] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.859122  5271 leader_election.cc:290] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 2 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:29.859916  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.860643  5348 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.867436  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:29.868218  5422 raft_consensus.cc:3055] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:29.868654  5422 raft_consensus.cc:740] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:29.869653  5422 consensus_queue.cc:260] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.872225  5490 raft_consensus.cc:993] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: : Instructing follower 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:29.873134  5481 raft_consensus.cc:1081] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Signalling peer 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:29.876089  5421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59908
I20260709 12:23:29.875902  5422 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.876721  5421 raft_consensus.cc:493] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:29.877133  5421 raft_consensus.cc:3060] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.879587  5348 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 2.
I20260709 12:23:29.881239  5238 leader_election.cc:304] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.882608  5481 raft_consensus.cc:2804] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:29.883292  5481 raft_consensus.cc:697] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:29.884428  5421 raft_consensus.cc:515] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.884532  5481 consensus_queue.cc:237] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.887610  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:29.888207  5271 raft_consensus.cc:3055] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:29.888819  5271 raft_consensus.cc:740] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:29.890033  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c1a48bb38974df9a4dc6b35d239966b" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:29.890956  5348 raft_consensus.cc:3060] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.890014  5271 consensus_queue.cc:260] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.891888  5271 raft_consensus.cc:3060] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:29.888197  5423 raft_consensus.cc:1240] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Rejecting Update request from peer b7b9869a31434137a0c6006721b9d82e for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:29.904331  5271 raft_consensus.cc:2468] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 2.
I20260709 12:23:29.905691  5389 leader_election.cc:304] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:29.906584  5504 raft_consensus.cc:2804] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:29.907038  5504 raft_consensus.cc:697] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:29.907836  5504 consensus_queue.cc:237] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:29.909459  5168 catalog_manager.cc:5697] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 1 to 2, leader changed from 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67) to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 2 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:29.909225  5422 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 2.
I20260709 12:23:29.890105  5421 leader_election.cc:290] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 2 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:29.916952  5348 raft_consensus.cc:2468] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 2.
I20260709 12:23:29.918045  5167 catalog_manager.cc:5697] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 1 to 2, leader changed from b7b9869a31434137a0c6006721b9d82e (127.5.4.65) to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:29.960090  5490 consensus_queue.cc:1048] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:29.972829  5546 consensus_queue.cc:1048] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:30.089177  5473 raft_consensus.cc:993] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: : Instructing follower b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:30.089718  5473 raft_consensus.cc:1081] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Signalling peer b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:30.091514  5271 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:33536
I20260709 12:23:30.092092  5271 raft_consensus.cc:493] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:30.092451  5271 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.099956  5271 raft_consensus.cc:515] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:30.102200  5271 leader_election.cc:290] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 2 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:30.103163  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:30.103499  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:30.103870  5348 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.104140  5421 raft_consensus.cc:3055] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:30.104547  5421 raft_consensus.cc:740] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:30.105327  5421 consensus_queue.cc:260] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:30.106287  5421 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.110859  5348 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 2.
I20260709 12:23:30.111088  5421 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 2.
I20260709 12:23:30.112363  5238 leader_election.cc:304] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:30.113323  5490 raft_consensus.cc:2804] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:30.113823  5490 raft_consensus.cc:697] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:30.114780  5490 consensus_queue.cc:237] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:30.123502  5168 catalog_manager.cc:5697] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 1 to 2, leader changed from 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67) to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 2 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:30.238217  5481 raft_consensus.cc:993] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:30.238809  5490 raft_consensus.cc:1081] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:30.240631  5348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59804
I20260709 12:23:30.241369  5348 raft_consensus.cc:493] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:30.241964  5348 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.248974  5348 raft_consensus.cc:515] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:30.251096  5348 leader_election.cc:290] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 2 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:30.252230  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:30.252686  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:30.253007  5271 raft_consensus.cc:3055] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:30.253297  5421 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.253396  5271 raft_consensus.cc:740] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:30.254385  5271 consensus_queue.cc:260] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:30.255800  5271 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:30.261595  5421 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 2.
I20260709 12:23:30.263360  5271 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 2.
I20260709 12:23:30.263252  5313 leader_election.cc:304] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:30.264468  5471 raft_consensus.cc:2804] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:30.265049  5471 raft_consensus.cc:697] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:30.265966  5471 consensus_queue.cc:237] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:30.267928  5271 raft_consensus.cc:1275] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.269373  5511 consensus_queue.cc:1048] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.277521  5167 catalog_manager.cc:5697] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 1 to 2, leader changed from b7b9869a31434137a0c6006721b9d82e (127.5.4.65) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:30.281594  5421 raft_consensus.cc:1275] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.283665  5511 consensus_queue.cc:1048] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.366515  5421 raft_consensus.cc:1275] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.368170  5490 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.378785  5348 raft_consensus.cc:1275] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.380427  5490 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.398880  5271 raft_consensus.cc:1275] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.400360  5473 consensus_queue.cc:1048] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.413575  5348 raft_consensus.cc:1275] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.415620  5549 consensus_queue.cc:1048] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:30.567252  5348 raft_consensus.cc:1275] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.568809  5490 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.577000  5421 raft_consensus.cc:1275] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.579113  5481 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.841226  5271 raft_consensus.cc:1275] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.842612  5534 consensus_queue.cc:1048] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.852593  5421 raft_consensus.cc:1275] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:30.854413  5511 consensus_queue.cc:1048] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:32.755595  5137 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:23:32.756500  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:32.777741  5270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
mode: GRACEFUL
new_leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:33558
I20260709 12:23:32.778414  5270 raft_consensus.cc:606] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Received request to transfer leadership to TS 2fd208acf5b3467cb5bc36c3f25bfc86
I20260709 12:23:32.790735  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
mode: GRACEFUL
new_leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59816
I20260709 12:23:32.791273  5348 raft_consensus.cc:606] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Received request to transfer leadership to TS 2fd208acf5b3467cb5bc36c3f25bfc86
I20260709 12:23:32.792474  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:32.792932  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:32.915747  5556 raft_consensus.cc:993] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: : Instructing follower 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:32.916266  5560 raft_consensus.cc:1081] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Signalling peer 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:32.917908  5421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59898
I20260709 12:23:32.918464  5421 raft_consensus.cc:493] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:32.918790  5421 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.923978  5421 raft_consensus.cc:515] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:32.925891  5421 leader_election.cc:290] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 2 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:32.926895  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:32.927227  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:32.927536  5270 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.927819  5348 raft_consensus.cc:3055] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:32.928167  5348 raft_consensus.cc:740] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:32.928840  5348 consensus_queue.cc:260] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:32.929755  5348 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:32.934314  5270 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 2.
I20260709 12:23:32.935254  5348 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 2.
I20260709 12:23:32.935711  5389 leader_election.cc:304] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:32.936513  5555 raft_consensus.cc:2804] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:32.937546  5555 raft_consensus.cc:697] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:32.938398  5555 consensus_queue.cc:237] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:32.946055  5167 catalog_manager.cc:5697] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 1 to 2, leader changed from 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66) to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:33.056078  5567 raft_consensus.cc:993] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: : Instructing follower 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:33.056603  5481 raft_consensus.cc:1081] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Signalling peer 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:33.058195  5421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59908
I20260709 12:23:33.058853  5421 raft_consensus.cc:493] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:33.059239  5421 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:33.064823  5421 raft_consensus.cc:515] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:33.066664  5421 leader_election.cc:290] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 2 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519)
I20260709 12:23:33.067349  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:33.067945  5348 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:33.067888  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:33.068535  5270 raft_consensus.cc:3055] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:33.068892  5270 raft_consensus.cc:740] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:33.069670  5270 consensus_queue.cc:260] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:33.070917  5270 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:33.074287  5348 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 2.
I20260709 12:23:33.075341  5389 leader_election.cc:304] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:33.076164  5555 raft_consensus.cc:2804] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:33.076685  5555 raft_consensus.cc:697] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:33.076682  5270 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 2.
I20260709 12:23:33.077539  5555 consensus_queue.cc:237] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:33.087805  5167 catalog_manager.cc:5697] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 1 to 2, leader changed from b7b9869a31434137a0c6006721b9d82e (127.5.4.65) to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:33.446025  5348 raft_consensus.cc:1275] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.447417  5555 consensus_queue.cc:1048] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.459808  5270 raft_consensus.cc:1275] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.462106  5555 consensus_queue.cc:1048] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.512408  5348 raft_consensus.cc:1275] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.513623  5555 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.523082  5270 raft_consensus.cc:1275] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:33.524585  5561 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:33.793896  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:33.798810  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:33.799220  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:34.800264  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:34.807687  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:34.808372  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:35.823076  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:35.830833  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:35.831313  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:36.832386  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:36.837471  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:36.837872  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:37.839299  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:37.846235  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:37.846755  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:38.847976  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:38.852906  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:38.853298  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:39.854241  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:39.859421  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:39.859849  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:40.860951  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.866587  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:40.867058  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:41.868235  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:41.873318  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:41.873823  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:42.904978  5137 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:23:42.909595  5422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59914
I20260709 12:23:42.910307  5422 raft_consensus.cc:606] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:42.915635  5270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:33548
I20260709 12:23:42.916129  5270 raft_consensus.cc:606] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:42.920714  5270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:33548
I20260709 12:23:42.921206  5270 raft_consensus.cc:606] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:42.925709  5270 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:33548
I20260709 12:23:42.926209  5270 raft_consensus.cc:606] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:42.938330  5422 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
mode: GRACEFUL
new_leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59914
I20260709 12:23:42.938918  5422 raft_consensus.cc:606] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Received request to transfer leadership to TS 1fd3a0f7a6784e5f93dd4385a263f6e0
I20260709 12:23:43.165262  5585 raft_consensus.cc:993] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.165853  5586 raft_consensus.cc:1081] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.167520  5348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59804
I20260709 12:23:43.168009  5348 raft_consensus.cc:493] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.168370  5348 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.174777  5348 raft_consensus.cc:515] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.176483  5348 leader_election.cc:290] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:43.177248  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:43.177539  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:43.178078  5270 raft_consensus.cc:3055] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:43.178176  5422 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.178401  5270 raft_consensus.cc:740] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:43.179234  5270 consensus_queue.cc:260] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.180199  5270 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.183419  5422 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.184530  5313 leader_election.cc:304] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:43.185104  5270 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.185294  5560 raft_consensus.cc:2804] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:43.187004  5560 raft_consensus.cc:697] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:43.187816  5560 consensus_queue.cc:237] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.194478  5168 catalog_manager.cc:5697] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 2 to 3, leader changed from b7b9869a31434137a0c6006721b9d82e (127.5.4.65) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.204348  5585 raft_consensus.cc:993] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.204885  5586 raft_consensus.cc:1081] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.206664  5348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59804
I20260709 12:23:43.207301  5348 raft_consensus.cc:493] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.207695  5348 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.212630  5348 raft_consensus.cc:515] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.213948  5348 leader_election.cc:290] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 2 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:43.214929  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:43.215512  5270 raft_consensus.cc:3055] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:43.215441  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:43.215886  5270 raft_consensus.cc:740] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:43.216050  5422 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.216687  5270 consensus_queue.cc:260] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.217814  5270 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.222527  5422 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 2.
I20260709 12:23:43.223569  5313 leader_election.cc:304] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:43.224232  5270 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 2.
I20260709 12:23:43.224320  5560 raft_consensus.cc:2804] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:43.224865  5560 raft_consensus.cc:697] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:43.225596  5560 consensus_queue.cc:237] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.232653  5168 catalog_manager.cc:5697] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 1 to 2, leader changed from b7b9869a31434137a0c6006721b9d82e (127.5.4.65) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.281271  5584 raft_consensus.cc:993] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.282442  5594 raft_consensus.cc:1081] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.283895  5594 raft_consensus.cc:993] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.284504  5595 raft_consensus.cc:1081] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.284360  5348 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59796
I20260709 12:23:43.285614  5348 raft_consensus.cc:493] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.286041  5348 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.286586  5347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59796
I20260709 12:23:43.287199  5347 raft_consensus.cc:493] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.287571  5347 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.291198  5348 raft_consensus.cc:515] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.292526  5348 leader_election.cc:290] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:43.293304  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:43.293740  5270 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.293609  5347 raft_consensus.cc:515] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:43.294986  5422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "595873b824354ecbad935eeec61cd3a9" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:43.295567  5422 raft_consensus.cc:3055] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:43.295863  5422 raft_consensus.cc:740] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:43.295955  5347 leader_election.cc:290] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Requested vote from peers 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361), b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519)
I20260709 12:23:43.297165  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:43.297573  5271 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.298974  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:43.299194  5270 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.299605  5421 raft_consensus.cc:3055] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:43.299988  5421 raft_consensus.cc:740] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:43.300609  5422 consensus_queue.cc:260] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.300863  5421 consensus_queue.cc:260] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:43.301740  5422 raft_consensus.cc:3060] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.302160  5421 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.305130  5271 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.305722  5422 raft_consensus.cc:2468] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.307298  5313 leader_election.cc:304] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:43.307960  5560 raft_consensus.cc:2804] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:43.308262  5313 leader_election.cc:304] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:43.308426  5560 raft_consensus.cc:697] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:43.309013  5421 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.309312  5587 raft_consensus.cc:2804] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:43.309386  5560 consensus_queue.cc:237] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.310914  5587 raft_consensus.cc:697] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:43.311647  5587 consensus_queue.cc:237] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:43.317950  5168 catalog_manager.cc:5697] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 2 to 3, leader changed from 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.328742  5168 catalog_manager.cc:5697] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 2 to 3, leader changed from 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.455237  5585 raft_consensus.cc:993] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: : Instructing follower 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.455801  5586 raft_consensus.cc:1081] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Signalling peer 1fd3a0f7a6784e5f93dd4385a263f6e0 to start an election
I20260709 12:23:43.457300  5347 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
 from {username='slave'} at 127.0.0.1:59804
I20260709 12:23:43.457798  5347 raft_consensus.cc:493] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.458113  5347 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.463004  5347 raft_consensus.cc:515] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.464695  5347 leader_election.cc:290] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:43.465509  5271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:43.465821  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:43.466101  5271 raft_consensus.cc:3055] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:43.466423  5421 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.466502  5271 raft_consensus.cc:740] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:43.467420  5271 consensus_queue.cc:260] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.468430  5271 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:43.471637  5421 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.472673  5313 leader_election.cc:304] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fd3a0f7a6784e5f93dd4385a263f6e0, 2fd208acf5b3467cb5bc36c3f25bfc86; no voters: 
I20260709 12:23:43.473146  5271 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fd3a0f7a6784e5f93dd4385a263f6e0 in term 3.
I20260709 12:23:43.473505  5587 raft_consensus.cc:2804] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:43.474022  5587 raft_consensus.cc:697] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Leader. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:43.474865  5587 consensus_queue.cc:237] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:43.481907  5168 catalog_manager.cc:5697] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 reported cstate change: term changed from 2 to 3, leader changed from b7b9869a31434137a0c6006721b9d82e (127.5.4.65) to 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.607239  5271 raft_consensus.cc:1275] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.608449  5587 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.618709  5421 raft_consensus.cc:1275] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.620131  5560 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.660737  5421 raft_consensus.cc:1275] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:43.662099  5560 consensus_queue.cc:1048] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:43.671675  5271 raft_consensus.cc:1275] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:43.673441  5587 consensus_queue.cc:1048] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:43.774291  5421 raft_consensus.cc:1275] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.775745  5606 consensus_queue.cc:1048] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:43.787665  5271 raft_consensus.cc:1275] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.788430  5423 raft_consensus.cc:1275] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.790048  5596 consensus_queue.cc:1048] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.790360  5606 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:43.802977  5271 raft_consensus.cc:1275] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.804548  5596 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.870745  5423 raft_consensus.cc:1275] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.872184  5596 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.881132  5271 raft_consensus.cc:1275] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Refusing update from remote peer 1fd3a0f7a6784e5f93dd4385a263f6e0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.882529  5606 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:45.974112  5137 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:23:45.975085  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.982324  5347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
mode: GRACEFUL
new_leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59816
I20260709 12:23:45.983022  5347 raft_consensus.cc:606] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Received request to transfer leadership to TS 2fd208acf5b3467cb5bc36c3f25bfc86
I20260709 12:23:45.985862  5347 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
mode: GRACEFUL
new_leader_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:59816
I20260709 12:23:45.986476  5347 raft_consensus.cc:606] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Received request to transfer leadership to TS b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:45.989595  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
mode: GRACEFUL
new_leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59816
I20260709 12:23:45.990260  5348 raft_consensus.cc:606] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Received request to transfer leadership to TS 2fd208acf5b3467cb5bc36c3f25bfc86
I20260709 12:23:45.992733  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
mode: GRACEFUL
new_leader_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:59816
I20260709 12:23:45.993211  5348 raft_consensus.cc:606] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Received request to transfer leadership to TS b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:45.995507  5348 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b"
dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
mode: GRACEFUL
new_leader_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:59816
I20260709 12:23:45.995901  5348 raft_consensus.cc:606] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Received request to transfer leadership to TS b7b9869a31434137a0c6006721b9d82e
I20260709 12:23:45.996905  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:45.997355  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:46.097565  5614 raft_consensus.cc:993] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: : Instructing follower b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:46.098173  5560 raft_consensus.cc:1081] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Signalling peer b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:46.099691  5270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:33526
I20260709 12:23:46.100147  5270 raft_consensus.cc:493] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.100425  5270 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.105842  5270 raft_consensus.cc:515] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.107883  5270 leader_election.cc:290] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 3 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:46.108929  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:46.109469  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4738de3cbeb44bc98d70ada18ae878b" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:46.109951  5348 raft_consensus.cc:3055] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:46.110312  5348 raft_consensus.cc:740] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:46.110361  5421 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.111198  5348 consensus_queue.cc:260] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.112231  5348 raft_consensus.cc:3060] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.116436  5421 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 3.
I20260709 12:23:46.118019  5238 leader_election.cc:304] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:46.118710  5348 raft_consensus.cc:2468] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 3.
I20260709 12:23:46.119926  5617 raft_consensus.cc:2804] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:46.121006  5617 raft_consensus.cc:697] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 3 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:46.122131  5617 consensus_queue.cc:237] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.130366  5167 catalog_manager.cc:5697] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 2 to 3, leader changed from 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66) to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 3 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.214887  5614 raft_consensus.cc:993] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: : Instructing follower b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:46.215503  5560 raft_consensus.cc:1081] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Signalling peer b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:46.217020  5270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:33526
I20260709 12:23:46.217588  5270 raft_consensus.cc:493] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.217885  5270 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.222425  5270 raft_consensus.cc:515] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.223960  5270 leader_election.cc:290] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 4 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:46.224980  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:46.225457  5348 raft_consensus.cc:3055] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:46.225273  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32b862bb13d94efe9bd649d8d7c1c48d" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:46.225761  5348 raft_consensus.cc:740] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:46.226014  5421 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.226639  5348 consensus_queue.cc:260] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.227569  5348 raft_consensus.cc:3060] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.230262  5421 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 4.
I20260709 12:23:46.231276  5238 leader_election.cc:304] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:46.231986  5617 raft_consensus.cc:2804] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:46.232483  5617 raft_consensus.cc:697] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 4 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:46.232780  5348 raft_consensus.cc:2468] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 4.
I20260709 12:23:46.233424  5617 consensus_queue.cc:237] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.240981  5167 catalog_manager.cc:5697] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 3 to 4, leader changed from 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66) to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 4 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.255012  5614 raft_consensus.cc:993] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: : Instructing follower b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:46.255476  5560 raft_consensus.cc:1081] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Signalling peer b7b9869a31434137a0c6006721b9d82e to start an election
I20260709 12:23:46.257061  5270 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6"
dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
 from {username='slave'} at 127.0.0.1:33526
I20260709 12:23:46.257607  5270 raft_consensus.cc:493] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.257958  5270 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.262717  5270 raft_consensus.cc:515] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.264108  5270 leader_election.cc:290] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 4 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67:44361)
I20260709 12:23:46.265095  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:46.265242  5421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d863001bd204499e86120d2e106d5fb6" candidate_uuid: "b7b9869a31434137a0c6006721b9d82e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
I20260709 12:23:46.265610  5348 raft_consensus.cc:3055] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:46.265812  5421 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.265937  5348 raft_consensus.cc:740] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:46.266829  5348 consensus_queue.cc:260] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.267742  5348 raft_consensus.cc:3060] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.270197  5421 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 4.
I20260709 12:23:46.271239  5238 leader_election.cc:304] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:46.272024  5617 raft_consensus.cc:2804] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:46.272441  5617 raft_consensus.cc:697] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 4 LEADER]: Becoming Leader. State: Replica: b7b9869a31434137a0c6006721b9d82e, State: Running, Role: LEADER
I20260709 12:23:46.272653  5348 raft_consensus.cc:2468] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7b9869a31434137a0c6006721b9d82e in term 4.
I20260709 12:23:46.273286  5617 consensus_queue.cc:237] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.280121  5167 catalog_manager.cc:5697] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e reported cstate change: term changed from 3 to 4, leader changed from 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66) to b7b9869a31434137a0c6006721b9d82e (127.5.4.65). New cstate: current_term: 4 leader_uuid: "b7b9869a31434137a0c6006721b9d82e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.323093  5614 raft_consensus.cc:993] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: : Instructing follower 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:46.323550  5560 raft_consensus.cc:1081] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Signalling peer 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:46.325071  5421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59898
I20260709 12:23:46.325459  5421 raft_consensus.cc:493] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.325682  5421 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.329766  5421 raft_consensus.cc:515] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.331812  5421 leader_election.cc:290] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 3 election: Requested vote from peers b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519), 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061)
I20260709 12:23:46.332863  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:46.333524  5348 raft_consensus.cc:3055] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:46.333420  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a96746f816946229f2dafb62041c3c7" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:46.333889  5348 raft_consensus.cc:740] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:46.334064  5270 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.334709  5348 consensus_queue.cc:260] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.335747  5614 raft_consensus.cc:993] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: : Instructing follower 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:46.335934  5348 raft_consensus.cc:3060] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.336249  5560 raft_consensus.cc:1081] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Signalling peer 2fd208acf5b3467cb5bc36c3f25bfc86 to start an election
I20260709 12:23:46.337586  5421 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e"
dest_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86"
 from {username='slave'} at 127.0.0.1:59898
I20260709 12:23:46.338078  5421 raft_consensus.cc:493] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.338414  5421 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.341261  5270 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 3.
I20260709 12:23:46.342768  5348 raft_consensus.cc:2468] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 3.
I20260709 12:23:46.342821  5389 leader_election.cc:304] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:46.343566  5421 raft_consensus.cc:515] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:46.343822  5595 raft_consensus.cc:2804] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:46.345309  5595 raft_consensus.cc:697] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:46.345551  5421 leader_election.cc:290] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 4 election: Requested vote from peers 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66:34061), b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519)
I20260709 12:23:46.346233  5595 consensus_queue.cc:237] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } }
I20260709 12:23:46.346668  5270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "b7b9869a31434137a0c6006721b9d82e"
I20260709 12:23:46.347251  5270 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.347231  5348 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39ce31bc32f54d39b4a1aadcf5c6673e" candidate_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0"
I20260709 12:23:46.347859  5348 raft_consensus.cc:3055] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:46.348174  5348 raft_consensus.cc:740] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1fd3a0f7a6784e5f93dd4385a263f6e0, State: Running, Role: LEADER
I20260709 12:23:46.348917  5348 consensus_queue.cc:260] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:46.350006  5348 raft_consensus.cc:3060] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:46.354259  5270 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 4.
I20260709 12:23:46.354254  5167 catalog_manager.cc:5697] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 2 to 3, leader changed from 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66) to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:46.355496  5389 leader_election.cc:304] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fd208acf5b3467cb5bc36c3f25bfc86, b7b9869a31434137a0c6006721b9d82e; no voters: 
I20260709 12:23:46.356281  5595 raft_consensus.cc:2804] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:46.356449  5348 raft_consensus.cc:2468] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fd208acf5b3467cb5bc36c3f25bfc86 in term 4.
I20260709 12:23:46.357501  5595 raft_consensus.cc:697] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 LEADER]: Becoming Leader. State: Replica: 2fd208acf5b3467cb5bc36c3f25bfc86, State: Running, Role: LEADER
I20260709 12:23:46.358230  5595 consensus_queue.cc:237] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } }
I20260709 12:23:46.365954  5167 catalog_manager.cc:5697] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 reported cstate change: term changed from 3 to 4, leader changed from 1fd3a0f7a6784e5f93dd4385a263f6e0 (127.5.4.66) to 2fd208acf5b3467cb5bc36c3f25bfc86 (127.5.4.67). New cstate: current_term: 4 leader_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.529526  5421 raft_consensus.cc:1275] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.530824  5617 consensus_queue.cc:1048] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.542739  5348 raft_consensus.cc:1275] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.545931  5617 consensus_queue.cc:1048] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.675810  5348 raft_consensus.cc:1275] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:46.677094  5617 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.687410  5421 raft_consensus.cc:1275] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:46.688876  5617 consensus_queue.cc:1048] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.752486  5348 raft_consensus.cc:1275] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:46.753800  5617 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.764626  5421 raft_consensus.cc:1275] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Refusing update from remote peer b7b9869a31434137a0c6006721b9d82e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:46.766606  5348 raft_consensus.cc:1275] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.766446  5618 consensus_queue.cc:1048] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd208acf5b3467cb5bc36c3f25bfc86" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:46.767983  5595 consensus_queue.cc:1048] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.782606  5270 raft_consensus.cc:1275] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.784408  5640 consensus_queue.cc:1048] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.858623  5347 raft_consensus.cc:1275] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:46.860096  5640 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fd3a0f7a6784e5f93dd4385a263f6e0" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.869968  5270 raft_consensus.cc:1275] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Refusing update from remote peer 2fd208acf5b3467cb5bc36c3f25bfc86: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:46.871883  5640 consensus_queue.cc:1048] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7b9869a31434137a0c6006721b9d82e" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.998556  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:47.003407  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:47.003801  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:48.004968  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:48.011554  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:48.012074  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:49.013244  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:49.018575  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:49.018980  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.019937  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.025347  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:50.025749  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:51.026926  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.033015  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:51.033691  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:52.034855  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:52.039676  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:52.040145  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:53.041231  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:53.046841  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:53.047304  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:54.048341  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:54.053059  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:54.053458  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:55.054539  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:55.059607  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:23:55.060092  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:56.090811  5137 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:23:56.091744  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
I20260709 12:23:56.125681  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:56.126590  5137 tablet_replica.cc:333] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.127578  5137 raft_consensus.cc:2243] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.128422  5137 raft_consensus.cc:2272] T 2a96746f816946229f2dafb62041c3c7 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.130750  5137 tablet_replica.cc:333] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.131275  5137 raft_consensus.cc:2243] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.132130  5137 raft_consensus.cc:2272] T 32b862bb13d94efe9bd649d8d7c1c48d P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.133711  5137 tablet_replica.cc:333] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.134222  5137 raft_consensus.cc:2243] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.134701  5137 raft_consensus.cc:2272] T 8b4dffe16edb489faa69eae70a4576dc P b7b9869a31434137a0c6006721b9d82e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.136369  5137 tablet_replica.cc:333] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.136833  5137 raft_consensus.cc:2243] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.137333  5137 raft_consensus.cc:2272] T 2c1a48bb38974df9a4dc6b35d239966b P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.138939  5137 tablet_replica.cc:333] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.139420  5137 raft_consensus.cc:2243] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.139830  5137 raft_consensus.cc:2272] T 595873b824354ecbad935eeec61cd3a9 P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.141417  5137 tablet_replica.cc:333] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.141993  5137 raft_consensus.cc:2243] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.142915  5137 raft_consensus.cc:2272] T f4738de3cbeb44bc98d70ada18ae878b P b7b9869a31434137a0c6006721b9d82e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.144510  5137 tablet_replica.cc:333] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.144958  5137 raft_consensus.cc:2243] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.145335  5137 raft_consensus.cc:2272] T d05c0136f5484f84b954faf3595fb8e0 P b7b9869a31434137a0c6006721b9d82e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.146888  5137 tablet_replica.cc:333] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.147329  5137 raft_consensus.cc:2243] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.148085  5137 raft_consensus.cc:2272] T d863001bd204499e86120d2e106d5fb6 P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.149602  5137 tablet_replica.cc:333] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e: stopping tablet replica
I20260709 12:23:56.150024  5137 raft_consensus.cc:2243] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.150521  5137 raft_consensus.cc:2272] T 39ce31bc32f54d39b4a1aadcf5c6673e P b7b9869a31434137a0c6006721b9d82e [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:56.383795  5389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39519: connect: Connection refused (error 111)
I20260709 12:23:56.387814  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
W20260709 12:23:56.392343  5389 consensus_peers.cc:597] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 -> Peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Couldn't send request to peer b7b9869a31434137a0c6006721b9d82e. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:56.393011  5389 consensus_peers.cc:597] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 -> Peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Couldn't send request to peer b7b9869a31434137a0c6006721b9d82e. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:56.393448  5389 consensus_peers.cc:597] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 -> Peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Couldn't send request to peer b7b9869a31434137a0c6006721b9d82e. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:56.394675  5313 consensus_peers.cc:597] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 -> Peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Couldn't send request to peer b7b9869a31434137a0c6006721b9d82e. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:56.413069  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
W20260709 12:23:56.423182  5313 consensus_peers.cc:597] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 -> Peer b7b9869a31434137a0c6006721b9d82e (127.5.4.65:39519): Couldn't send request to peer b7b9869a31434137a0c6006721b9d82e. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39519: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:56.449636  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:56.450423  5137 tablet_replica.cc:333] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.451083  5137 raft_consensus.cc:2243] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.451562  5137 raft_consensus.cc:2272] T 39ce31bc32f54d39b4a1aadcf5c6673e P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.453660  5137 tablet_replica.cc:333] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.454133  5137 raft_consensus.cc:2243] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.454566  5137 raft_consensus.cc:2272] T 2a96746f816946229f2dafb62041c3c7 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.456127  5137 tablet_replica.cc:333] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.456553  5137 raft_consensus.cc:2243] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.456907  5137 raft_consensus.cc:2272] T 2c1a48bb38974df9a4dc6b35d239966b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.458285  5137 tablet_replica.cc:333] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.458799  5137 raft_consensus.cc:2243] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.459525  5137 raft_consensus.cc:2272] T 595873b824354ecbad935eeec61cd3a9 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.461094  5137 tablet_replica.cc:333] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.461526  5137 raft_consensus.cc:2243] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.461866  5137 raft_consensus.cc:2272] T f4738de3cbeb44bc98d70ada18ae878b P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.463375  5137 tablet_replica.cc:333] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.463768  5137 raft_consensus.cc:2243] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.464389  5137 raft_consensus.cc:2272] T 8b4dffe16edb489faa69eae70a4576dc P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.465833  5137 tablet_replica.cc:333] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.466311  5137 raft_consensus.cc:2243] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.466750  5137 raft_consensus.cc:2272] T 32b862bb13d94efe9bd649d8d7c1c48d P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.468324  5137 tablet_replica.cc:333] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.468741  5137 raft_consensus.cc:2243] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.469372  5137 raft_consensus.cc:2272] T d05c0136f5484f84b954faf3595fb8e0 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.470857  5137 tablet_replica.cc:333] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0: stopping tablet replica
I20260709 12:23:56.471251  5137 raft_consensus.cc:2243] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.471709  5137 raft_consensus.cc:2272] T d863001bd204499e86120d2e106d5fb6 P 1fd3a0f7a6784e5f93dd4385a263f6e0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.500509  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
I20260709 12:23:56.519785  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
I20260709 12:23:56.552210  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:56.552979  5137 tablet_replica.cc:333] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.553719  5137 raft_consensus.cc:2243] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.554338  5137 raft_consensus.cc:2272] T 32b862bb13d94efe9bd649d8d7c1c48d P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.556440  5137 tablet_replica.cc:333] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.557004  5137 raft_consensus.cc:2243] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.557442  5137 raft_consensus.cc:2272] T 8b4dffe16edb489faa69eae70a4576dc P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.559316  5137 tablet_replica.cc:333] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.559870  5137 raft_consensus.cc:2243] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.560338  5137 raft_consensus.cc:2272] T f4738de3cbeb44bc98d70ada18ae878b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.562103  5137 tablet_replica.cc:333] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.562700  5137 raft_consensus.cc:2243] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.563170  5137 raft_consensus.cc:2272] T d863001bd204499e86120d2e106d5fb6 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.564913  5137 tablet_replica.cc:333] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.565510  5137 raft_consensus.cc:2243] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.566001  5137 raft_consensus.cc:2272] T d05c0136f5484f84b954faf3595fb8e0 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.567714  5137 tablet_replica.cc:333] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.568233  5137 raft_consensus.cc:2243] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.569214  5137 raft_consensus.cc:2272] T 39ce31bc32f54d39b4a1aadcf5c6673e P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.570957  5137 tablet_replica.cc:333] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.571444  5137 raft_consensus.cc:2243] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.572228  5137 raft_consensus.cc:2272] T 2c1a48bb38974df9a4dc6b35d239966b P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.573976  5137 tablet_replica.cc:333] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.574469  5137 raft_consensus.cc:2243] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:56.574967  5137 raft_consensus.cc:2272] T 595873b824354ecbad935eeec61cd3a9 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.576674  5137 tablet_replica.cc:333] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86: stopping tablet replica
I20260709 12:23:56.577136  5137 raft_consensus.cc:2243] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.577894  5137 raft_consensus.cc:2272] T 2a96746f816946229f2dafb62041c3c7 P 2fd208acf5b3467cb5bc36c3f25bfc86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.607241  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:23:56.630584  5137 master.cc:562] Master@127.5.4.126:34055 shutting down...
I20260709 12:23:56.787598  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:56.788236  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:56.788532  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P 089e202e6d55477594676d6916a9424b: stopping tablet replica
I20260709 12:23:56.808463  5137 master.cc:584] Master@127.5.4.126:34055 shutdown complete.
1fd3a0f7a6784e5f93dd4385a263f6e0  4
2fd208acf5b3467cb5bc36c3f25bfc86  1
b7b9869a31434137a0c6006721b9d82e  4
1fd3a0f7a6784e5f93dd4385a263f6e0  3
2fd208acf5b3467cb5bc36c3f25bfc86  3
b7b9869a31434137a0c6006721b9d82e  3
1fd3a0f7a6784e5f93dd4385a263f6e0  8
2fd208acf5b3467cb5bc36c3f25bfc86  1
b7b9869a31434137a0c6006721b9d82e  0
1fd3a0f7a6784e5f93dd4385a263f6e0  3
2fd208acf5b3467cb5bc36c3f25bfc86  3
b7b9869a31434137a0c6006721b9d82e  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31078 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:23:56.852258  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:36811
I20260709 12:23:56.853330  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:56.859055  5659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:56.859745  5660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:56.860745  5662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:56.862213  5137 server_base.cc:1061] running on GCE node
I20260709 12:23:56.863160  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:56.863350  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:56.863565  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599836863546 us; error 0 us; skew 500 ppm
I20260709 12:23:56.864116  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:56.866640  5137 webserver.cc:533] Webserver started at http://127.5.4.126:42029/ using document root <none> and password file <none>
I20260709 12:23:56.867208  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:56.867410  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:56.867707  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:56.868826  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "62b5ffc2872a4abb9b86c3a47afb6a94"
format_stamp: "Formatted at 2026-07-09 12:23:56 on dist-test-slave-lg8f"
I20260709 12:23:56.873684  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.001s
I20260709 12:23:56.877007  5667 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:56.877895  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:23:56.878211  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "62b5ffc2872a4abb9b86c3a47afb6a94"
format_stamp: "Formatted at 2026-07-09 12:23:56 on dist-test-slave-lg8f"
I20260709 12:23:56.878548  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:56.898940  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:56.900092  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:56.939282  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:36811
I20260709 12:23:56.939405  5719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:36811 every 8 connection(s)
I20260709 12:23:56.943436  5720 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:56.953636  5720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94: Bootstrap starting.
I20260709 12:23:56.958853  5720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:56.963522  5720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94: No bootstrap required, opened a new log
I20260709 12:23:56.965788  5720 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" member_type: VOTER }
I20260709 12:23:56.966236  5720 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:56.966434  5720 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62b5ffc2872a4abb9b86c3a47afb6a94, State: Initialized, Role: FOLLOWER
I20260709 12:23:56.966962  5720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" member_type: VOTER }
I20260709 12:23:56.967387  5720 raft_consensus.cc:399] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:56.967592  5720 raft_consensus.cc:493] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:56.967801  5720 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:56.972034  5720 raft_consensus.cc:515] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" member_type: VOTER }
I20260709 12:23:56.972524  5720 leader_election.cc:304] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62b5ffc2872a4abb9b86c3a47afb6a94; no voters: 
I20260709 12:23:56.973757  5720 leader_election.cc:290] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:56.974363  5723 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:56.975945  5723 raft_consensus.cc:697] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 1 LEADER]: Becoming Leader. State: Replica: 62b5ffc2872a4abb9b86c3a47afb6a94, State: Running, Role: LEADER
I20260709 12:23:56.976683  5723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" member_type: VOTER }
I20260709 12:23:56.977029  5720 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:56.981513  5725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62b5ffc2872a4abb9b86c3a47afb6a94. Latest consensus state: current_term: 1 leader_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" member_type: VOTER } }
I20260709 12:23:56.981798  5724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62b5ffc2872a4abb9b86c3a47afb6a94" member_type: VOTER } }
I20260709 12:23:56.982168  5725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:56.982272  5724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:56.985016  5730 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:56.991657  5730 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:56.992944  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:57.001201  5730 catalog_manager.cc:1383] Generated new cluster ID: 17f6657543de4cf9a77ca30c202d5014
I20260709 12:23:57.001518  5730 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:57.026718  5730 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:57.027878  5730 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:57.050410  5730 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94: Generated new TSK 0
I20260709 12:23:57.051044  5730 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:57.059104  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:57.065387  5741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:57.066375  5742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:57.067768  5744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:57.068632  5137 server_base.cc:1061] running on GCE node
I20260709 12:23:57.069555  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:57.069738  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:57.069933  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599837069902 us; error 0 us; skew 500 ppm
I20260709 12:23:57.070461  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:57.072907  5137 webserver.cc:533] Webserver started at http://127.5.4.65:44799/ using document root <none> and password file <none>
I20260709 12:23:57.073426  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:57.073611  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:57.073899  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:57.075006  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "7ec4352e7b9a4546891ba6c6c9464308"
format_stamp: "Formatted at 2026-07-09 12:23:57 on dist-test-slave-lg8f"
I20260709 12:23:57.079730  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:23:57.083464  5749 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.084359  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:23:57.084695  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "7ec4352e7b9a4546891ba6c6c9464308"
format_stamp: "Formatted at 2026-07-09 12:23:57 on dist-test-slave-lg8f"
I20260709 12:23:57.084965  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:57.106433  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:57.107733  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:57.109251  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:57.111550  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:57.111770  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.112046  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:57.112233  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.153858  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:43867
I20260709 12:23:57.154006  5811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:43867 every 8 connection(s)
I20260709 12:23:57.159219  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:57.166688  5816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:57.168329  5817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:57.171125  5137 server_base.cc:1061] running on GCE node
W20260709 12:23:57.172261  5819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:57.173458  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:57.173774  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:57.174016  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599837173991 us; error 0 us; skew 500 ppm
I20260709 12:23:57.174604  5812 heartbeater.cc:344] Connected to a master server at 127.5.4.126:36811
I20260709 12:23:57.174825  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:57.175014  5812 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:57.175961  5812 heartbeater.cc:507] Master 127.5.4.126:36811 requested a full tablet report, sending...
I20260709 12:23:57.177938  5137 webserver.cc:533] Webserver started at http://127.5.4.66:40833/ using document root <none> and password file <none>
I20260709 12:23:57.178532  5684 ts_manager.cc:194] Registered new tserver with Master: 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867)
I20260709 12:23:57.178735  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:57.179082  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:57.179392  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:57.180387  5684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51486
I20260709 12:23:57.180923  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
format_stamp: "Formatted at 2026-07-09 12:23:57 on dist-test-slave-lg8f"
I20260709 12:23:57.186049  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.002s
I20260709 12:23:57.189826  5824 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.190800  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:23:57.191129  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
format_stamp: "Formatted at 2026-07-09 12:23:57 on dist-test-slave-lg8f"
I20260709 12:23:57.191442  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:57.215654  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:57.217283  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:57.219118  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:57.221791  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:57.222064  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.222358  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:57.222586  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.001s
I20260709 12:23:57.271544  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:41313
I20260709 12:23:57.271668  5886 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:41313 every 8 connection(s)
I20260709 12:23:57.277213  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:57.285773  5890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:57.288079  5891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:57.289717  5893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:57.290369  5137 server_base.cc:1061] running on GCE node
I20260709 12:23:57.291455  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:23:57.291527  5887 heartbeater.cc:344] Connected to a master server at 127.5.4.126:36811
W20260709 12:23:57.291710  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:57.291949  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599837291930 us; error 0 us; skew 500 ppm
I20260709 12:23:57.292025  5887 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:57.292661  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:57.293053  5887 heartbeater.cc:507] Master 127.5.4.126:36811 requested a full tablet report, sending...
I20260709 12:23:57.295763  5684 ts_manager.cc:194] Registered new tserver with Master: 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:57.297230  5684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51500
I20260709 12:23:57.298023  5137 webserver.cc:533] Webserver started at http://127.5.4.67:45767/ using document root <none> and password file <none>
I20260709 12:23:57.298682  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:57.298905  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:57.299170  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:57.300329  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "a5947ff1044b47cc93c0083b4a1f939f"
format_stamp: "Formatted at 2026-07-09 12:23:57 on dist-test-slave-lg8f"
I20260709 12:23:57.304893  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.001s
I20260709 12:23:57.308348  5898 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.309172  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:23:57.309474  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "a5947ff1044b47cc93c0083b4a1f939f"
format_stamp: "Formatted at 2026-07-09 12:23:57 on dist-test-slave-lg8f"
I20260709 12:23:57.309764  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:57.339215  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:57.340705  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:57.342340  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:57.344619  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:57.344851  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.345090  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:57.345268  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:57.389871  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:44395
I20260709 12:23:57.390040  5961 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:44395 every 8 connection(s)
I20260709 12:23:57.406653  5962 heartbeater.cc:344] Connected to a master server at 127.5.4.126:36811
I20260709 12:23:57.406996  5962 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:57.407713  5962 heartbeater.cc:507] Master 127.5.4.126:36811 requested a full tablet report, sending...
I20260709 12:23:57.409626  5684 ts_manager.cc:194] Registered new tserver with Master: a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:57.409832  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015759835s
I20260709 12:23:57.411430  5684 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51514
I20260709 12:23:57.439352  5684 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51530:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:57.442389  5684 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:57.520332  5852 tablet_service.cc:1511] Processing CreateTablet for tablet ee09c423e2ce4d8c82a2a8528478f0cc (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:23:57.521865  5852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee09c423e2ce4d8c82a2a8528478f0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.522455  5925 tablet_service.cc:1511] Processing CreateTablet for tablet 8db795ca990d4664bcf6fa16bc3a28ab (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:23:57.523989  5925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db795ca990d4664bcf6fa16bc3a28ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.523674  5850 tablet_service.cc:1511] Processing CreateTablet for tablet e8bc2054813a4327977ecc742b0d68c7 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:23:57.525015  5850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bc2054813a4327977ecc742b0d68c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.525126  5848 tablet_service.cc:1511] Processing CreateTablet for tablet ee3687476c37453c8a5e1da7c3ca3dc2 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:23:57.525429  5923 tablet_service.cc:1511] Processing CreateTablet for tablet 5d3558278af94262bffa2f467a837472 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:23:57.526404  5848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee3687476c37453c8a5e1da7c3ca3dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.526858  5923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d3558278af94262bffa2f467a837472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.527688  5851 tablet_service.cc:1511] Processing CreateTablet for tablet 8db795ca990d4664bcf6fa16bc3a28ab (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:23:57.528991  5851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db795ca990d4664bcf6fa16bc3a28ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.520494  5926 tablet_service.cc:1511] Processing CreateTablet for tablet ee09c423e2ce4d8c82a2a8528478f0cc (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:23:57.522148  5849 tablet_service.cc:1511] Processing CreateTablet for tablet 5d3558278af94262bffa2f467a837472 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:23:57.531414  5926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee09c423e2ce4d8c82a2a8528478f0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.531834  5924 tablet_service.cc:1511] Processing CreateTablet for tablet e8bc2054813a4327977ecc742b0d68c7 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:23:57.533219  5924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bc2054813a4327977ecc742b0d68c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.539148  5849 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d3558278af94262bffa2f467a837472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.560977  5777 tablet_service.cc:1511] Processing CreateTablet for tablet ee09c423e2ce4d8c82a2a8528478f0cc (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:23:57.562937  5777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee09c423e2ce4d8c82a2a8528478f0cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.570459  5776 tablet_service.cc:1511] Processing CreateTablet for tablet 8db795ca990d4664bcf6fa16bc3a28ab (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:23:57.572067  5773 tablet_service.cc:1511] Processing CreateTablet for tablet ee3687476c37453c8a5e1da7c3ca3dc2 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:23:57.572177  5774 tablet_service.cc:1511] Processing CreateTablet for tablet 5d3558278af94262bffa2f467a837472 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:23:57.573565  5776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8db795ca990d4664bcf6fa16bc3a28ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.571884  5775 tablet_service.cc:1511] Processing CreateTablet for tablet e8bc2054813a4327977ecc742b0d68c7 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:23:57.575367  5774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5d3558278af94262bffa2f467a837472. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.576239  5775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8bc2054813a4327977ecc742b0d68c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.577006  5773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee3687476c37453c8a5e1da7c3ca3dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.588433  5772 tablet_service.cc:1511] Processing CreateTablet for tablet 2370d693fdc84419bac799cc5d782d6c (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:23:57.589946  5772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2370d693fdc84419bac799cc5d782d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.596532  5848 tablet_service.cc:1511] Processing CreateTablet for tablet 2370d693fdc84419bac799cc5d782d6c (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:23:57.597868  5848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2370d693fdc84419bac799cc5d782d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.599277  5977 tablet_bootstrap.cc:492] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:57.610730  5923 tablet_service.cc:1511] Processing CreateTablet for tablet ee3687476c37453c8a5e1da7c3ca3dc2 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:23:57.612844  5923 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee3687476c37453c8a5e1da7c3ca3dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.624735  5771 tablet_service.cc:1511] Processing CreateTablet for tablet 2f2f6d6183a847578f8d89ea3f12463b (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:23:57.628332  5771 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f2f6d6183a847578f8d89ea3f12463b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.635434  5850 tablet_service.cc:1511] Processing CreateTablet for tablet 2f2f6d6183a847578f8d89ea3f12463b (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:23:57.641448  5978 tablet_bootstrap.cc:492] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:57.646104  5977 tablet_bootstrap.cc:654] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.646414  5850 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f2f6d6183a847578f8d89ea3f12463b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.642083  5925 tablet_service.cc:1511] Processing CreateTablet for tablet 2370d693fdc84419bac799cc5d782d6c (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:23:57.651271  5772 tablet_service.cc:1511] Processing CreateTablet for tablet b4e9d92a4296478b93b88d7e31cfd0e3 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:23:57.652427  5925 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2370d693fdc84419bac799cc5d782d6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.652628  5772 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e9d92a4296478b93b88d7e31cfd0e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.654964  5851 tablet_service.cc:1511] Processing CreateTablet for tablet b4e9d92a4296478b93b88d7e31cfd0e3 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:23:57.656210  5851 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e9d92a4296478b93b88d7e31cfd0e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.651484  5924 tablet_service.cc:1511] Processing CreateTablet for tablet 2f2f6d6183a847578f8d89ea3f12463b (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:23:57.658921  5924 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f2f6d6183a847578f8d89ea3f12463b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.668365  5926 tablet_service.cc:1511] Processing CreateTablet for tablet b4e9d92a4296478b93b88d7e31cfd0e3 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:23:57.670013  5926 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4e9d92a4296478b93b88d7e31cfd0e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.679222  5852 tablet_service.cc:1511] Processing CreateTablet for tablet 146522f2e2c5416a915558edd9e040cd (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:23:57.680444  5852 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 146522f2e2c5416a915558edd9e040cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.690626  5922 tablet_service.cc:1511] Processing CreateTablet for tablet 146522f2e2c5416a915558edd9e040cd (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:23:57.691936  5922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 146522f2e2c5416a915558edd9e040cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.692361  5921 tablet_service.cc:1511] Processing CreateTablet for tablet 6a3050b37c2a4bc2883f94d8ea7e8be7 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:23:57.696967  5848 tablet_service.cc:1511] Processing CreateTablet for tablet 6a3050b37c2a4bc2883f94d8ea7e8be7 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:23:57.697414  5921 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a3050b37c2a4bc2883f94d8ea7e8be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.698304  5848 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a3050b37c2a4bc2883f94d8ea7e8be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.707705  5777 tablet_service.cc:1511] Processing CreateTablet for tablet 6a3050b37c2a4bc2883f94d8ea7e8be7 (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:23:57.711017  5979 tablet_bootstrap.cc:492] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:57.696590  5773 tablet_service.cc:1511] Processing CreateTablet for tablet 146522f2e2c5416a915558edd9e040cd (DEFAULT_TABLE table=test-workload [id=abb603b00a3043c688c5984d1fc068d1]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:23:57.709076  5777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a3050b37c2a4bc2883f94d8ea7e8be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.721508  5978 tablet_bootstrap.cc:654] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.727808  5773 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 146522f2e2c5416a915558edd9e040cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.743359  5979 tablet_bootstrap.cc:654] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.750927  5977 tablet_bootstrap.cc:492] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:57.751434  5977 ts_tablet_manager.cc:1403] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.152s	user 0.019s	sys 0.022s
I20260709 12:23:57.755720  5977 raft_consensus.cc:359] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.756747  5977 raft_consensus.cc:385] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.757154  5977 raft_consensus.cc:740] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.758193  5977 consensus_queue.cc:260] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.763628  5978 tablet_bootstrap.cc:492] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:57.764288  5978 ts_tablet_manager.cc:1403] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.123s	user 0.001s	sys 0.013s
I20260709 12:23:57.769438  5978 raft_consensus.cc:359] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.770166  5978 raft_consensus.cc:385] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.770577  5979 tablet_bootstrap.cc:492] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:57.771272  5979 ts_tablet_manager.cc:1403] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.014s
I20260709 12:23:57.770478  5978 raft_consensus.cc:740] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.772243  5977 ts_tablet_manager.cc:1434] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.005s
I20260709 12:23:57.772473  5978 consensus_queue.cc:260] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.774410  5962 heartbeater.cc:499] Master 127.5.4.126:36811 was elected leader, sending a full tablet report...
I20260709 12:23:57.774703  5979 raft_consensus.cc:359] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.775363  5887 heartbeater.cc:499] Master 127.5.4.126:36811 was elected leader, sending a full tablet report...
I20260709 12:23:57.775513  5979 raft_consensus.cc:385] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.775731  5978 ts_tablet_manager.cc:1434] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 12:23:57.775903  5979 raft_consensus.cc:740] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.776820  5979 consensus_queue.cc:260] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.777411  5978 tablet_bootstrap.cc:492] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:57.778581  5977 tablet_bootstrap.cc:492] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:57.785929  5977 tablet_bootstrap.cc:654] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.786360  5978 tablet_bootstrap.cc:654] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.797648  5979 ts_tablet_manager.cc:1434] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.015s
I20260709 12:23:57.798766  5812 heartbeater.cc:499] Master 127.5.4.126:36811 was elected leader, sending a full tablet report...
I20260709 12:23:57.799170  5979 tablet_bootstrap.cc:492] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:57.806216  5979 tablet_bootstrap.cc:654] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.811308  5978 tablet_bootstrap.cc:492] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:57.812157  5978 ts_tablet_manager.cc:1403] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.005s
I20260709 12:23:57.813848  5977 tablet_bootstrap.cc:492] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:57.814383  5977 ts_tablet_manager.cc:1403] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.018s
I20260709 12:23:57.816458  5978 raft_consensus.cc:359] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.817512  5978 raft_consensus.cc:385] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.817396  5977 raft_consensus.cc:359] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.818038  5978 raft_consensus.cc:740] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.818305  5977 raft_consensus.cc:385] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.818758  5977 raft_consensus.cc:740] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.819298  5978 consensus_queue.cc:260] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.819589  5977 consensus_queue.cc:260] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.822332  5977 ts_tablet_manager.cc:1434] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:57.823293  5977 tablet_bootstrap.cc:492] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:57.823549  5978 ts_tablet_manager.cc:1434] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:23:57.823776  5979 tablet_bootstrap.cc:492] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:57.824250  5979 ts_tablet_manager.cc:1403] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.025s	user 0.009s	sys 0.014s
I20260709 12:23:57.824553  5978 tablet_bootstrap.cc:492] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:57.827186  5979 raft_consensus.cc:359] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.827947  5979 raft_consensus.cc:385] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.828305  5979 raft_consensus.cc:740] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.829097  5979 consensus_queue.cc:260] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.829746  5977 tablet_bootstrap.cc:654] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.831401  5979 ts_tablet_manager.cc:1434] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:23:57.832229  5978 tablet_bootstrap.cc:654] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.832587  5979 tablet_bootstrap.cc:492] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:57.852545  5979 tablet_bootstrap.cc:654] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.859089  5977 tablet_bootstrap.cc:492] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:57.859586  5977 ts_tablet_manager.cc:1403] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.015s
I20260709 12:23:57.862371  5977 raft_consensus.cc:359] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.863134  5977 raft_consensus.cc:385] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.863418  5977 raft_consensus.cc:740] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.864017  5977 consensus_queue.cc:260] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.865685  5978 tablet_bootstrap.cc:492] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:57.866585  5978 ts_tablet_manager.cc:1403] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.042s	user 0.021s	sys 0.014s
I20260709 12:23:57.865851  5977 ts_tablet_manager.cc:1434] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:57.868052  5977 tablet_bootstrap.cc:492] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:57.868992  5979 tablet_bootstrap.cc:492] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:57.869580  5979 ts_tablet_manager.cc:1403] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.007s
I20260709 12:23:57.869613  5978 raft_consensus.cc:359] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.870478  5978 raft_consensus.cc:385] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.870814  5978 raft_consensus.cc:740] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.871790  5978 consensus_queue.cc:260] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.873139  5979 raft_consensus.cc:359] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.873909  5979 raft_consensus.cc:385] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.874255  5979 raft_consensus.cc:740] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.874419  5978 ts_tablet_manager.cc:1434] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:57.874727  5977 tablet_bootstrap.cc:654] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.875613  5978 tablet_bootstrap.cc:492] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:57.875365  5979 consensus_queue.cc:260] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.877640  5979 ts_tablet_manager.cc:1434] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.002s
I20260709 12:23:57.878675  5979 tablet_bootstrap.cc:492] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:57.885573  5979 tablet_bootstrap.cc:654] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.886237  5977 tablet_bootstrap.cc:492] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:57.886844  5977 ts_tablet_manager.cc:1403] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.011s
I20260709 12:23:57.889825  5977 raft_consensus.cc:359] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.890552  5977 raft_consensus.cc:385] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.890897  5977 raft_consensus.cc:740] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.891837  5977 consensus_queue.cc:260] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.895988  5977 ts_tablet_manager.cc:1434] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:23:57.896646  5979 tablet_bootstrap.cc:492] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:57.896970  5977 tablet_bootstrap.cc:492] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:57.897310  5978 tablet_bootstrap.cc:654] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.897248  5979 ts_tablet_manager.cc:1403] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.006s
I20260709 12:23:57.900899  5979 raft_consensus.cc:359] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.901540  5979 raft_consensus.cc:385] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.901860  5979 raft_consensus.cc:740] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.902802  5979 consensus_queue.cc:260] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.903410  5978 tablet_bootstrap.cc:492] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:57.905095  5979 ts_tablet_manager.cc:1434] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 12:23:57.906241  5977 tablet_bootstrap.cc:654] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.906901  5978 ts_tablet_manager.cc:1403] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 12:23:57.907760  5979 tablet_bootstrap.cc:492] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:57.912325  5978 raft_consensus.cc:359] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.913288  5978 raft_consensus.cc:385] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.913676  5978 raft_consensus.cc:740] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.915021  5979 tablet_bootstrap.cc:654] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.914757  5978 consensus_queue.cc:260] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.919902  5978 ts_tablet_manager.cc:1434] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:23:57.920814  5978 tablet_bootstrap.cc:492] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:57.920900  5977 tablet_bootstrap.cc:492] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:57.921104  5979 tablet_bootstrap.cc:492] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:57.921694  5979 ts_tablet_manager.cc:1403] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.006s
I20260709 12:23:57.921690  5977 ts_tablet_manager.cc:1403] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.010s
I20260709 12:23:57.924942  5977 raft_consensus.cc:359] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.925046  5979 raft_consensus.cc:359] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.925837  5979 raft_consensus.cc:385] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.926260  5979 raft_consensus.cc:740] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.927011  5979 consensus_queue.cc:260] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.927551  5978 tablet_bootstrap.cc:654] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.929299  5979 ts_tablet_manager.cc:1434] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 12:23:57.925800  5977 raft_consensus.cc:385] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.929767  5977 raft_consensus.cc:740] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.930727  5977 consensus_queue.cc:260] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.933064  5977 ts_tablet_manager.cc:1434] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 12:23:57.933415  5978 tablet_bootstrap.cc:492] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:57.933885  5978 ts_tablet_manager.cc:1403] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:23:57.934082  5977 tablet_bootstrap.cc:492] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:57.935001  5983 raft_consensus.cc:493] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.935475  5983 raft_consensus.cc:515] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.937974  5983 leader_election.cc:290] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:57.938016  5984 raft_consensus.cc:493] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.938580  5984 raft_consensus.cc:515] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.940266  5979 tablet_bootstrap.cc:492] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:57.940917  5984 leader_election.cc:290] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:57.941408  5977 tablet_bootstrap.cc:654] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.948717  5979 tablet_bootstrap.cc:654] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:57.956386  5979 tablet_bootstrap.cc:492] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:57.957175  5979 ts_tablet_manager.cc:1403] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 12:23:57.961325  5979 raft_consensus.cc:359] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.962438  5979 raft_consensus.cc:385] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.962918  5979 raft_consensus.cc:740] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.964118  5979 consensus_queue.cc:260] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.965175  5983 raft_consensus.cc:493] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.965715  5983 raft_consensus.cc:515] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.967036  5978 raft_consensus.cc:359] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.967752  5983 leader_election.cc:290] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:57.968793  5985 raft_consensus.cc:493] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.969244  5985 raft_consensus.cc:515] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:57.975473  5984 raft_consensus.cc:493] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.975973  5984 raft_consensus.cc:515] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:57.977105  5979 ts_tablet_manager.cc:1434] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:23:57.978478  5984 leader_election.cc:290] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395), 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867)
I20260709 12:23:57.979830  5985 leader_election.cc:290] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:57.986321  5979 tablet_bootstrap.cc:492] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
W20260709 12:23:57.988844  5733 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:57.967751  5978 raft_consensus.cc:385] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:57.993686  5978 raft_consensus.cc:740] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:57.998719  5985 raft_consensus.cc:493] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:57.999334  5985 raft_consensus.cc:515] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.001348  5985 leader_election.cc:290] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.003239  5984 raft_consensus.cc:493] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.004055  5984 raft_consensus.cc:515] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.005872  5787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.005679  5978 consensus_queue.cc:260] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.006644  5984 leader_election.cc:290] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.007421  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.008077  5786 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 0.
I20260709 12:23:58.010627  5979 tablet_bootstrap.cc:654] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.014627  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.015290  5786 raft_consensus.cc:2468] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 0.
I20260709 12:23:58.017704  5985 raft_consensus.cc:493] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.018258  5985 raft_consensus.cc:515] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.019166  5978 ts_tablet_manager.cc:1434] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.085s	user 0.014s	sys 0.001s
I20260709 12:23:58.020423  5985 leader_election.cc:290] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
W20260709 12:23:58.023207  5826 leader_election.cc:343] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.023958  5978 tablet_bootstrap.cc:492] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:58.023981  5826 leader_election.cc:304] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.025329  5826 leader_election.cc:304] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.026120  5984 raft_consensus.cc:2804] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.026544  5984 raft_consensus.cc:493] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.026914  5984 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.031076  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.031352  5787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.032012  5787 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 0.
I20260709 12:23:58.031960  5786 raft_consensus.cc:2468] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 0.
I20260709 12:23:58.033897  5900 leader_election.cc:304] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.036479  5978 tablet_bootstrap.cc:654] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.036922  5984 raft_consensus.cc:515] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:58.038978  5984 leader_election.cc:290] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Requested vote from peers a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395), 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867)
I20260709 12:23:58.039539  5984 raft_consensus.cc:2804] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.039657  5983 raft_consensus.cc:2804] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.039754  5900 leader_election.cc:304] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.040532  5983 raft_consensus.cc:493] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.040927  5983 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.039949  5984 raft_consensus.cc:493] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.044572  5984 raft_consensus.cc:3060] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.049691  5977 tablet_bootstrap.cc:492] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:58.051646  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.052066  5979 tablet_bootstrap.cc:492] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:58.062984  5979 ts_tablet_manager.cc:1403] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.026s
I20260709 12:23:58.063706  5934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.065762  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.066669  5933 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.050141  5983 raft_consensus.cc:515] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.069078  5983 leader_election.cc:290] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:58.069797  5983 raft_consensus.cc:2804] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.053020  5984 raft_consensus.cc:515] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.060263  5935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.061069  5786 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.053081  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.071745  5936 raft_consensus.cc:2468] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 0.
I20260709 12:23:58.053030  5977 ts_tablet_manager.cc:1403] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.119s	user 0.026s	sys 0.018s
I20260709 12:23:58.074431  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.070253  5983 raft_consensus.cc:493] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.075237  5860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.076243  5861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
W20260709 12:23:58.076300  5826 leader_election.cc:343] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.076936  5861 raft_consensus.cc:2468] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 0.
W20260709 12:23:58.077826  5751 leader_election.cc:343] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.078281  5826 leader_election.cc:304] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.078737  5751 leader_election.cc:304] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.079372  6009 raft_consensus.cc:2804] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.080545  6009 raft_consensus.cc:493] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.080681  5862 raft_consensus.cc:2393] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7ec4352e7b9a4546891ba6c6c9464308 in current term 1: Already voted for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in this term.
I20260709 12:23:58.080958  6009 raft_consensus.cc:3060] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.081501  5787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.081694  5984 leader_election.cc:290] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.082242  5787 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.080161  5985 raft_consensus.cc:2804] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.083101  5985 raft_consensus.cc:493] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.083425  5985 raft_consensus.cc:3060] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.085628  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.086349  5785 raft_consensus.cc:3060] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.089628  6009 raft_consensus.cc:515] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.090910  5985 raft_consensus.cc:515] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.091904  6009 leader_election.cc:290] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.093473  5977 raft_consensus.cc:359] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.093814  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:58.075747  5979 raft_consensus.cc:359] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.094439  5862 raft_consensus.cc:3060] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.094261  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.094959  5979 raft_consensus.cc:385] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.095363  5979 raft_consensus.cc:740] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.094272  5977 raft_consensus.cc:385] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.094328  5786 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.097185  5977 raft_consensus.cc:740] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.097232  5985 leader_election.cc:290] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
W20260709 12:23:58.099557  5826 leader_election.cc:343] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.099943  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.100744  5978 tablet_bootstrap.cc:492] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:58.100955  5936 raft_consensus.cc:3060] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.101277  5978 ts_tablet_manager.cc:1403] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.078s	user 0.010s	sys 0.025s
I20260709 12:23:58.102619  5933 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.103157  5862 raft_consensus.cc:2468] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 1.
I20260709 12:23:58.103972  5935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.104598  5751 leader_election.cc:304] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.090270  5787 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 1.
I20260709 12:23:58.100253  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.105055  5979 consensus_queue.cc:260] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.105592  5935 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.105962  5786 raft_consensus.cc:3060] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.107003  5934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.107283  5785 raft_consensus.cc:2468] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.107630  5934 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 0.
I20260709 12:23:58.079666  5983 raft_consensus.cc:3060] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.109131  5934 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
W20260709 12:23:58.110716  5751 leader_election.cc:343] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.111188  5751 leader_election.cc:304] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:23:58.104346  5978 raft_consensus.cc:359] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.111866  5978 raft_consensus.cc:385] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.112685  5978 raft_consensus.cc:740] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.107791  5826 leader_election.cc:304] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.113633  5985 raft_consensus.cc:2804] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.114624  6009 raft_consensus.cc:2804] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.114807  5826 leader_election.cc:304] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:23:58.115883  5984 raft_consensus.cc:2804] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.115821  5983 raft_consensus.cc:515] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.117326  5935 raft_consensus.cc:2393] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7ec4352e7b9a4546891ba6c6c9464308 in current term 1: Already voted for candidate a5947ff1044b47cc93c0083b4a1f939f in this term.
I20260709 12:23:58.116072  6009 raft_consensus.cc:697] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:58.118137  5983 leader_election.cc:290] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:58.111390  5936 raft_consensus.cc:2468] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.116169  5900 leader_election.cc:304] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.118954  6009 consensus_queue.cc:237] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:58.121054  5826 leader_election.cc:304] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.121613  5983 raft_consensus.cc:2804] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.122289  5983 raft_consensus.cc:697] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 LEADER]: Becoming Leader. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:23:58.123080  5861 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.119531  5977 consensus_queue.cc:260] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.122018  5978 consensus_queue.cc:260] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.124290  5860 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:58.125479  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:58.123366  5983 consensus_queue.cc:237] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.126315  5859 raft_consensus.cc:2393] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a5947ff1044b47cc93c0083b4a1f939f in current term 1: Already voted for candidate 7ec4352e7b9a4546891ba6c6c9464308 in this term.
I20260709 12:23:58.128079  5786 raft_consensus.cc:2468] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.128466  6013 raft_consensus.cc:2749] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:58.130620  6016 raft_consensus.cc:2804] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.136030  5977 ts_tablet_manager.cc:1434] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.060s	user 0.005s	sys 0.004s
I20260709 12:23:58.138312  5751 leader_election.cc:304] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2
I20260709 12:23:58.131173  6016 raft_consensus.cc:697] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
W20260709 12:23:58.141031  5900 leader_election.cc:343] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:58.144431  5900 leader_election.cc:343] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.146450  5984 raft_consensus.cc:697] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:58.132720  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.147843  5984 consensus_queue.cc:237] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.148667  5862 raft_consensus.cc:2393] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5947ff1044b47cc93c0083b4a1f939f in current term 1: Already voted for candidate 7ec4352e7b9a4546891ba6c6c9464308 in this term.
I20260709 12:23:58.152208  5977 tablet_bootstrap.cc:492] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:58.151357  5985 raft_consensus.cc:697] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:58.154116  6013 raft_consensus.cc:493] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.150972  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8db795ca990d4664bcf6fa16bc3a28ab" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.156898  5985 consensus_queue.cc:237] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.156581  6013 raft_consensus.cc:515] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.167574  5978 ts_tablet_manager.cc:1434] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.066s	user 0.006s	sys 0.014s
I20260709 12:23:58.167999  5900 leader_election.cc:304] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5947ff1044b47cc93c0083b4a1f939f; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.170189  6017 raft_consensus.cc:2749] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:58.160156  6016 consensus_queue.cc:237] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.170272  5978 tablet_bootstrap.cc:492] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:58.178943  6013 leader_election.cc:290] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.179840  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.180545  5862 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 0.
I20260709 12:23:58.182410  5978 tablet_bootstrap.cc:654] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.184484  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.182598  5751 leader_election.cc:304] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.185889  6020 raft_consensus.cc:2804] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:58.186076  5751 leader_election.cc:343] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.186458  6020 raft_consensus.cc:493] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.187017  6020 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.194870  5977 tablet_bootstrap.cc:654] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.201242  6014 raft_consensus.cc:2804] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.201810  6014 raft_consensus.cc:493] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.202245  6014 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.216892  5979 ts_tablet_manager.cc:1434] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.153s	user 0.035s	sys 0.039s
I20260709 12:23:58.229369  5979 tablet_bootstrap.cc:492] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:58.231221  5684 catalog_manager.cc:5697] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f reported cstate change: term changed from 0 to 1, leader changed from <none> to a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67). New cstate: current_term: 1 leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:58.235857  5683 catalog_manager.cc:5697] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.241305  5979 tablet_bootstrap.cc:654] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.276300  5977 tablet_bootstrap.cc:492] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:58.277002  5977 ts_tablet_manager.cc:1403] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.125s	user 0.017s	sys 0.028s
I20260709 12:23:58.280105  5983 raft_consensus.cc:493] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.280614  5983 raft_consensus.cc:515] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.281633  6014 raft_consensus.cc:515] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.287849  5681 catalog_manager.cc:5697] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.292790  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.293035  5983 leader_election.cc:290] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
W20260709 12:23:58.296115  5900 leader_election.cc:343] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.296746  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.297214  5977 raft_consensus.cc:359] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.297863  5977 raft_consensus.cc:385] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.298188  5977 raft_consensus.cc:740] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
W20260709 12:23:58.298430  5900 leader_election.cc:343] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.298936  5900 leader_election.cc:304] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5947ff1044b47cc93c0083b4a1f939f; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.300093  5983 raft_consensus.cc:2749] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:58.301549  5978 tablet_bootstrap.cc:492] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:58.301961  5978 ts_tablet_manager.cc:1403] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.133s	user 0.025s	sys 0.028s
I20260709 12:23:58.290429  6020 raft_consensus.cc:515] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.305703  5979 tablet_bootstrap.cc:492] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:58.305359  5978 raft_consensus.cc:359] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.306216  5979 ts_tablet_manager.cc:1403] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.010s
I20260709 12:23:58.306216  6020 leader_election.cc:290] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.306386  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:58.307296  5862 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.308372  5979 raft_consensus.cc:359] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.308825  5979 raft_consensus.cc:385] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.308980  5979 raft_consensus.cc:740] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.309432  5979 consensus_queue.cc:260] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.306223  5978 raft_consensus.cc:385] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.310806  5978 raft_consensus.cc:740] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.311518  5979 ts_tablet_manager.cc:1434] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:23:58.311904  5978 consensus_queue.cc:260] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.312642  5979 tablet_bootstrap.cc:492] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:58.312197  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:58.313668  5859 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.313771  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
W20260709 12:23:58.315372  5751 leader_election.cc:343] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.317399  5978 ts_tablet_manager.cc:1434] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.005s
I20260709 12:23:58.318372  5978 tablet_bootstrap.cc:492] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:58.318822  5979 tablet_bootstrap.cc:654] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.325711  5978 tablet_bootstrap.cc:654] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.329869  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.330611  5936 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.337170  5936 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 1.
I20260709 12:23:58.338522  5978 tablet_bootstrap.cc:492] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:58.338397  5751 leader_election.cc:304] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.339524  6020 raft_consensus.cc:2804] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.340361  6020 raft_consensus.cc:697] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:58.304119  5977 consensus_queue.cc:260] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.341636  6020 consensus_queue.cc:237] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.344440  5977 ts_tablet_manager.cc:1434] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.067s	user 0.008s	sys 0.000s
I20260709 12:23:58.345513  5977 tablet_bootstrap.cc:492] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:58.340634  6014 leader_election.cc:290] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.348554  5859 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 1.
I20260709 12:23:58.340466  5862 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 1.
I20260709 12:23:58.352957  5978 ts_tablet_manager.cc:1403] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.004s
I20260709 12:23:58.361943  5983 raft_consensus.cc:493] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.363502  5978 raft_consensus.cc:359] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.364444  5978 raft_consensus.cc:385] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.364333  5751 leader_election.cc:304] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:23:58.365074  5978 raft_consensus.cc:740] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.366127  5983 raft_consensus.cc:515] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.366379  5978 consensus_queue.cc:260] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.368479  6016 raft_consensus.cc:493] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.369066  6016 raft_consensus.cc:515] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.372049  5983 leader_election.cc:290] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:58.372323  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.372975  6014 raft_consensus.cc:2804] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.373421  5786 raft_consensus.cc:2393] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5947ff1044b47cc93c0083b4a1f939f in current term 1: Already voted for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in this term.
I20260709 12:23:58.374842  6014 raft_consensus.cc:697] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:58.375484  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.374182  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.376322  5786 raft_consensus.cc:2468] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 0.
I20260709 12:23:58.377013  6014 consensus_queue.cc:237] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.377836  5826 leader_election.cc:304] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.384370  5978 ts_tablet_manager.cc:1434] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:23:58.383477  5984 raft_consensus.cc:2804] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.385314  5978 tablet_bootstrap.cc:492] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:58.385260  5984 raft_consensus.cc:493] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.385870  5984 raft_consensus.cc:3060] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.385958  5900 leader_election.cc:304] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5947ff1044b47cc93c0083b4a1f939f; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.387321  5983 raft_consensus.cc:3060] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.388990  5979 tablet_bootstrap.cc:492] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:58.390033  5979 ts_tablet_manager.cc:1403] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.078s	user 0.027s	sys 0.040s
I20260709 12:23:58.393193  5979 raft_consensus.cc:359] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.394376  5979 raft_consensus.cc:385] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.395231  5979 raft_consensus.cc:740] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.396335  5979 consensus_queue.cc:260] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.397670  5684 catalog_manager.cc:5697] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.400105  5979 ts_tablet_manager.cc:1434] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:23:58.400029  5983 raft_consensus.cc:2749] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:58.399607  5684 catalog_manager.cc:5697] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.403687  5979 tablet_bootstrap.cc:492] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: Bootstrap starting.
I20260709 12:23:58.403941  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.401803  6016 leader_election.cc:290] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
W20260709 12:23:58.406227  5826 leader_election.cc:343] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.410456  5978 tablet_bootstrap.cc:654] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.415505  5979 tablet_bootstrap.cc:654] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.415982  5984 raft_consensus.cc:515] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.425470  5977 tablet_bootstrap.cc:654] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.427312  5979 tablet_bootstrap.cc:492] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: No bootstrap required, opened a new log
I20260709 12:23:58.427830  5979 ts_tablet_manager.cc:1403] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.001s
I20260709 12:23:58.428012  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.428580  5786 raft_consensus.cc:3060] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.433677  5984 leader_election.cc:290] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.433683  5681 catalog_manager.cc:5697] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.435110  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.436798  5786 raft_consensus.cc:2468] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
W20260709 12:23:58.436925  5826 leader_election.cc:343] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:58.438222  5826 leader_election.cc:304] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:23:58.439002  5984 raft_consensus.cc:2804] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.439531  5984 raft_consensus.cc:697] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:58.440285  5984 consensus_queue.cc:237] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.444128  5979 raft_consensus.cc:359] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.445250  5979 raft_consensus.cc:385] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.445611  5979 raft_consensus.cc:740] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.447204  5979 consensus_queue.cc:260] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.448966  5978 tablet_bootstrap.cc:492] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:58.449409  5978 ts_tablet_manager.cc:1403] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.015s
I20260709 12:23:58.449460  5979 ts_tablet_manager.cc:1434] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:23:58.451787  5977 tablet_bootstrap.cc:492] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:58.452661  5977 ts_tablet_manager.cc:1403] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.107s	user 0.017s	sys 0.008s
I20260709 12:23:58.451957  5978 raft_consensus.cc:359] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.453014  5978 raft_consensus.cc:385] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.453287  5978 raft_consensus.cc:740] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.454450  5681 catalog_manager.cc:5697] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.456245  5977 raft_consensus.cc:359] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.457180  5977 raft_consensus.cc:385] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.457460  5977 raft_consensus.cc:740] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.457415  5978 consensus_queue.cc:260] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.458256  5977 consensus_queue.cc:260] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.460137  5977 ts_tablet_manager.cc:1434] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:23:58.460142  5978 ts_tablet_manager.cc:1434] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.007s
I20260709 12:23:58.461390  5978 tablet_bootstrap.cc:492] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2: Bootstrap starting.
I20260709 12:23:58.462685  5977 tablet_bootstrap.cc:492] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:58.467344  5978 tablet_bootstrap.cc:654] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.469587  5977 tablet_bootstrap.cc:654] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.480703  5978 tablet_bootstrap.cc:492] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2: No bootstrap required, opened a new log
I20260709 12:23:58.481688  5978 ts_tablet_manager.cc:1403] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 12:23:58.482306  5977 tablet_bootstrap.cc:492] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:58.482882  5977 ts_tablet_manager.cc:1403] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.006s
I20260709 12:23:58.482517  5683 catalog_manager.cc:5697] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.484236  5978 raft_consensus.cc:359] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.484860  5978 raft_consensus.cc:385] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.485121  5978 raft_consensus.cc:740] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.485026  5977 raft_consensus.cc:359] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.485710  5977 raft_consensus.cc:385] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.486030  5977 raft_consensus.cc:740] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.485898  5978 consensus_queue.cc:260] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.486904  5977 consensus_queue.cc:260] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.488023  5978 ts_tablet_manager.cc:1434] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:58.489199  5977 ts_tablet_manager.cc:1434] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:58.490237  5977 tablet_bootstrap.cc:492] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f: Bootstrap starting.
I20260709 12:23:58.495746  5977 tablet_bootstrap.cc:654] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:58.500698  5977 tablet_bootstrap.cc:492] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f: No bootstrap required, opened a new log
I20260709 12:23:58.501149  5977 ts_tablet_manager.cc:1403] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f: Time spent bootstrapping tablet: real 0.011s	user 0.002s	sys 0.007s
I20260709 12:23:58.503089  5977 raft_consensus.cc:359] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.503641  5977 raft_consensus.cc:385] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:58.503824  5977 raft_consensus.cc:740] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Initialized, Role: FOLLOWER
I20260709 12:23:58.504333  5977 consensus_queue.cc:260] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.505877  5977 ts_tablet_manager.cc:1434] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:23:58.533156  5983 raft_consensus.cc:493] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.533622  5983 raft_consensus.cc:515] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.535214  5983 leader_election.cc:290] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:58.536036  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.536471  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.536639  5786 raft_consensus.cc:2393] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5947ff1044b47cc93c0083b4a1f939f in current term 1: Already voted for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in this term.
I20260709 12:23:58.537984  5900 leader_election.cc:304] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5947ff1044b47cc93c0083b4a1f939f; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.538647  5983 raft_consensus.cc:3060] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.543488  5983 raft_consensus.cc:2749] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:58.555349  5859 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.561237  5983 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.573120  6011 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:58.579607  6014 consensus_queue.cc:1048] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:58.592250  5985 consensus_queue.cc:1048] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.599021  6020 raft_consensus.cc:493] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.599530  6020 raft_consensus.cc:515] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.601608  6020 leader_election.cc:290] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.603045  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.603772  5859 raft_consensus.cc:2468] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 0.
I20260709 12:23:58.604722  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.605361  5933 raft_consensus.cc:2468] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 0.
I20260709 12:23:58.606238  5751 leader_election.cc:304] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.606937  6020 raft_consensus.cc:2804] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.607295  6020 raft_consensus.cc:493] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.607620  6020 raft_consensus.cc:3060] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.609025  5984 consensus_queue.cc:1048] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.613735  6020 raft_consensus.cc:515] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.616644  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:58.617231  5859 raft_consensus.cc:3060] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.617223  5984 consensus_queue.cc:1048] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:58.617460  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.618158  5933 raft_consensus.cc:3060] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.617460  6020 leader_election.cc:290] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.625391  5933 raft_consensus.cc:2468] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 1.
I20260709 12:23:58.626555  5751 leader_election.cc:304] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.627367  6020 raft_consensus.cc:2804] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.627805  6020 raft_consensus.cc:697] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:58.628556  6020 consensus_queue.cc:237] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.629118  5859 raft_consensus.cc:2468] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 1.
I20260709 12:23:58.640183  5684 catalog_manager.cc:5697] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.693128  6011 raft_consensus.cc:493] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.693194  6009 raft_consensus.cc:493] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:58.693745  6011 raft_consensus.cc:515] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.693822  6009 raft_consensus.cc:515] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.695705  6011 leader_election.cc:290] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:58.695961  6009 leader_election.cc:290] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.696808  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.696856  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" is_pre_election: true
I20260709 12:23:58.696884  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308" is_pre_election: true
I20260709 12:23:58.697598  5786 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 0.
I20260709 12:23:58.697842  5859 raft_consensus.cc:2393] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5947ff1044b47cc93c0083b4a1f939f in current term 1: Already voted for candidate 7ec4352e7b9a4546891ba6c6c9464308 in this term.
I20260709 12:23:58.697803  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f" is_pre_election: true
I20260709 12:23:58.698874  5826 leader_election.cc:304] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:58.699091  5900 leader_election.cc:304] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5947ff1044b47cc93c0083b4a1f939f; no voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.699437  5933 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 0.
I20260709 12:23:58.699585  6009 raft_consensus.cc:2804] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:58.699968  6011 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.700141  6009 raft_consensus.cc:493] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:58.700549  6009 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.705878  6016 consensus_queue.cc:1048] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.707685  6011 raft_consensus.cc:2749] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:58.707685  6009 raft_consensus.cc:515] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.709851  6009 leader_election.cc:290] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:58.710605  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:58.711251  5786 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.711608  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:58.712193  5936 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:58.718384  6016 consensus_queue.cc:1048] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.719339  5936 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.723904  5826 leader_election.cc:304] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:58.724321  5786 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 1.
I20260709 12:23:58.724962  6009 raft_consensus.cc:2804] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:58.725615  6009 raft_consensus.cc:697] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:58.726790  6009 consensus_queue.cc:237] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:58.733142  6016 consensus_queue.cc:1048] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.737251  5684 catalog_manager.cc:5697] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:58.750581  6009 consensus_queue.cc:1048] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.799695  6020 consensus_queue.cc:1048] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.813778  6020 consensus_queue.cc:1048] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.814528  5985 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.827909  6020 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.938222  6016 consensus_queue.cc:1048] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:58.950278  6016 consensus_queue.cc:1048] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:58.972826  5859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:47446
I20260709 12:23:58.973449  5859 raft_consensus.cc:606] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:58.981748  5859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:47446
I20260709 12:23:58.982374  5859 raft_consensus.cc:606] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:23:59.007632  5936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
mode: GRACEFUL
new_leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:40530
I20260709 12:23:59.008157  5936 raft_consensus.cc:606] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 LEADER]: Received request to transfer leadership to TS 1269ff02f4d64f5a960ddc33f4abb6c2
I20260709 12:23:59.012490  6020 consensus_queue.cc:1048] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:59.025496  5985 consensus_queue.cc:1048] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:59.028744  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:53846
I20260709 12:23:59.029434  5785 raft_consensus.cc:606] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Received request to transfer leadership to TS 1269ff02f4d64f5a960ddc33f4abb6c2
I20260709 12:23:59.036610  6014 raft_consensus.cc:993] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:23:59.037112  6014 raft_consensus.cc:1081] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Signalling peer 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:23:59.036926  5859 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
mode: GRACEFUL
new_leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:47446
I20260709 12:23:59.037806  5859 raft_consensus.cc:606] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Received request to transfer leadership to TS a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:23:59.039558  5859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:23:59.040109  5859 raft_consensus.cc:493] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.040475  5859 raft_consensus.cc:3060] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.045203  5859 raft_consensus.cc:515] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.047289  5859 leader_election.cc:290] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 2 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:59.048166  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:59.048476  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "146522f2e2c5416a915558edd9e040cd" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:59.048851  5785 raft_consensus.cc:3055] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.049098  5936 raft_consensus.cc:3060] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.049283  5785 raft_consensus.cc:740] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:59.050248  5785 consensus_queue.cc:260] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.051715  5785 raft_consensus.cc:3060] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.055223  5936 raft_consensus.cc:2468] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 2.
I20260709 12:23:59.056279  5826 leader_election.cc:304] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:59.057111  5984 raft_consensus.cc:2804] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.057570  5984 raft_consensus.cc:697] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:59.057889  5785 raft_consensus.cc:2468] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 2.
I20260709 12:23:59.058703  5984 consensus_queue.cc:237] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.065196  5681 catalog_manager.cc:5697] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 1 to 2, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.121266  5983 raft_consensus.cc:993] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: : Instructing follower 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:23:59.121739  6072 raft_consensus.cc:1081] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 LEADER]: Signalling peer 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:23:59.123297  5859 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:47440
I20260709 12:23:59.123785  5859 raft_consensus.cc:493] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.124078  5859 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.129264  5859 raft_consensus.cc:515] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.130883  5859 leader_election.cc:290] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 2 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:59.131932  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:59.132190  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:59.132710  5785 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.132721  5936 raft_consensus.cc:3055] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.133275  5936 raft_consensus.cc:740] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:23:59.134179  5936 consensus_queue.cc:260] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.135514  5936 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.140638  5785 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 2.
I20260709 12:23:59.142139  5826 leader_election.cc:304] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:23:59.142479  5936 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 2.
I20260709 12:23:59.142923  5984 raft_consensus.cc:2804] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.143391  5984 raft_consensus.cc:697] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:59.144191  5984 consensus_queue.cc:237] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.150329  5681 catalog_manager.cc:5697] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 1 to 2, leader changed from a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67) to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.211964  6016 consensus_queue.cc:1048] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:59.224253  6016 consensus_queue.cc:1048] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:59.264837  6018 raft_consensus.cc:993] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:23:59.265378  6016 raft_consensus.cc:1081] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:23:59.267071  5785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53840
I20260709 12:23:59.267630  5785 raft_consensus.cc:493] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.267959  5785 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.273202  5785 raft_consensus.cc:515] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:59.275310  5785 leader_election.cc:290] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Requested vote from peers a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:59.276405  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:59.277087  6018 raft_consensus.cc:993] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:23:59.277060  5933 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.277140  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:59.277730  6018 raft_consensus.cc:1081] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:23:59.277882  5859 raft_consensus.cc:3055] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.278224  5859 raft_consensus.cc:740] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:59.279126  5859 consensus_queue.cc:260] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:59.279975  5785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53840
I20260709 12:23:59.280489  5785 raft_consensus.cc:493] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.280433  5859 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.281039  5785 raft_consensus.cc:3060] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.283443  5933 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:23:59.284698  5751 leader_election.cc:304] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:59.285524  6019 raft_consensus.cc:2804] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.286023  6019 raft_consensus.cc:697] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:59.286929  6019 consensus_queue.cc:237] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:23:59.288295  5785 raft_consensus.cc:515] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.290421  5785 leader_election.cc:290] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:23:59.291427  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:59.291487  5859 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:23:59.292239  5862 raft_consensus.cc:3055] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.292343  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8bc2054813a4327977ecc742b0d68c7" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:23:59.292629  5862 raft_consensus.cc:740] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:59.292991  5933 raft_consensus.cc:3060] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.293479  5862 consensus_queue.cc:260] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.294654  5862 raft_consensus.cc:3060] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.299034  5684 catalog_manager.cc:5697] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 1 to 2, leader changed from 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.299942  5933 raft_consensus.cc:2468] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:23:59.301021  5751 leader_election.cc:304] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:59.301131  5862 raft_consensus.cc:2468] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:23:59.301829  6019 raft_consensus.cc:2804] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.302402  6019 raft_consensus.cc:697] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:23:59.303277  6019 consensus_queue.cc:237] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.312420  5684 catalog_manager.cc:5697] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 1 to 2, leader changed from 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.350131  5984 raft_consensus.cc:993] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: : Instructing follower a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:23:59.350684  6016 raft_consensus.cc:1081] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Signalling peer a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:23:59.352341  5933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:40512
I20260709 12:23:59.352779  5933 raft_consensus.cc:493] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:59.353083  5933 raft_consensus.cc:3060] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.357623  5933 raft_consensus.cc:515] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.359157  5933 leader_election.cc:290] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 2 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:23:59.360140  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:23:59.360378  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a3050b37c2a4bc2883f94d8ea7e8be7" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:23:59.360882  5785 raft_consensus.cc:3060] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.360883  5862 raft_consensus.cc:3055] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.361508  5862 raft_consensus.cc:740] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:23:59.362427  5862 consensus_queue.cc:260] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.363785  5862 raft_consensus.cc:3060] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.369258  5785 raft_consensus.cc:2468] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 2.
I20260709 12:23:59.370673  5900 leader_election.cc:304] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:23:59.371304  5862 raft_consensus.cc:2468] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 2.
I20260709 12:23:59.371640  6072 raft_consensus.cc:2804] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:59.372418  6072 raft_consensus.cc:697] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Becoming Leader. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:23:59.373184  6072 consensus_queue.cc:237] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:23:59.379840  5684 catalog_manager.cc:5697] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f reported cstate change: term changed from 1 to 2, leader changed from 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66) to a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67). New cstate: current_term: 2 leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:59.475061  5785 raft_consensus.cc:1275] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.476418  6016 consensus_queue.cc:1048] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.485105  5933 raft_consensus.cc:1275] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.486675  6018 consensus_queue.cc:1048] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.572348  5933 raft_consensus.cc:1275] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.573985  6018 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.586179  5785 raft_consensus.cc:1275] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.588013  5984 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:59.683136  5862 raft_consensus.cc:1275] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.684964  6019 consensus_queue.cc:1048] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.693485  5933 raft_consensus.cc:1275] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.695392  6014 consensus_queue.cc:1048] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.741511  5933 raft_consensus.cc:1275] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.743284  6014 consensus_queue.cc:1048] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.752442  5862 raft_consensus.cc:1275] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.754271  6013 consensus_queue.cc:1048] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.779390  5785 raft_consensus.cc:1275] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.780869  6072 consensus_queue.cc:1048] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:59.789260  5862 raft_consensus.cc:1275] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:59.791374  5983 consensus_queue.cc:1048] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.069243  5137 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:24:02.069943  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:02.086403  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d3558278af94262bffa2f467a837472"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:53848
I20260709 12:24:02.087055  5785 raft_consensus.cc:606] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Received request to transfer leadership to TS a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:24:02.089792  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:53848
I20260709 12:24:02.090309  5785 raft_consensus.cc:606] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Received request to transfer leadership to TS a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:24:02.091324  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:24:02.091827  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:02.413828  6086 raft_consensus.cc:993] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:02.414661  6085 raft_consensus.cc:1081] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Signalling peer a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:02.416366  5936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d3558278af94262bffa2f467a837472"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:40526
I20260709 12:24:02.416918  5936 raft_consensus.cc:493] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.417323  5936 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.417380  6087 raft_consensus.cc:993] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:02.417934  6087 raft_consensus.cc:1081] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Signalling peer a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:02.419471  5933 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:40526
I20260709 12:24:02.420194  5933 raft_consensus.cc:493] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:02.420648  5933 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.422405  5936 raft_consensus.cc:515] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:02.424117  5936 leader_election.cc:290] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 2 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:24:02.425037  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:24:02.425482  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:02.425617  5785 raft_consensus.cc:3055] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.426061  5862 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.426388  5785 raft_consensus.cc:740] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:02.427460  5933 raft_consensus.cc:515] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:02.427258  5785 consensus_queue.cc:260] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:02.428696  5785 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.429602  5933 leader_election.cc:290] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 2 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:24:02.430681  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:24:02.430814  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:02.431354  5859 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.431305  5786 raft_consensus.cc:3055] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:02.432307  5786 raft_consensus.cc:740] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:02.432739  5862 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 2.
I20260709 12:24:02.433189  5786 consensus_queue.cc:260] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:02.434257  5786 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:02.434113  5900 leader_election.cc:304] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:02.435108  6082 raft_consensus.cc:2804] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.436539  6082 raft_consensus.cc:697] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Becoming Leader. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:24:02.438010  5859 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 2.
I20260709 12:24:02.438459  6082 consensus_queue.cc:237] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:02.439152  5900 leader_election.cc:304] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:02.440016  6088 raft_consensus.cc:2804] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:02.445348  6088 raft_consensus.cc:697] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Becoming Leader. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:24:02.445607  5785 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 2.
I20260709 12:24:02.446192  6088 consensus_queue.cc:237] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:02.450073  5786 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 2.
I20260709 12:24:02.463130  5684 catalog_manager.cc:5697] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f reported cstate change: term changed from 1 to 2, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67). New cstate: current_term: 2 leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:02.464404  5684 catalog_manager.cc:5697] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f reported cstate change: term changed from 1 to 2, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67). New cstate: current_term: 2 leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:02.871071  5786 raft_consensus.cc:1275] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:02.872892  6088 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.883575  5859 raft_consensus.cc:1275] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:02.885286  6088 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.924881  5859 raft_consensus.cc:1275] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:02.926167  6088 consensus_queue.cc:1048] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:02.935488  5786 raft_consensus.cc:1275] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:02.937506  6082 consensus_queue.cc:1048] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:03.092959  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:03.098675  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:03.099203  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:04.100144  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:04.105816  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:04.106312  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:05.107251  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:05.113442  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:05.113884  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:06.115152  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:06.121145  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:06.121595  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:07.122839  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:07.128489  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:07.128901  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:08.129942  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:08.135363  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:08.135756  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:09.136665  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:09.142136  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:09.142624  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:10.143568  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:10.149222  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:10.149621  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:11.150792  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:11.156876  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:11.157450  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:12.189167  5137 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:24:12.202585  5936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5d3558278af94262bffa2f467a837472"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:40530
I20260709 12:24:12.203413  5936 raft_consensus.cc:606] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:24:12.208706  5936 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:40530
I20260709 12:24:12.209247  5936 raft_consensus.cc:606] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:24:12.214355  5862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:47446
I20260709 12:24:12.214915  5862 raft_consensus.cc:606] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:24:12.219741  5862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:47446
I20260709 12:24:12.220247  5862 raft_consensus.cc:606] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:24:12.224727  5862 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
mode: GRACEFUL
new_leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:47446
I20260709 12:24:12.225194  5862 raft_consensus.cc:606] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Received request to transfer leadership to TS 7ec4352e7b9a4546891ba6c6c9464308
I20260709 12:24:12.436537  6111 raft_consensus.cc:993] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.437203  6114 raft_consensus.cc:1081] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.439322  5785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53828
I20260709 12:24:12.439930  5785 raft_consensus.cc:493] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:12.440393  5785 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.447254  5785 raft_consensus.cc:515] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.449108  5785 leader_election.cc:290] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 3 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:12.449914  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:12.450161  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:12.450837  5862 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.450922  5936 raft_consensus.cc:3055] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.451294  5936 raft_consensus.cc:740] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:24:12.452026  5936 consensus_queue.cc:260] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.453038  5936 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.456238  5862 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 3.
I20260709 12:24:12.457499  5751 leader_election.cc:304] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:24:12.458392  6113 raft_consensus.cc:2804] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:12.458626  5936 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 3.
I20260709 12:24:12.459625  6113 raft_consensus.cc:697] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:12.460444  6113 consensus_queue.cc:237] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.467254  5684 catalog_manager.cc:5697] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 2 to 3, leader changed from a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.646231  6112 raft_consensus.cc:993] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.646795  6096 raft_consensus.cc:1081] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.648141  5785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53840
I20260709 12:24:12.648690  5785 raft_consensus.cc:493] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:12.649022  5785 raft_consensus.cc:3060] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.653811  5785 raft_consensus.cc:515] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.655370  5785 leader_election.cc:290] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:12.656129  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:12.656450  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2370d693fdc84419bac799cc5d782d6c" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:12.656665  5862 raft_consensus.cc:3055] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:12.656922  5862 raft_consensus.cc:740] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:24:12.656955  5936 raft_consensus.cc:3060] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.657666  5862 consensus_queue.cc:260] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.658648  5862 raft_consensus.cc:3060] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.661412  5936 raft_consensus.cc:2468] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:24:12.662438  5751 leader_election.cc:304] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:12.663174  6113 raft_consensus.cc:2804] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:12.663204  5862 raft_consensus.cc:2468] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:24:12.663867  6113 raft_consensus.cc:697] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:12.664825  6113 consensus_queue.cc:237] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.667583  6112 raft_consensus.cc:993] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.668146  6096 raft_consensus.cc:1081] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.669700  5785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53840
I20260709 12:24:12.670285  5785 raft_consensus.cc:493] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:12.670598  5785 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.672922  5684 catalog_manager.cc:5697] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 1 to 2, leader changed from 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.674721  5785 raft_consensus.cc:515] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.676049  5785 leader_election.cc:290] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 3 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:12.676981  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:12.677313  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:12.677922  5936 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.677920  5862 raft_consensus.cc:3055] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.679086  5862 raft_consensus.cc:740] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:24:12.679916  5862 consensus_queue.cc:260] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.681321  5862 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.684597  6111 raft_consensus.cc:993] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.685061  6111 raft_consensus.cc:1081] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.685182  5936 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 3.
I20260709 12:24:12.686295  5751 leader_election.cc:304] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:12.686553  6096 raft_consensus.cc:993] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: : Instructing follower 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.686997  6112 raft_consensus.cc:1081] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Signalling peer 7ec4352e7b9a4546891ba6c6c9464308 to start an election
I20260709 12:24:12.687198  6113 raft_consensus.cc:2804] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:12.687706  6113 raft_consensus.cc:697] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:12.688640  6113 consensus_queue.cc:237] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.689245  5786 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53840
I20260709 12:24:12.689806  5786 raft_consensus.cc:493] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:12.690194  5786 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.690528  5862 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 3.
I20260709 12:24:12.691731  5785 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5d3558278af94262bffa2f467a837472"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
 from {username='slave'} at 127.0.0.1:53828
I20260709 12:24:12.692389  5785 raft_consensus.cc:493] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:12.693759  5785 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.697340  5786 raft_consensus.cc:515] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.696976  5684 catalog_manager.cc:5697] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 2 to 3, leader changed from 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.699097  5786 leader_election.cc:290] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:12.699945  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:12.700141  5785 raft_consensus.cc:515] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.701419  5936 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.700924  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:12.703037  5862 raft_consensus.cc:3055] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:12.703419  5862 raft_consensus.cc:740] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:24:12.703599  5859 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:12.704288  5859 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.704496  5862 consensus_queue.cc:260] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.705832  5862 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.710083  5936 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:24:12.710937  5785 leader_election.cc:290] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 3 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:12.711575  5751 leader_election.cc:304] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7ec4352e7b9a4546891ba6c6c9464308, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:12.712656  6113 raft_consensus.cc:2804] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:12.712594  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5d3558278af94262bffa2f467a837472" candidate_uuid: "7ec4352e7b9a4546891ba6c6c9464308" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:12.713039  5859 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 3.
I20260709 12:24:12.713158  5862 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 2.
I20260709 12:24:12.713246  5933 raft_consensus.cc:3055] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.713768  5933 raft_consensus.cc:740] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:24:12.713162  6113 raft_consensus.cc:697] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:12.714380  5751 leader_election.cc:304] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, 7ec4352e7b9a4546891ba6c6c9464308; no voters: 
I20260709 12:24:12.715054  6113 consensus_queue.cc:237] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.715461  6115 raft_consensus.cc:2804] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:12.715050  5933 consensus_queue.cc:260] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.716616  5933 raft_consensus.cc:3060] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.716997  6115 raft_consensus.cc:697] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Becoming Leader. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:12.717931  6115 consensus_queue.cc:237] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:12.723937  5933 raft_consensus.cc:2468] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ec4352e7b9a4546891ba6c6c9464308 in term 3.
I20260709 12:24:12.724362  5684 catalog_manager.cc:5697] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 1 to 2, leader changed from 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.736238  5684 catalog_manager.cc:5697] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 reported cstate change: term changed from 2 to 3, leader changed from a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67) to 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "7ec4352e7b9a4546891ba6c6c9464308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:12.844123  5933 raft_consensus.cc:1275] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:12.845403  6115 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:12.857776  5862 raft_consensus.cc:1275] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:12.859856  6113 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:13.072089  5933 raft_consensus.cc:1275] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:13.073433  6113 consensus_queue.cc:1048] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:13.083675  5862 raft_consensus.cc:1275] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:13.085881  6128 consensus_queue.cc:1048] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:13.156792  5933 raft_consensus.cc:1275] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:13.158202  6128 consensus_queue.cc:1048] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:13.162053  5862 raft_consensus.cc:1275] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:13.163753  6128 consensus_queue.cc:1048] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:13.198729  5933 raft_consensus.cc:1275] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:13.199909  6113 consensus_queue.cc:1048] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:13.208910  5862 raft_consensus.cc:1275] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:13.210745  6128 consensus_queue.cc:1048] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:13.265817  5862 raft_consensus.cc:1275] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:13.266979  6128 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:13.276559  5933 raft_consensus.cc:1275] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Refusing update from remote peer 7ec4352e7b9a4546891ba6c6c9464308: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:13.279229  6113 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:15.262234  5137 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:24:15.262991  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:15.269995  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:53848
I20260709 12:24:15.270424  5785 raft_consensus.cc:606] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Received request to transfer leadership to TS 1269ff02f4d64f5a960ddc33f4abb6c2
I20260709 12:24:15.273296  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:53848
I20260709 12:24:15.273830  5785 raft_consensus.cc:606] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Received request to transfer leadership to TS 1269ff02f4d64f5a960ddc33f4abb6c2
I20260709 12:24:15.276386  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:53848
I20260709 12:24:15.276816  5785 raft_consensus.cc:606] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Received request to transfer leadership to TS a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:24:15.279152  5785 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2"
dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
mode: GRACEFUL
new_leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:53848
I20260709 12:24:15.279558  5785 raft_consensus.cc:606] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Received request to transfer leadership to TS a5947ff1044b47cc93c0083b4a1f939f
I20260709 12:24:15.280721  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:24:15.281174  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:15.391192  6113 raft_consensus.cc:993] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:15.392148  6141 raft_consensus.cc:1081] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Signalling peer a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:15.393651  5936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:40526
I20260709 12:24:15.394127  5936 raft_consensus.cc:493] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:15.394363  5936 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.399045  5936 raft_consensus.cc:515] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:24:15.400964  5936 leader_election.cc:290] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 3 election: Requested vote from peers 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313), 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867)
I20260709 12:24:15.401768  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:15.401997  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee09c423e2ce4d8c82a2a8528478f0cc" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:24:15.402343  5862 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.402736  5785 raft_consensus.cc:3055] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:15.403035  5785 raft_consensus.cc:740] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:15.403609  5785 consensus_queue.cc:260] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:24:15.404424  5785 raft_consensus.cc:3060] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.407392  5862 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 3.
I20260709 12:24:15.408612  5900 leader_election.cc:304] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:15.409040  5785 raft_consensus.cc:2468] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 3.
I20260709 12:24:15.409493  6111 raft_consensus.cc:2804] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:15.410588  6111 raft_consensus.cc:697] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 3 LEADER]: Becoming Leader. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:24:15.411610  6111 consensus_queue.cc:237] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } }
I20260709 12:24:15.418723  5683 catalog_manager.cc:5697] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f reported cstate change: term changed from 2 to 3, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67). New cstate: current_term: 3 leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:15.582824  6141 raft_consensus.cc:993] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:24:15.583371  6142 raft_consensus.cc:1081] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Signalling peer 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:24:15.584853  5862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:24:15.585289  5862 raft_consensus.cc:493] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:15.585556  5862 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.589766  5862 raft_consensus.cc:515] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.591467  5862 leader_election.cc:290] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 4 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:15.592402  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:24:15.592739  5936 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f2f6d6183a847578f8d89ea3f12463b" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:15.593129  5785 raft_consensus.cc:3055] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.593255  5936 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.593475  5785 raft_consensus.cc:740] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:15.594237  5785 consensus_queue.cc:260] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.595407  5785 raft_consensus.cc:3060] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.600090  5936 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 4.
I20260709 12:24:15.601339  5826 leader_election.cc:304] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:15.601799  5785 raft_consensus.cc:2468] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 4.
I20260709 12:24:15.602030  6140 raft_consensus.cc:2804] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:15.603576  6140 raft_consensus.cc:697] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:24:15.604341  6140 consensus_queue.cc:237] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.610781  5683 catalog_manager.cc:5697] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 3 to 4, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:15.699903  6113 raft_consensus.cc:993] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:15.700124  6141 raft_consensus.cc:993] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: : Instructing follower 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:24:15.700562  6142 raft_consensus.cc:1081] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Signalling peer a5947ff1044b47cc93c0083b4a1f939f to start an election
I20260709 12:24:15.700755  6113 raft_consensus.cc:1081] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Signalling peer 1269ff02f4d64f5a960ddc33f4abb6c2 to start an election
I20260709 12:24:15.702219  5936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2"
dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
 from {username='slave'} at 127.0.0.1:40526
I20260709 12:24:15.702616  5862 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3"
dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
 from {username='slave'} at 127.0.0.1:47438
I20260709 12:24:15.702867  5936 raft_consensus.cc:493] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:15.703161  5862 raft_consensus.cc:493] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:24:15.703258  5936 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.703522  5862 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.707723  5862 raft_consensus.cc:515] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.708209  5936 raft_consensus.cc:515] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.709316  5862 leader_election.cc:290] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 3 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67:44395)
I20260709 12:24:15.710124  5785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:24:15.710304  5933 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4e9d92a4296478b93b88d7e31cfd0e3" candidate_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a5947ff1044b47cc93c0083b4a1f939f"
I20260709 12:24:15.710886  5933 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.710843  5936 leader_election.cc:290] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 4 election: Requested vote from peers 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867), 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313)
I20260709 12:24:15.711302  5786 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "7ec4352e7b9a4546891ba6c6c9464308"
I20260709 12:24:15.711416  5862 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee3687476c37453c8a5e1da7c3ca3dc2" candidate_uuid: "a5947ff1044b47cc93c0083b4a1f939f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2"
I20260709 12:24:15.711918  5786 raft_consensus.cc:3055] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.712016  5862 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.712311  5786 raft_consensus.cc:740] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:15.712993  5786 consensus_queue.cc:260] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.713747  5786 raft_consensus.cc:3060] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.710779  5785 raft_consensus.cc:3055] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:15.714270  5785 raft_consensus.cc:740] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 7ec4352e7b9a4546891ba6c6c9464308, State: Running, Role: LEADER
I20260709 12:24:15.715046  5785 consensus_queue.cc:260] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.716347  5785 raft_consensus.cc:3060] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.717516  5933 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 3.
I20260709 12:24:15.718612  5826 leader_election.cc:304] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:15.719405  6140 raft_consensus.cc:2804] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:15.719131  5862 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 4.
I20260709 12:24:15.719868  6140 raft_consensus.cc:697] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 LEADER]: Becoming Leader. State: Replica: 1269ff02f4d64f5a960ddc33f4abb6c2, State: Running, Role: LEADER
I20260709 12:24:15.720557  5900 leader_election.cc:304] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1269ff02f4d64f5a960ddc33f4abb6c2, a5947ff1044b47cc93c0083b4a1f939f; no voters: 
I20260709 12:24:15.720669  6140 consensus_queue.cc:237] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.721421  6111 raft_consensus.cc:2804] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:15.721941  6111 raft_consensus.cc:697] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 4 LEADER]: Becoming Leader. State: Replica: a5947ff1044b47cc93c0083b4a1f939f, State: Running, Role: LEADER
I20260709 12:24:15.722846  6111 consensus_queue.cc:237] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } }
I20260709 12:24:15.724548  5786 raft_consensus.cc:2468] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5947ff1044b47cc93c0083b4a1f939f in term 4.
I20260709 12:24:15.726855  5785 raft_consensus.cc:2468] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1269ff02f4d64f5a960ddc33f4abb6c2 in term 3.
I20260709 12:24:15.730044  5683 catalog_manager.cc:5697] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 reported cstate change: term changed from 2 to 3, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:15.732456  5684 catalog_manager.cc:5697] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f reported cstate change: term changed from 3 to 4, leader changed from 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65) to a5947ff1044b47cc93c0083b4a1f939f (127.5.4.67). New cstate: current_term: 4 leader_uuid: "a5947ff1044b47cc93c0083b4a1f939f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:15.821517  5785 raft_consensus.cc:1275] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:15.822893  6111 consensus_queue.cc:1048] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:15.826118  5862 raft_consensus.cc:1275] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:15.827416  6111 consensus_queue.cc:1048] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:16.008682  5933 raft_consensus.cc:1275] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:16.010604  6140 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.024423  5785 raft_consensus.cc:1275] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:16.026243  6149 consensus_queue.cc:1048] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.185218  5785 raft_consensus.cc:1275] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:16.186720  6143 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:16.200449  5862 raft_consensus.cc:1275] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Refusing update from remote peer a5947ff1044b47cc93c0083b4a1f939f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:16.203447  6143 consensus_queue.cc:1048] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1269ff02f4d64f5a960ddc33f4abb6c2" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:16.242856  5933 raft_consensus.cc:1275] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:16.244168  6149 consensus_queue.cc:1048] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5947ff1044b47cc93c0083b4a1f939f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:16.255752  5785 raft_consensus.cc:1275] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Refusing update from remote peer 1269ff02f4d64f5a960ddc33f4abb6c2: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:16.257489  6165 consensus_queue.cc:1048] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ec4352e7b9a4546891ba6c6c9464308" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 43867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:16.282564  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:16.288319  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:16.288729  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:17.289896  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:17.295555  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:17.295972  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:18.296902  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:18.302690  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:18.303102  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:19.304122  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:19.309821  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:19.310293  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:20.311365  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:20.317397  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:20.317859  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:21.318873  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:21.324150  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:21.324591  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:22.325500  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:22.330904  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:22.331359  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:23.332495  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:23.338781  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:23.339313  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:24.340317  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:24.345795  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:24:24.346284  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:25.379171  5137 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:24:25.379738  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
I20260709 12:24:25.424489  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:25.425439  5137 tablet_replica.cc:333] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.426509  5137 raft_consensus.cc:2243] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.427245  5137 raft_consensus.cc:2272] T 146522f2e2c5416a915558edd9e040cd P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.429237  5137 tablet_replica.cc:333] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.429805  5137 raft_consensus.cc:2243] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.430243  5137 raft_consensus.cc:2272] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.431969  5137 tablet_replica.cc:333] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.432519  5137 raft_consensus.cc:2243] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.432943  5137 raft_consensus.cc:2272] T b4e9d92a4296478b93b88d7e31cfd0e3 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.434440  5137 tablet_replica.cc:333] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.435030  5137 raft_consensus.cc:2243] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.435811  5137 raft_consensus.cc:2272] T 5d3558278af94262bffa2f467a837472 P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.437371  5137 tablet_replica.cc:333] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.437875  5137 raft_consensus.cc:2243] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.438687  5137 raft_consensus.cc:2272] T 8db795ca990d4664bcf6fa16bc3a28ab P 7ec4352e7b9a4546891ba6c6c9464308 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.440394  5137 tablet_replica.cc:333] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.440881  5137 raft_consensus.cc:2243] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.441560  5137 raft_consensus.cc:2272] T 2370d693fdc84419bac799cc5d782d6c P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.443286  5137 tablet_replica.cc:333] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.443778  5137 raft_consensus.cc:2243] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.444178  5137 raft_consensus.cc:2272] T 2f2f6d6183a847578f8d89ea3f12463b P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.445662  5137 tablet_replica.cc:333] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.446134  5137 raft_consensus.cc:2243] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.446916  5137 raft_consensus.cc:2272] T e8bc2054813a4327977ecc742b0d68c7 P 7ec4352e7b9a4546891ba6c6c9464308 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.448531  5137 tablet_replica.cc:333] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.449030  5137 raft_consensus.cc:2243] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.449430  5137 raft_consensus.cc:2272] T ee3687476c37453c8a5e1da7c3ca3dc2 P 7ec4352e7b9a4546891ba6c6c9464308 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.451176  5137 tablet_replica.cc:333] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308: stopping tablet replica
I20260709 12:24:25.451648  5137 raft_consensus.cc:2243] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.452050  5137 raft_consensus.cc:2272] T ee09c423e2ce4d8c82a2a8528478f0cc P 7ec4352e7b9a4546891ba6c6c9464308 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:25.477106  5826 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:43867: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20260709 12:24:25.481869  5826 consensus_peers.cc:597] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 -> Peer 7ec4352e7b9a4546891ba6c6c9464308 (127.5.4.65:43867): Couldn't send request to peer 7ec4352e7b9a4546891ba6c6c9464308. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:43867: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.483866  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
I20260709 12:24:25.508018  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
W20260709 12:24:25.521559  5900 consensus_peers.cc:597] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f -> Peer 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313): Couldn't send request to peer 1269ff02f4d64f5a960ddc33f4abb6c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.541428  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:25.542215  5137 tablet_replica.cc:333] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.542846  5137 raft_consensus.cc:2243] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.543634  5137 raft_consensus.cc:2272] T 146522f2e2c5416a915558edd9e040cd P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.545946  5137 tablet_replica.cc:333] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.546805  5137 raft_consensus.cc:2243] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.547326  5137 raft_consensus.cc:2272] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:25.548731  5900 consensus_peers.cc:597] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f -> Peer 1269ff02f4d64f5a960ddc33f4abb6c2 (127.5.4.66:41313): Couldn't send request to peer 1269ff02f4d64f5a960ddc33f4abb6c2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:25.549449  5137 tablet_replica.cc:333] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.550004  5137 raft_consensus.cc:2243] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.550760  5137 raft_consensus.cc:2272] T b4e9d92a4296478b93b88d7e31cfd0e3 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.552577  5137 tablet_replica.cc:333] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.553077  5137 raft_consensus.cc:2243] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.553459  5137 raft_consensus.cc:2272] T 5d3558278af94262bffa2f467a837472 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.555188  5137 tablet_replica.cc:333] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.555680  5137 raft_consensus.cc:2243] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.556061  5137 raft_consensus.cc:2272] T 8db795ca990d4664bcf6fa16bc3a28ab P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.557564  5137 tablet_replica.cc:333] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.558058  5137 raft_consensus.cc:2243] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.558753  5137 raft_consensus.cc:2272] T 2f2f6d6183a847578f8d89ea3f12463b P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.560248  5137 tablet_replica.cc:333] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.560725  5137 raft_consensus.cc:2243] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.561132  5137 raft_consensus.cc:2272] T 2370d693fdc84419bac799cc5d782d6c P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.562757  5137 tablet_replica.cc:333] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.563225  5137 raft_consensus.cc:2243] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.563597  5137 raft_consensus.cc:2272] T ee3687476c37453c8a5e1da7c3ca3dc2 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.565137  5137 tablet_replica.cc:333] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.565588  5137 raft_consensus.cc:2243] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.565996  5137 raft_consensus.cc:2272] T e8bc2054813a4327977ecc742b0d68c7 P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.567464  5137 tablet_replica.cc:333] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2: stopping tablet replica
I20260709 12:24:25.567896  5137 raft_consensus.cc:2243] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.568292  5137 raft_consensus.cc:2272] T ee09c423e2ce4d8c82a2a8528478f0cc P 1269ff02f4d64f5a960ddc33f4abb6c2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.596918  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
I20260709 12:24:25.616462  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
I20260709 12:24:25.653524  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:24:25.654279  5137 tablet_replica.cc:333] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.654960  5137 raft_consensus.cc:2243] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.655827  5137 raft_consensus.cc:2272] T 6a3050b37c2a4bc2883f94d8ea7e8be7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.657666  5137 tablet_replica.cc:333] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.658269  5137 raft_consensus.cc:2243] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.658746  5137 raft_consensus.cc:2272] T 146522f2e2c5416a915558edd9e040cd P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.660503  5137 tablet_replica.cc:333] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.661032  5137 raft_consensus.cc:2243] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.661478  5137 raft_consensus.cc:2272] T 2370d693fdc84419bac799cc5d782d6c P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.663254  5137 tablet_replica.cc:333] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.663780  5137 raft_consensus.cc:2243] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.664237  5137 raft_consensus.cc:2272] T 2f2f6d6183a847578f8d89ea3f12463b P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.665843  5137 tablet_replica.cc:333] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.666380  5137 raft_consensus.cc:2243] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.667136  5137 raft_consensus.cc:2272] T b4e9d92a4296478b93b88d7e31cfd0e3 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.668728  5137 tablet_replica.cc:333] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.669203  5137 raft_consensus.cc:2243] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.669611  5137 raft_consensus.cc:2272] T 5d3558278af94262bffa2f467a837472 P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.671284  5137 tablet_replica.cc:333] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.671746  5137 raft_consensus.cc:2243] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.672454  5137 raft_consensus.cc:2272] T ee3687476c37453c8a5e1da7c3ca3dc2 P a5947ff1044b47cc93c0083b4a1f939f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.673970  5137 tablet_replica.cc:333] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.674614  5137 raft_consensus.cc:2243] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.675000  5137 raft_consensus.cc:2272] T e8bc2054813a4327977ecc742b0d68c7 P a5947ff1044b47cc93c0083b4a1f939f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.676489  5137 tablet_replica.cc:333] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.676935  5137 raft_consensus.cc:2243] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.677608  5137 raft_consensus.cc:2272] T ee09c423e2ce4d8c82a2a8528478f0cc P a5947ff1044b47cc93c0083b4a1f939f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.679267  5137 tablet_replica.cc:333] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f: stopping tablet replica
I20260709 12:24:25.679701  5137 raft_consensus.cc:2243] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:25.680083  5137 raft_consensus.cc:2272] T 8db795ca990d4664bcf6fa16bc3a28ab P a5947ff1044b47cc93c0083b4a1f939f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.708482  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:24:25.731211  5137 master.cc:562] Master@127.5.4.126:36811 shutting down...
I20260709 12:24:25.756556  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:24:25.757140  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:25.757423  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P 62b5ffc2872a4abb9b86c3a47afb6a94: stopping tablet replica
I20260709 12:24:25.778304  5137 master.cc:584] Master@127.5.4.126:36811 shutdown complete.
1269ff02f4d64f5a960ddc33f4abb6c2  4
7ec4352e7b9a4546891ba6c6c9464308  5
a5947ff1044b47cc93c0083b4a1f939f  1
1269ff02f4d64f5a960ddc33f4abb6c2  4
7ec4352e7b9a4546891ba6c6c9464308  3
a5947ff1044b47cc93c0083b4a1f939f  3
1269ff02f4d64f5a960ddc33f4abb6c2  1
7ec4352e7b9a4546891ba6c6c9464308  8
a5947ff1044b47cc93c0083b4a1f939f  1
1269ff02f4d64f5a960ddc33f4abb6c2  3
7ec4352e7b9a4546891ba6c6c9464308  4
a5947ff1044b47cc93c0083b4a1f939f  3
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28969 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:24:25.822044  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:35813
I20260709 12:24:25.823287  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:25.829702  6182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:25.829960  6181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:25.830224  6184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:25.830907  5137 server_base.cc:1061] running on GCE node
I20260709 12:24:25.831977  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:25.832180  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:25.832384  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599865832365 us; error 0 us; skew 500 ppm
I20260709 12:24:25.833032  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:25.835768  5137 webserver.cc:533] Webserver started at http://127.5.4.126:41849/ using document root <none> and password file <none>
I20260709 12:24:25.836321  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:25.836531  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:25.836855  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:25.838438  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "08c731ab128f4ef881e4c3c73ff48bd8"
format_stamp: "Formatted at 2026-07-09 12:24:25 on dist-test-slave-lg8f"
I20260709 12:24:25.848307  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.009s	user 0.005s	sys 0.003s
I20260709 12:24:25.853006  6189 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:25.853798  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:24:25.854135  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "08c731ab128f4ef881e4c3c73ff48bd8"
format_stamp: "Formatted at 2026-07-09 12:24:25 on dist-test-slave-lg8f"
I20260709 12:24:25.854416  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:25.864892  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:25.866266  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:25.911032  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:35813
I20260709 12:24:25.911157  6240 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:35813 every 8 connection(s)
I20260709 12:24:25.915236  6241 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:25.925678  6241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Bootstrap starting.
I20260709 12:24:25.930213  6241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:25.934651  6241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: No bootstrap required, opened a new log
I20260709 12:24:25.936702  6241 raft_consensus.cc:359] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" member_type: VOTER }
I20260709 12:24:25.937067  6241 raft_consensus.cc:385] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:25.937342  6241 raft_consensus.cc:740] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08c731ab128f4ef881e4c3c73ff48bd8, State: Initialized, Role: FOLLOWER
I20260709 12:24:25.937954  6241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" member_type: VOTER }
I20260709 12:24:25.938431  6241 raft_consensus.cc:399] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:24:25.938697  6241 raft_consensus.cc:493] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:24:25.939003  6241 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:25.943414  6241 raft_consensus.cc:515] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" member_type: VOTER }
I20260709 12:24:25.944006  6241 leader_election.cc:304] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08c731ab128f4ef881e4c3c73ff48bd8; no voters: 
I20260709 12:24:25.945238  6241 leader_election.cc:290] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:24:25.945734  6244 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:25.947417  6244 raft_consensus.cc:697] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 1 LEADER]: Becoming Leader. State: Replica: 08c731ab128f4ef881e4c3c73ff48bd8, State: Running, Role: LEADER
I20260709 12:24:25.948271  6244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" member_type: VOTER }
I20260709 12:24:25.948537  6241 sys_catalog.cc:565] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:24:25.954093  6245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" member_type: VOTER } }
I20260709 12:24:25.955260  6245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:25.954384  6246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08c731ab128f4ef881e4c3c73ff48bd8. Latest consensus state: current_term: 1 leader_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08c731ab128f4ef881e4c3c73ff48bd8" member_type: VOTER } }
I20260709 12:24:25.957733  6246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [sys.catalog]: This master's current role is: LEADER
I20260709 12:24:25.959300  6253 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:24:25.967024  6253 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:24:25.968384  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:24:25.977128  6253 catalog_manager.cc:1383] Generated new cluster ID: 8edfa0c2a6dc45849587aa2c087efc65
I20260709 12:24:25.977545  6253 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:24:25.992919  6253 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:24:25.994355  6253 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:24:26.012161  6253 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Generated new TSK 0
I20260709 12:24:26.012961  6253 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:24:26.035634  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:26.044076  6263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:26.044245  6264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:26.045886  6266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:26.047000  5137 server_base.cc:1061] running on GCE node
I20260709 12:24:26.048045  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:26.048261  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:26.048465  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599866048446 us; error 0 us; skew 500 ppm
I20260709 12:24:26.049022  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:26.052008  5137 webserver.cc:533] Webserver started at http://127.5.4.65:41625/ using document root <none> and password file <none>
I20260709 12:24:26.052572  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:26.052788  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:26.053115  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:26.054577  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "4268023ecea149cd83cba77db53c3100"
format_stamp: "Formatted at 2026-07-09 12:24:26 on dist-test-slave-lg8f"
I20260709 12:24:26.059696  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:24:26.063419  6271 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.064307  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:24:26.064646  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "4268023ecea149cd83cba77db53c3100"
format_stamp: "Formatted at 2026-07-09 12:24:26 on dist-test-slave-lg8f"
I20260709 12:24:26.064913  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:26.087625  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:26.088958  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:26.090654  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:26.093145  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:26.093398  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.093670  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:26.093855  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.141767  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:37395
I20260709 12:24:26.141959  6334 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:37395 every 8 connection(s)
I20260709 12:24:26.147056  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:26.154531  6339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:26.155916  6340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:26.159721  6343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:26.160413  5137 server_base.cc:1061] running on GCE node
I20260709 12:24:26.162707  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:24:26.162787  6335 heartbeater.cc:344] Connected to a master server at 127.5.4.126:35813
W20260709 12:24:26.162904  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:26.163461  6335 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:26.163429  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599866163407 us; error 0 us; skew 500 ppm
I20260709 12:24:26.164340  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:26.164672  6335 heartbeater.cc:507] Master 127.5.4.126:35813 requested a full tablet report, sending...
I20260709 12:24:26.167382  5137 webserver.cc:533] Webserver started at http://127.5.4.66:44345/ using document root <none> and password file <none>
I20260709 12:24:26.167712  6206 ts_manager.cc:194] Registered new tserver with Master: 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:24:26.167912  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:26.168242  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:26.168529  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:26.169703  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
format_stamp: "Formatted at 2026-07-09 12:24:26 on dist-test-slave-lg8f"
I20260709 12:24:26.170107  6206 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49010
I20260709 12:24:26.174466  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:24:26.178386  6347 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.179404  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:24:26.179761  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
format_stamp: "Formatted at 2026-07-09 12:24:26 on dist-test-slave-lg8f"
I20260709 12:24:26.180063  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:26.198275  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:26.199574  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:26.201164  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:26.203447  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:26.203680  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.203930  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:26.204104  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.248275  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:34641
I20260709 12:24:26.248473  6409 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:34641 every 8 connection(s)
I20260709 12:24:26.253376  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:24:26.260532  6413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:26.262956  6414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:26.264787  5137 server_base.cc:1061] running on GCE node
W20260709 12:24:26.266044  6416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:26.266943  6410 heartbeater.cc:344] Connected to a master server at 127.5.4.126:35813
I20260709 12:24:26.267383  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:24:26.267385  6410 heartbeater.cc:461] Registering TS with master...
W20260709 12:24:26.267796  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:26.268019  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599866267999 us; error 0 us; skew 500 ppm
I20260709 12:24:26.268556  6410 heartbeater.cc:507] Master 127.5.4.126:35813 requested a full tablet report, sending...
I20260709 12:24:26.268796  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:26.271100  6206 ts_manager.cc:194] Registered new tserver with Master: 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:26.272253  5137 webserver.cc:533] Webserver started at http://127.5.4.67:42327/ using document root <none> and password file <none>
I20260709 12:24:26.272965  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:26.273124  6206 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49024
I20260709 12:24:26.273252  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:26.273721  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:26.275204  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "4e0e521cb08c420f981a29576eee90a6"
format_stamp: "Formatted at 2026-07-09 12:24:26 on dist-test-slave-lg8f"
I20260709 12:24:26.280130  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:24:26.284035  6421 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.285038  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:24:26.285334  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "4e0e521cb08c420f981a29576eee90a6"
format_stamp: "Formatted at 2026-07-09 12:24:26 on dist-test-slave-lg8f"
I20260709 12:24:26.285607  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:26.317253  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:26.318902  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:26.320369  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:26.322741  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:26.322935  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.323242  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:26.323392  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:26.369979  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:45591
I20260709 12:24:26.370162  6483 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:45591 every 8 connection(s)
I20260709 12:24:26.387521  6484 heartbeater.cc:344] Connected to a master server at 127.5.4.126:35813
I20260709 12:24:26.387977  6484 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:26.388772  6484 heartbeater.cc:507] Master 127.5.4.126:35813 requested a full tablet report, sending...
I20260709 12:24:26.390991  6206 ts_manager.cc:194] Registered new tserver with Master: 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:26.391255  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017196051s
I20260709 12:24:26.392870  6206 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49040
I20260709 12:24:26.421080  6206 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49052:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:24:26.428248  6206 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:24:26.640286  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 4db5d1ae5f524ad799ccb78373ce3ff7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:26.642313  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db5d1ae5f524ad799ccb78373ce3ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.648442  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 906fbea0458f4a28bfad3b9dde8c6b35 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:26.650178  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906fbea0458f4a28bfad3b9dde8c6b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.652154  6297 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0563d158eb400f86c11eb8e04fc8fc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:26.653734  6297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0563d158eb400f86c11eb8e04fc8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.655689  6296 tablet_service.cc:1511] Processing CreateTablet for tablet 28020383d3c344fda935a94bca07ad82 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:26.657048  6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28020383d3c344fda935a94bca07ad82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.658972  6443 tablet_service.cc:1511] Processing CreateTablet for tablet dd3cb6d1db5d481b9589e84d04e83d8e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:26.659225  6449 tablet_service.cc:1511] Processing CreateTablet for tablet 4db5d1ae5f524ad799ccb78373ce3ff7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:26.660298  6443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3cb6d1db5d481b9589e84d04e83d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.666502  6295 tablet_service.cc:1511] Processing CreateTablet for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:26.666801  6292 tablet_service.cc:1511] Processing CreateTablet for tablet f73da77e7bb0404e9f79f06e3dd200f4 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:26.667212  6293 tablet_service.cc:1511] Processing CreateTablet for tablet dd3cb6d1db5d481b9589e84d04e83d8e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:26.667747  6291 tablet_service.cc:1511] Processing CreateTablet for tablet b18717d2a8534f54b725363ce75d49ed (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:26.668437  6448 tablet_service.cc:1511] Processing CreateTablet for tablet 906fbea0458f4a28bfad3b9dde8c6b35 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:26.669241  6444 tablet_service.cc:1511] Processing CreateTablet for tablet e600ce2bd05948cf8ce5a5f292f57f28 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:26.670436  6294 tablet_service.cc:1511] Processing CreateTablet for tablet e600ce2bd05948cf8ce5a5f292f57f28 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:26.671154  6445 tablet_service.cc:1511] Processing CreateTablet for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:26.671852  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db5d1ae5f524ad799ccb78373ce3ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.672498  6448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906fbea0458f4a28bfad3b9dde8c6b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.673369  6446 tablet_service.cc:1511] Processing CreateTablet for tablet 28020383d3c344fda935a94bca07ad82 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:26.674726  6446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28020383d3c344fda935a94bca07ad82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.676468  6290 tablet_service.cc:1511] Processing CreateTablet for tablet a2f5503246494eed8b91ac10cdf742e2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:26.677306  6447 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0563d158eb400f86c11eb8e04fc8fc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:26.678879  6445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.679925  6447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0563d158eb400f86c11eb8e04fc8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.680608  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e600ce2bd05948cf8ce5a5f292f57f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.684248  6441 tablet_service.cc:1511] Processing CreateTablet for tablet b18717d2a8534f54b725363ce75d49ed (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:26.685652  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18717d2a8534f54b725363ce75d49ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.687047  6442 tablet_service.cc:1511] Processing CreateTablet for tablet f73da77e7bb0404e9f79f06e3dd200f4 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:26.688411  6442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73da77e7bb0404e9f79f06e3dd200f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.689522  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.690372  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3cb6d1db5d481b9589e84d04e83d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.691368  6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f5503246494eed8b91ac10cdf742e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.694893  6291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18717d2a8534f54b725363ce75d49ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.695155  6440 tablet_service.cc:1511] Processing CreateTablet for tablet a2f5503246494eed8b91ac10cdf742e2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:26.696568  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f5503246494eed8b91ac10cdf742e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.702862  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73da77e7bb0404e9f79f06e3dd200f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.707562  6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e600ce2bd05948cf8ce5a5f292f57f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.745658  6440 tablet_service.cc:1511] Processing CreateTablet for tablet c2d3ed2ffee2408ebcb407323a99660b (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:26.747186  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d3ed2ffee2408ebcb407323a99660b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.747779  6375 tablet_service.cc:1511] Processing CreateTablet for tablet 4db5d1ae5f524ad799ccb78373ce3ff7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:24:26.749258  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db5d1ae5f524ad799ccb78373ce3ff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.751550  6374 tablet_service.cc:1511] Processing CreateTablet for tablet 906fbea0458f4a28bfad3b9dde8c6b35 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:24:26.753429  6368 tablet_service.cc:1511] Processing CreateTablet for tablet f73da77e7bb0404e9f79f06e3dd200f4 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:24:26.753924  6372 tablet_service.cc:1511] Processing CreateTablet for tablet 28020383d3c344fda935a94bca07ad82 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:24:26.751660  6369 tablet_service.cc:1511] Processing CreateTablet for tablet dd3cb6d1db5d481b9589e84d04e83d8e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:24:26.750075  6373 tablet_service.cc:1511] Processing CreateTablet for tablet 3a0563d158eb400f86c11eb8e04fc8fc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:24:26.757822  6371 tablet_service.cc:1511] Processing CreateTablet for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:24:26.758317  6374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 906fbea0458f4a28bfad3b9dde8c6b35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.770061  6366 tablet_service.cc:1511] Processing CreateTablet for tablet a2f5503246494eed8b91ac10cdf742e2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:24:26.770819  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73da77e7bb0404e9f79f06e3dd200f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.774868  6373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a0563d158eb400f86c11eb8e04fc8fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.759512  6370 tablet_service.cc:1511] Processing CreateTablet for tablet e600ce2bd05948cf8ce5a5f292f57f28 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:24:26.759657  6367 tablet_service.cc:1511] Processing CreateTablet for tablet b18717d2a8534f54b725363ce75d49ed (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:24:26.765740  6290 tablet_service.cc:1511] Processing CreateTablet for tablet c2d3ed2ffee2408ebcb407323a99660b (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:26.769572  6499 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:26.776553  6370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e600ce2bd05948cf8ce5a5f292f57f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.787837  6372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28020383d3c344fda935a94bca07ad82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.790184  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.790815  6367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b18717d2a8534f54b725363ce75d49ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.791507  6369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd3cb6d1db5d481b9589e84d04e83d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.794878  6366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f5503246494eed8b91ac10cdf742e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.819777  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 252dcff751be4a3d948b0f83cb081100 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:26.821254  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252dcff751be4a3d948b0f83cb081100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.827739  6443 tablet_service.cc:1511] Processing CreateTablet for tablet 252dcff751be4a3d948b0f83cb081100 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:26.829223  6443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252dcff751be4a3d948b0f83cb081100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.830744  6292 tablet_service.cc:1511] Processing CreateTablet for tablet f4d316e983e24c05a1bda96f3c053d46 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:26.837162  6449 tablet_service.cc:1511] Processing CreateTablet for tablet f4d316e983e24c05a1bda96f3c053d46 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:26.841271  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d316e983e24c05a1bda96f3c053d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.844750  6444 tablet_service.cc:1511] Processing CreateTablet for tablet bdf58974e308489d9ae9a07e0a775c71 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:26.845616  6293 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe2196990ce4d60adc195d394d4c01c (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:26.850229  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe2196990ce4d60adc195d394d4c01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.841948  6296 tablet_service.cc:1511] Processing CreateTablet for tablet bdf58974e308489d9ae9a07e0a775c71 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:26.860460  6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf58974e308489d9ae9a07e0a775c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.838430  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d316e983e24c05a1bda96f3c053d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.862787  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf58974e308489d9ae9a07e0a775c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.854132  6447 tablet_service.cc:1511] Processing CreateTablet for tablet 402a1967982740658c70049fd62be44e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:26.852519  6448 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe2196990ce4d60adc195d394d4c01c (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:26.853523  6446 tablet_service.cc:1511] Processing CreateTablet for tablet cd67024481794a7db3a98df0cd400c9d (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:26.869570  6448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe2196990ce4d60adc195d394d4c01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.872140  6447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402a1967982740658c70049fd62be44e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.872849  6446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd67024481794a7db3a98df0cd400c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.855379  6500 tablet_bootstrap.cc:492] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:26.855509  6499 tablet_bootstrap.cc:654] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:26.890826  6500 tablet_bootstrap.cc:654] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:26.896502  6442 tablet_service.cc:1511] Processing CreateTablet for tablet 40665848f81d4ce9a753d2417536ae0a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:26.897272  6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d3ed2ffee2408ebcb407323a99660b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.899710  6445 tablet_service.cc:1511] Processing CreateTablet for tablet d53ab19d424b4a85af2153ef24520678 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:26.901170  6445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53ab19d424b4a85af2153ef24520678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.904212  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 402a1967982740658c70049fd62be44e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:26.905507  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402a1967982740658c70049fd62be44e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.907022  6442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40665848f81d4ce9a753d2417536ae0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.886003  6375 tablet_service.cc:1511] Processing CreateTablet for tablet 252dcff751be4a3d948b0f83cb081100 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:24:26.909361  6441 tablet_service.cc:1511] Processing CreateTablet for tablet c413b06caa444496aecb13ac2142c246 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:26.910502  6294 tablet_service.cc:1511] Processing CreateTablet for tablet 40665848f81d4ce9a753d2417536ae0a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:26.908538  6371 tablet_service.cc:1511] Processing CreateTablet for tablet c2d3ed2ffee2408ebcb407323a99660b (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:24:26.909976  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 252dcff751be4a3d948b0f83cb081100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.902722  6297 tablet_service.cc:1511] Processing CreateTablet for tablet cd67024481794a7db3a98df0cd400c9d (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:24:26.921103  6367 tablet_service.cc:1511] Processing CreateTablet for tablet f4d316e983e24c05a1bda96f3c053d46 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:24:26.926851  6370 tablet_service.cc:1511] Processing CreateTablet for tablet bdf58974e308489d9ae9a07e0a775c71 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:24:26.929445  6372 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe2196990ce4d60adc195d394d4c01c (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:24:26.930302  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c413b06caa444496aecb13ac2142c246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.942730  6297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd67024481794a7db3a98df0cd400c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.946310  6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40665848f81d4ce9a753d2417536ae0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.948272  6440 tablet_service.cc:1511] Processing CreateTablet for tablet f745d20b9f8c4aca9adf20ed14f509ac (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:26.949715  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d3ed2ffee2408ebcb407323a99660b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.951547  6295 tablet_service.cc:1511] Processing CreateTablet for tablet d53ab19d424b4a85af2153ef24520678 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:26.952883  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53ab19d424b4a85af2153ef24520678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.954941  6372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe2196990ce4d60adc195d394d4c01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.956393  6368 tablet_service.cc:1511] Processing CreateTablet for tablet c413b06caa444496aecb13ac2142c246 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:26.956946  6374 tablet_service.cc:1511] Processing CreateTablet for tablet cd67024481794a7db3a98df0cd400c9d (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
W20260709 12:24:26.960927  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:26.961314  6373 tablet_service.cc:1511] Processing CreateTablet for tablet 402a1967982740658c70049fd62be44e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:24:26.963063  6501 tablet_bootstrap.cc:492] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:26.973109  6366 tablet_service.cc:1511] Processing CreateTablet for tablet 40665848f81d4ce9a753d2417536ae0a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:24:26.974289  6449 tablet_service.cc:1511] Processing CreateTablet for tablet 8729b74dbaf94d7092d811f0a55205f2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:26.975633  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729b74dbaf94d7092d811f0a55205f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.976163  6291 tablet_service.cc:1511] Processing CreateTablet for tablet f745d20b9f8c4aca9adf20ed14f509ac (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:26.969978  6290 tablet_service.cc:1511] Processing CreateTablet for tablet c413b06caa444496aecb13ac2142c246 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:24:26.983593  6500 tablet_bootstrap.cc:492] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:26.984442  6443 tablet_service.cc:1511] Processing CreateTablet for tablet 7878a824e1e94db6a0b83ab47738f4b0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:27.008000  6500 ts_tablet_manager.cc:1403] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.153s	user 0.005s	sys 0.013s
I20260709 12:24:27.009829  6367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d316e983e24c05a1bda96f3c053d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.011857  6370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdf58974e308489d9ae9a07e0a775c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.012504  6374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd67024481794a7db3a98df0cd400c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.012871  6500 raft_consensus.cc:359] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.013841  6500 raft_consensus.cc:385] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.014182  6500 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.014730  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c413b06caa444496aecb13ac2142c246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.015005  6500 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.018646  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f745d20b9f8c4aca9adf20ed14f509ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.018934  6366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 40665848f81d4ce9a753d2417536ae0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.022795  6443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7878a824e1e94db6a0b83ab47738f4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.023026  6373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402a1967982740658c70049fd62be44e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.024991  6501 tablet_bootstrap.cc:654] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.036265  6292 tablet_service.cc:1511] Processing CreateTablet for tablet 7878a824e1e94db6a0b83ab47738f4b0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:27.045384  6371 tablet_service.cc:1511] Processing CreateTablet for tablet f745d20b9f8c4aca9adf20ed14f509ac (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:24:27.046773  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f745d20b9f8c4aca9adf20ed14f509ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.053582  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 540a67ae2f134850bfa9955841e5f56a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:26.987815  6444 tablet_service.cc:1511] Processing CreateTablet for tablet f0250a69b3914cb38265a7457b73a158 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:26.989089  6447 tablet_service.cc:1511] Processing CreateTablet for tablet eede017c955d4746bbbd40d9384932e7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:26.991485  6499 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:27.055061  6499 ts_tablet_manager.cc:1403] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.286s	user 0.010s	sys 0.012s
I20260709 12:24:27.055452  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0250a69b3914cb38265a7457b73a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.056212  6447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eede017c955d4746bbbd40d9384932e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.991627  6293 tablet_service.cc:1511] Processing CreateTablet for tablet 8729b74dbaf94d7092d811f0a55205f2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:26.998775  6291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f745d20b9f8c4aca9adf20ed14f509ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:26.999641  6446 tablet_service.cc:1511] Processing CreateTablet for tablet 540a67ae2f134850bfa9955841e5f56a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:27.059923  6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c413b06caa444496aecb13ac2142c246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.060034  6446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540a67ae2f134850bfa9955841e5f56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.060632  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729b74dbaf94d7092d811f0a55205f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.003206  6294 tablet_service.cc:1511] Processing CreateTablet for tablet f0250a69b3914cb38265a7457b73a158 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:27.062848  6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0250a69b3914cb38265a7457b73a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.058377  6499 raft_consensus.cc:359] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.063480  6499 raft_consensus.cc:385] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.063922  6499 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.003834  6369 tablet_service.cc:1511] Processing CreateTablet for tablet d53ab19d424b4a85af2153ef24520678 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:24:27.066776  6369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53ab19d424b4a85af2153ef24520678. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.070895  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7878a824e1e94db6a0b83ab47738f4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.073127  6375 tablet_service.cc:1511] Processing CreateTablet for tablet 8729b74dbaf94d7092d811f0a55205f2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:24:27.073206  6499 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.089908  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729b74dbaf94d7092d811f0a55205f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.090067  6335 heartbeater.cc:499] Master 127.5.4.126:35813 was elected leader, sending a full tablet report...
I20260709 12:24:27.090552  6499 ts_tablet_manager.cc:1434] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.002s
I20260709 12:24:27.097785  6449 tablet_service.cc:1511] Processing CreateTablet for tablet 481d3efac77d41e6bba0961a140626dc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:27.099146  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481d3efac77d41e6bba0961a140626dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.092404  6296 tablet_service.cc:1511] Processing CreateTablet for tablet eede017c955d4746bbbd40d9384932e7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:27.106017  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540a67ae2f134850bfa9955841e5f56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.113911  6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eede017c955d4746bbbd40d9384932e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.115773  6441 tablet_service.cc:1511] Processing CreateTablet for tablet 44fa024e8a7d48a6bc67d49f348bc0c0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:27.117276  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fa024e8a7d48a6bc67d49f348bc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.119310  6499 tablet_bootstrap.cc:492] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:27.131685  6368 tablet_service.cc:1511] Processing CreateTablet for tablet f0250a69b3914cb38265a7457b73a158 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:24:27.133122  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0250a69b3914cb38265a7457b73a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.136027  6372 tablet_service.cc:1511] Processing CreateTablet for tablet 7878a824e1e94db6a0b83ab47738f4b0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:24:27.137339  6372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7878a824e1e94db6a0b83ab47738f4b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.137598  6370 tablet_service.cc:1511] Processing CreateTablet for tablet eede017c955d4746bbbd40d9384932e7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:24:27.133122  6445 tablet_service.cc:1511] Processing CreateTablet for tablet 62bd177b33be425d81796cc414f86f76 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:27.141146  6445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bd177b33be425d81796cc414f86f76. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:27.146822  6336 tablet.cc:2406] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:24:27.139181  6370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eede017c955d4746bbbd40d9384932e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.170713  6499 tablet_bootstrap.cc:654] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.186571  6295 tablet_service.cc:1511] Processing CreateTablet for tablet 481d3efac77d41e6bba0961a140626dc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:27.164748  6375 tablet_service.cc:1511] Processing CreateTablet for tablet 540a67ae2f134850bfa9955841e5f56a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:24:27.183938  6297 tablet_service.cc:1511] Processing CreateTablet for tablet 44fa024e8a7d48a6bc67d49f348bc0c0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:27.192127  6448 tablet_service.cc:1511] Processing CreateTablet for tablet a5012c92e5bb4fe188dca8f60efceeb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:27.193519  6448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5012c92e5bb4fe188dca8f60efceeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.193632  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481d3efac77d41e6bba0961a140626dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.194898  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 62bd177b33be425d81796cc414f86f76 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:27.196725  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bd177b33be425d81796cc414f86f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.197275  6374 tablet_service.cc:1511] Processing CreateTablet for tablet 44fa024e8a7d48a6bc67d49f348bc0c0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:24:27.221529  6297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fa024e8a7d48a6bc67d49f348bc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.158969  6443 tablet_service.cc:1511] Processing CreateTablet for tablet 4e75b2c5204044d2847e78c187388fd3 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:27.222733  6443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e75b2c5204044d2847e78c187388fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.226992  6374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fa024e8a7d48a6bc67d49f348bc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.218364  6484 heartbeater.cc:499] Master 127.5.4.126:35813 was elected leader, sending a full tablet report...
I20260709 12:24:27.232374  6366 tablet_service.cc:1511] Processing CreateTablet for tablet 4e75b2c5204044d2847e78c187388fd3 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:27.241606  6293 tablet_service.cc:1511] Processing CreateTablet for tablet a5012c92e5bb4fe188dca8f60efceeb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:27.203720  6373 tablet_service.cc:1511] Processing CreateTablet for tablet 481d3efac77d41e6bba0961a140626dc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:24:27.227417  6500 ts_tablet_manager.cc:1434] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.219s	user 0.012s	sys 0.020s
I20260709 12:24:27.237932  6291 tablet_service.cc:1511] Processing CreateTablet for tablet 2ea11ad8cada4bb682c147f8a26cceb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:27.229928  6446 tablet_service.cc:1511] Processing CreateTablet for tablet 2ea11ad8cada4bb682c147f8a26cceb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:27.231504  6290 tablet_service.cc:1511] Processing CreateTablet for tablet 4d797adfa10541918dfc0ebf49b90784 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:27.231801  6369 tablet_service.cc:1511] Processing CreateTablet for tablet 62bd177b33be425d81796cc414f86f76 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:24:27.228488  6292 tablet_service.cc:1511] Processing CreateTablet for tablet 4e75b2c5204044d2847e78c187388fd3 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:24:27.215524  6440 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea0c2d7699e4559823a28a280a0e713 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:27.253031  6373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481d3efac77d41e6bba0961a140626dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.253018  6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d797adfa10541918dfc0ebf49b90784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.174734  6442 tablet_service.cc:1511] Processing CreateTablet for tablet 4d797adfa10541918dfc0ebf49b90784 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:27.262809  6366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e75b2c5204044d2847e78c187388fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.265506  6444 tablet_service.cc:1511] Processing CreateTablet for tablet b776445e68c14533bcde1f784caa703a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:27.267009  6446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ea11ad8cada4bb682c147f8a26cceb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.267503  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 540a67ae2f134850bfa9955841e5f56a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.270052  6367 tablet_service.cc:1511] Processing CreateTablet for tablet 2ea11ad8cada4bb682c147f8a26cceb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:24:27.277140  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5012c92e5bb4fe188dca8f60efceeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.277524  6410 heartbeater.cc:499] Master 127.5.4.126:35813 was elected leader, sending a full tablet report...
I20260709 12:24:27.279579  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b776445e68c14533bcde1f784caa703a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.285008  6442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d797adfa10541918dfc0ebf49b90784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.287819  6445 tablet_service.cc:1511] Processing CreateTablet for tablet 34bae875d7104f298673c223da6caf2e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:27.288967  6505 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.289225  6445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bae875d7104f298673c223da6caf2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.289541  6505 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.293207  6368 tablet_service.cc:1511] Processing CreateTablet for tablet a5012c92e5bb4fe188dca8f60efceeb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:24:27.299088  6505 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:27.310817  6369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bd177b33be425d81796cc414f86f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.311048  6500 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:27.327157  6504 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.327708  6504 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.329183  6447 tablet_service.cc:1511] Processing CreateTablet for tablet f53146508e564f8aae279a61e6696e80 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:27.338744  6370 tablet_service.cc:1511] Processing CreateTablet for tablet 4d797adfa10541918dfc0ebf49b90784 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:24:27.339615  6296 tablet_service.cc:1511] Processing CreateTablet for tablet b776445e68c14533bcde1f784caa703a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:27.340970  6501 tablet_bootstrap.cc:492] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:27.341447  6501 ts_tablet_manager.cc:1403] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.379s	user 0.009s	sys 0.007s
I20260709 12:24:27.334342  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e75b2c5204044d2847e78c187388fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.334874  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0c2d7699e4559823a28a280a0e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.309029  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea0c2d7699e4559823a28a280a0e713 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:27.364069  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5012c92e5bb4fe188dca8f60efceeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.366837  6367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ea11ad8cada4bb682c147f8a26cceb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.367328  6291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ea11ad8cada4bb682c147f8a26cceb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.367121  6449 tablet_service.cc:1511] Processing CreateTablet for tablet e44e2bcbe27f4a9192cc9acd16b966ad (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:27.369340  6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b776445e68c14533bcde1f784caa703a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.370973  6370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d797adfa10541918dfc0ebf49b90784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.371744  6447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53146508e564f8aae279a61e6696e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.373011  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0c2d7699e4559823a28a280a0e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.344472  6504 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:27.379307  6371 tablet_service.cc:1511] Processing CreateTablet for tablet 6ea0c2d7699e4559823a28a280a0e713 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:24:27.381588  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0c2d7699e4559823a28a280a0e713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.373817  6501 raft_consensus.cc:359] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.393533  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44e2bcbe27f4a9192cc9acd16b966ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.402333  6295 tablet_service.cc:1511] Processing CreateTablet for tablet 34bae875d7104f298673c223da6caf2e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:27.387503  6501 raft_consensus.cc:385] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.355773  6448 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4301d25f8649e496bcbcb9f954c74e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:27.408627  6499 tablet_bootstrap.cc:492] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:27.415556  6294 tablet_service.cc:1511] Processing CreateTablet for tablet e44e2bcbe27f4a9192cc9acd16b966ad (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:27.416970  6294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44e2bcbe27f4a9192cc9acd16b966ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.418709  6499 ts_tablet_manager.cc:1403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.300s	user 0.008s	sys 0.013s
I20260709 12:24:27.426038  6448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4301d25f8649e496bcbcb9f954c74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.432446  6441 tablet_service.cc:1511] Processing CreateTablet for tablet 676a90ff0b8a4bb5ba0028f73a5d589f (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:27.433965  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676a90ff0b8a4bb5ba0028f73a5d589f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.437634  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:27.438452  6499 raft_consensus.cc:359] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.453969  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:27.454416  6499 raft_consensus.cc:385] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.454828  6499 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.438702  6500 tablet_bootstrap.cc:654] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.409870  6501 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.456792  6501 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.442910  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bae875d7104f298673c223da6caf2e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:27.443012  6272 leader_election.cc:343] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:27.462853  6274 leader_election.cc:343] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:27.465562  6274 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:27.466499  6505 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:27.481313  6368 tablet_service.cc:1511] Processing CreateTablet for tablet b776445e68c14533bcde1f784caa703a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:24:27.482826  6499 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.501974  6446 tablet_service.cc:1511] Processing CreateTablet for tablet 034bd5c36e71466da14c6a4850a920cb (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:27.502988  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4301d25f8649e496bcbcb9f954c74e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:27.503774  6446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034bd5c36e71466da14c6a4850a920cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.504329  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4301d25f8649e496bcbcb9f954c74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.507462  6372 tablet_service.cc:1511] Processing CreateTablet for tablet 34bae875d7104f298673c223da6caf2e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:24:27.526942  6297 tablet_service.cc:1511] Processing CreateTablet for tablet f53146508e564f8aae279a61e6696e80 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:27.528929  6297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53146508e564f8aae279a61e6696e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.537796  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 676a90ff0b8a4bb5ba0028f73a5d589f (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:27.544186  6501 ts_tablet_manager.cc:1434] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.202s	user 0.039s	sys 0.039s
I20260709 12:24:27.545056  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b776445e68c14533bcde1f784caa703a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.546255  6371 tablet_service.cc:1511] Processing CreateTablet for tablet e44e2bcbe27f4a9192cc9acd16b966ad (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:24:27.547710  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e44e2bcbe27f4a9192cc9acd16b966ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.555809  6499 ts_tablet_manager.cc:1434] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.135s	user 0.005s	sys 0.002s
I20260709 12:24:27.561084  6296 tablet_service.cc:1511] Processing CreateTablet for tablet 034bd5c36e71466da14c6a4850a920cb (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:27.563244  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:27.564163  6369 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4301d25f8649e496bcbcb9f954c74e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:24:27.565498  6369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4301d25f8649e496bcbcb9f954c74e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:27.567044  6424 leader_election.cc:343] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:27.575294  6299 tablet_service.cc:1511] Processing CreateTablet for tablet a96cfc92d6c6455eae7d1204353604ec (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:27.579967  6373 tablet_service.cc:1511] Processing CreateTablet for tablet 034bd5c36e71466da14c6a4850a920cb (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:24:27.580669  6372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bae875d7104f298673c223da6caf2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.586947  6501 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:27.585019  6499 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:27.597345  6292 tablet_service.cc:1511] Processing CreateTablet for tablet f24bf2d0bc904aa495cdb5995b470fb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:27.597703  6444 tablet_service.cc:1511] Processing CreateTablet for tablet a96cfc92d6c6455eae7d1204353604ec (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:27.599146  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96cfc92d6c6455eae7d1204353604ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.599833  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:27.601542  6425 leader_election.cc:343] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:27.602011  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:27.602855  6504 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:27.609408  6440 tablet_service.cc:1511] Processing CreateTablet for tablet f24bf2d0bc904aa495cdb5995b470fb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:27.610165  6366 tablet_service.cc:1511] Processing CreateTablet for tablet a96cfc92d6c6455eae7d1204353604ec (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:24:27.610626  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676a90ff0b8a4bb5ba0028f73a5d589f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.610821  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24bf2d0bc904aa495cdb5995b470fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.611022  6370 tablet_service.cc:1511] Processing CreateTablet for tablet 676a90ff0b8a4bb5ba0028f73a5d589f (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:24:27.612653  6375 tablet_service.cc:1511] Processing CreateTablet for tablet f53146508e564f8aae279a61e6696e80 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:24:27.617573  6373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034bd5c36e71466da14c6a4850a920cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.623327  6293 tablet_service.cc:1511] Processing CreateTablet for tablet 0ddf1d667df04473a598939605e43a1e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:27.624235  6500 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:27.624764  6500 ts_tablet_manager.cc:1403] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.326s	user 0.011s	sys 0.033s
I20260709 12:24:27.627758  6500 raft_consensus.cc:359] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.628494  6500 raft_consensus.cc:385] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.628770  6500 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.629475  6500 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.631753  6367 tablet_service.cc:1511] Processing CreateTablet for tablet f24bf2d0bc904aa495cdb5995b470fb0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:24:27.632643  6500 ts_tablet_manager.cc:1434] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:24:27.633908  6500 tablet_bootstrap.cc:492] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:27.643400  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24bf2d0bc904aa495cdb5995b470fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.643333  6441 tablet_service.cc:1511] Processing CreateTablet for tablet 0ddf1d667df04473a598939605e43a1e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:27.646377  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96cfc92d6c6455eae7d1204353604ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.647205  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf1d667df04473a598939605e43a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.650275  6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034bd5c36e71466da14c6a4850a920cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.650724  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf1d667df04473a598939605e43a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.654384  6499 tablet_bootstrap.cc:654] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.659368  6500 tablet_bootstrap.cc:654] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.674827  6367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24bf2d0bc904aa495cdb5995b470fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.682418  6366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96cfc92d6c6455eae7d1204353604ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.685387  6501 tablet_bootstrap.cc:654] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.686812  6370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 676a90ff0b8a4bb5ba0028f73a5d589f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.700634  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f53146508e564f8aae279a61e6696e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.701855  6296 tablet_service.cc:1511] Processing CreateTablet for tablet dda9448d18794a44a75d0b570e998c3a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:27.703637  6296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda9448d18794a44a75d0b570e998c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.708779  6444 tablet_service.cc:1511] Processing CreateTablet for tablet dda9448d18794a44a75d0b570e998c3a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:27.710081  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda9448d18794a44a75d0b570e998c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.716085  6295 tablet_service.cc:1511] Processing CreateTablet for tablet cdd6e8c2e3c642dc806079ce6070d975 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:27.717376  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd6e8c2e3c642dc806079ce6070d975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.723583  6374 tablet_service.cc:1511] Processing CreateTablet for tablet 0ddf1d667df04473a598939605e43a1e (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:24:27.724825  6374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf1d667df04473a598939605e43a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.740141  6440 tablet_service.cc:1511] Processing CreateTablet for tablet cdd6e8c2e3c642dc806079ce6070d975 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:27.741079  6499 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:27.741465  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd6e8c2e3c642dc806079ce6070d975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.741731  6499 ts_tablet_manager.cc:1403] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.168s	user 0.020s	sys 0.004s
I20260709 12:24:27.744997  6499 raft_consensus.cc:359] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.745819  6499 raft_consensus.cc:385] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.746201  6499 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.747140  6499 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.752856  6499 ts_tablet_manager.cc:1434] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.003s
I20260709 12:24:27.750674  6293 tablet_service.cc:1511] Processing CreateTablet for tablet 55b778a085744d27a1adc35598438230 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:27.755779  6499 tablet_bootstrap.cc:492] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:27.756759  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b778a085744d27a1adc35598438230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.758050  6371 tablet_service.cc:1511] Processing CreateTablet for tablet dda9448d18794a44a75d0b570e998c3a (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:24:27.762809  6375 tablet_service.cc:1511] Processing CreateTablet for tablet cdd6e8c2e3c642dc806079ce6070d975 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:24:27.777386  6448 tablet_service.cc:1511] Processing CreateTablet for tablet 55b778a085744d27a1adc35598438230 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:27.778848  6448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b778a085744d27a1adc35598438230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.783150  6368 tablet_service.cc:1511] Processing CreateTablet for tablet 55b778a085744d27a1adc35598438230 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:24:27.779412  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda9448d18794a44a75d0b570e998c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.780385  6500 tablet_bootstrap.cc:492] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:27.784742  6500 ts_tablet_manager.cc:1403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.151s	user 0.001s	sys 0.032s
I20260709 12:24:27.788013  6297 tablet_service.cc:1511] Processing CreateTablet for tablet 45579ad5e67d4843a6f90bf59939a2d7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:27.795590  6297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45579ad5e67d4843a6f90bf59939a2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.796301  6292 tablet_service.cc:1511] Processing CreateTablet for tablet 4cde8a89c1124a31810e39cb4a6c2518 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:27.788938  6500 raft_consensus.cc:359] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.781121  6501 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:27.797371  6500 raft_consensus.cc:385] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.797586  6501 ts_tablet_manager.cc:1403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.211s	user 0.010s	sys 0.008s
I20260709 12:24:27.797849  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cde8a89c1124a31810e39cb4a6c2518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.797775  6500 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.798955  6500 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.801250  6500 ts_tablet_manager.cc:1434] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:24:27.802350  6500 tablet_bootstrap.cc:492] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:27.826570  6441 tablet_service.cc:1511] Processing CreateTablet for tablet 4cde8a89c1124a31810e39cb4a6c2518 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:27.827836  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cde8a89c1124a31810e39cb4a6c2518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.827920  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 30efe8bf06814d23bf20818fb23139be (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:27.829115  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30efe8bf06814d23bf20818fb23139be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.834187  6504 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.834708  6504 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.836951  6449 tablet_service.cc:1511] Processing CreateTablet for tablet 30efe8bf06814d23bf20818fb23139be (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:27.836299  6371 tablet_service.cc:1511] Processing CreateTablet for tablet 30efe8bf06814d23bf20818fb23139be (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:24:27.838107  6293 tablet_service.cc:1511] Processing CreateTablet for tablet 4a0ddc1149fd4b03af2abd36909591d0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:27.803287  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd6e8c2e3c642dc806079ce6070d975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.839542  6505 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.840327  6505 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.841636  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b778a085744d27a1adc35598438230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.808879  6444 tablet_service.cc:1511] Processing CreateTablet for tablet 45579ad5e67d4843a6f90bf59939a2d7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:27.821450  6370 tablet_service.cc:1511] Processing CreateTablet for tablet 4cde8a89c1124a31810e39cb4a6c2518 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:24:27.823256  6499 tablet_bootstrap.cc:654] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.838371  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30efe8bf06814d23bf20818fb23139be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.847167  6505 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:27.847488  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45579ad5e67d4843a6f90bf59939a2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.805924  6372 tablet_service.cc:1511] Processing CreateTablet for tablet 45579ad5e67d4843a6f90bf59939a2d7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:24:27.802628  6501 raft_consensus.cc:359] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:27.852952  6501 raft_consensus.cc:385] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.853303  6501 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.853947  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:27.836766  6504 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:27.850943  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:27.854066  6501 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:27.851661  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:27.855736  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30efe8bf06814d23bf20818fb23139be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.909471  6516 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.910214  6516 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:27.913293  6370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cde8a89c1124a31810e39cb4a6c2518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.868322  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:27.915788  6310 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:27.916661  6516 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:27.859124  6274 leader_election.cc:343] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:27.897786  6500 tablet_bootstrap.cc:654] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:27.868274  6293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0ddc1149fd4b03af2abd36909591d0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:27.859843  6424 leader_election.cc:343] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:27.929092  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:27.918397  6459 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:27.884809  6440 tablet_service.cc:1511] Processing CreateTablet for tablet 4a0ddc1149fd4b03af2abd36909591d0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:27.932341  6504 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:27.932646  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0ddc1149fd4b03af2abd36909591d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.933581  6516 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.934126  6516 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.934758  6504 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.861177  6369 tablet_service.cc:1511] Processing CreateTablet for tablet 4a0ddc1149fd4b03af2abd36909591d0 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:24:27.937933  6516 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:27.936997  6505 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.938654  6505 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.939203  6295 tablet_service.cc:1511] Processing CreateTablet for tablet 64dfdc039c8140e5bb288a15c2e827ce (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:27.940562  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64dfdc039c8140e5bb288a15c2e827ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.940639  6505 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:27.937517  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:24:27.927430  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67633.4us
I20260709 12:24:27.942775  6505 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:27.943153  6505 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.943486  6505 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:27.923377  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64249.9us
I20260709 12:24:27.904006  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 7db3e095e8b44b23b1a439899d305bd7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:27.945845  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e095e8b44b23b1a439899d305bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.935230  6504 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:27.949787  6499 tablet_bootstrap.cc:492] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:27.950359  6499 ts_tablet_manager.cc:1403] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.195s	user 0.016s	sys 0.002s
I20260709 12:24:27.950902  6292 tablet_service.cc:1511] Processing CreateTablet for tablet 1b749ec31e454255bbbba4a4bb493f28 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:27.952266  6292 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b749ec31e454255bbbba4a4bb493f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.953442  6499 raft_consensus.cc:359] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.936570  6372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45579ad5e67d4843a6f90bf59939a2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.954958  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:27.956238  6272 leader_election.cc:343] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:27.887346  6366 tablet_service.cc:1511] Processing CreateTablet for tablet 7db3e095e8b44b23b1a439899d305bd7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:27.959034  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 150319986ee143d5a746f901c5b4adcc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:27.959893  6366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e095e8b44b23b1a439899d305bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:27.960338  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150319986ee143d5a746f901c5b4adcc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:27.963498  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:27.967628  6505 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.965694  6504 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.954317  6499 raft_consensus.cc:385] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:27.970794  6499 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:27.974233  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:27.975086  6310 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:27.976835  6499 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:27.981280  6504 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:27.984180  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:27.990595  6505 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:27.997922  6369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0ddc1149fd4b03af2abd36909591d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.010385  6459 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:28.018889  6290 tablet_service.cc:1511] Processing CreateTablet for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:28.020416  6290 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.023377  6501 ts_tablet_manager.cc:1434] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.225s	user 0.005s	sys 0.020s
I20260709 12:24:28.030697  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.030797  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.032488  6501 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.029742  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.014680  6505 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.035060  6505 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.032778  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:28.033666  6274 leader_election.cc:343] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:28.038074  6351 leader_election.cc:343] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.049088  6374 tablet_service.cc:1511] Processing CreateTablet for tablet 1b749ec31e454255bbbba4a4bb493f28 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:28.049557  6373 tablet_service.cc:1511] Processing CreateTablet for tablet 150319986ee143d5a746f901c5b4adcc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:28.050657  6374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b749ec31e454255bbbba4a4bb493f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.050750  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.051378  6373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150319986ee143d5a746f901c5b4adcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.049019  6375 tablet_service.cc:1511] Processing CreateTablet for tablet 64dfdc039c8140e5bb288a15c2e827ce (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:28.052428  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.053500  6527 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.058908  6375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64dfdc039c8140e5bb288a15c2e827ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.063695  6501 tablet_bootstrap.cc:654] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.064424  6367 tablet_service.cc:1511] Processing CreateTablet for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:28.065774  6367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.067984  6504 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.068661  6504 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:28.073036  6424 leader_election.cc:343] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.073418  6504 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.073493  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:28.075846  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.076505  6310 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.077234  6504 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.077775  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.078786  6504 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.079656  6505 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.080632  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.081523  6459 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:28.079353  6504 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.083086  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.083252  6504 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.084019  6505 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.084409  6505 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.084758  6505 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.071321  6500 tablet_bootstrap.cc:492] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.089171  6441 tablet_service.cc:1511] Processing CreateTablet for tablet 64dfdc039c8140e5bb288a15c2e827ce (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:24:28.089884  6500 ts_tablet_manager.cc:1403] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.288s	user 0.016s	sys 0.016s
I20260709 12:24:28.090570  6441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64dfdc039c8140e5bb288a15c2e827ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.090364  6448 tablet_service.cc:1511] Processing CreateTablet for tablet 1b749ec31e454255bbbba4a4bb493f28 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:24:28.092620  6440 tablet_service.cc:1511] Processing CreateTablet for tablet 7db3e095e8b44b23b1a439899d305bd7 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:24:28.092293  6448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b749ec31e454255bbbba4a4bb493f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.089171  6298 tablet_service.cc:1511] Processing CreateTablet for tablet 3b2f25afefb44300a1d6297e2cc140f5 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:28.095705  6298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2f25afefb44300a1d6297e2cc140f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.108999  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:28.110383  6424 leader_election.cc:343] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:28.098666  6351 leader_election.cc:343] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:28.111722  6274 leader_election.cc:343] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.112159  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.113312  6527 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.116066  6500 raft_consensus.cc:359] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.116858  6500 raft_consensus.cc:385] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.117159  6500 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.117211  6440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e095e8b44b23b1a439899d305bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.124543  6295 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddf475f266c47adb1a50aa986a0c44c (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:28.125463  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.126461  6295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddf475f266c47adb1a50aa986a0c44c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:24:28.127274  6274 leader_election.cc:343] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.125358  6504 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.129360  6500 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.131357  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.131968  6310 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.132393  6371 tablet_service.cc:1511] Processing CreateTablet for tablet 3b2f25afefb44300a1d6297e2cc140f5 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:28.134034  6449 tablet_service.cc:1511] Processing CreateTablet for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:24:28.135236  6449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.136444  6444 tablet_service.cc:1511] Processing CreateTablet for tablet 150319986ee143d5a746f901c5b4adcc (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:24:28.138211  6444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150319986ee143d5a746f901c5b4adcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.140594  6445 tablet_service.cc:1511] Processing CreateTablet for tablet 3b2f25afefb44300a1d6297e2cc140f5 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:24:28.141949  6445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2f25afefb44300a1d6297e2cc140f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.142827  6505 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.143991  6446 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddf475f266c47adb1a50aa986a0c44c (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:28.144336  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.145179  6446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddf475f266c47adb1a50aa986a0c44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.145882  6505 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.145882  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.147555  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:28.147679  6458 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.148197  6459 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.132426  6368 tablet_service.cc:1511] Processing CreateTablet for tablet 2ddf475f266c47adb1a50aa986a0c44c (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:24:28.154592  6371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2f25afefb44300a1d6297e2cc140f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.155375  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:28.155670  6348 leader_election.cc:343] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.156147  6348 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.157189  6524 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.158015  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:28.158849  6524 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.159215  6524 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.159544  6524 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.159595  6368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddf475f266c47adb1a50aa986a0c44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.167500  6499 ts_tablet_manager.cc:1434] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.217s	user 0.039s	sys 0.024s
W20260709 12:24:28.168481  6274 leader_election.cc:343] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.171808  6459 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:28.173044  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:28.178234  6499 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.179400  6442 tablet_service.cc:1511] Processing CreateTablet for tablet 2483d7801887403e873adc62429fe046 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:28.180775  6442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2483d7801887403e873adc62429fe046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.188313  6524 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.189155  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:28.192461  6424 leader_election.cc:343] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.192812  6310 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:28.191704  6500 ts_tablet_manager.cc:1434] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.008s
I20260709 12:24:28.193286  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.194203  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:28.194535  6500 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:28.194968  6351 leader_election.cc:343] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.201649  6500 tablet_bootstrap.cc:654] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.204963  6530 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.205475  6530 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:28.206444  6530 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.185391  6505 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.217185  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:28.218113  6459 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.221588  6299 tablet_service.cc:1511] Processing CreateTablet for tablet 2483d7801887403e873adc62429fe046 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:28.223526  6501 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.224086  6501 ts_tablet_manager.cc:1403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.192s	user 0.006s	sys 0.018s
I20260709 12:24:28.192063  6504 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.226572  6524 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.227720  6501 raft_consensus.cc:359] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.228471  6501 raft_consensus.cc:385] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.228762  6501 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.229528  6501 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.234375  6505 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:28.239626  6505 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.246987  6299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2483d7801887403e873adc62429fe046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.249373  6499 tablet_bootstrap.cc:654] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.250844  6366 tablet_service.cc:1511] Processing CreateTablet for tablet 2483d7801887403e873adc62429fe046 (DEFAULT_TABLE table=test-workload [id=caa276ebe4814c8197f43c3434fd5228]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:24:28.251183  6501 ts_tablet_manager.cc:1434] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
W20260709 12:24:28.254808  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60709.4us
I20260709 12:24:28.255026  6366 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2483d7801887403e873adc62429fe046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:28.266840  6501 tablet_bootstrap.cc:492] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.277200  6500 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.274019  6459 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:28.277732  6500 ts_tablet_manager.cc:1403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.084s	user 0.009s	sys 0.003s
I20260709 12:24:28.280862  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:28.281509  6500 raft_consensus.cc:359] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.282310  6500 raft_consensus.cc:385] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.282282  6524 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.282728  6500 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.283067  6524 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:28.283694  6500 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.284013  6524 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.286396  6500 ts_tablet_manager.cc:1434] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:24:28.287592  6500 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.298030  6204 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.307842  6501 tablet_bootstrap.cc:654] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.318864  6500 tablet_bootstrap.cc:654] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.323894  6205 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.330133  6499 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.330687  6499 ts_tablet_manager.cc:1403] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.153s	user 0.009s	sys 0.026s
I20260709 12:24:28.335409  6501 tablet_bootstrap.cc:492] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.336621  6501 ts_tablet_manager.cc:1403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.070s	user 0.020s	sys 0.005s
I20260709 12:24:28.335402  6500 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.339545  6500 ts_tablet_manager.cc:1403] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.052s	user 0.016s	sys 0.000s
I20260709 12:24:28.340265  6499 raft_consensus.cc:359] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.341032  6499 raft_consensus.cc:385] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.341317  6499 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.342047  6499 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.344240  6499 ts_tablet_manager.cc:1434] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 12:24:28.345072  6499 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.344448  6501 raft_consensus.cc:359] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.345503  6501 raft_consensus.cc:385] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.345901  6501 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.346987  6501 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.349231  6501 ts_tablet_manager.cc:1434] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:24:28.350600  6500 raft_consensus.cc:359] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.351330  6500 raft_consensus.cc:385] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.351645  6499 tablet_bootstrap.cc:654] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.351635  6500 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.352619  6500 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.353808  6501 tablet_bootstrap.cc:492] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.353257  6206 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.358227  6500 ts_tablet_manager.cc:1434] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:24:28.360306  6505 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.360956  6505 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.362393  6501 tablet_bootstrap.cc:654] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.364821  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.365899  6500 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:28.367769  6274 leader_election.cc:343] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.372449  6500 tablet_bootstrap.cc:654] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.373548  6505 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.374656  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:28.376051  6272 leader_election.cc:343] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.376495  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.377700  6530 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.378504  6530 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.377990  6505 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.380735  6530 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.382434  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.383857  6384 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.385522  6501 tablet_bootstrap.cc:492] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.387598  6501 ts_tablet_manager.cc:1403] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.034s	user 0.007s	sys 0.005s
I20260709 12:24:28.385582  6424 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.390832  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.393131  6530 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.393618  6530 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.393983  6530 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.395951  6499 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.396522  6499 ts_tablet_manager.cc:1403] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.052s	user 0.019s	sys 0.017s
I20260709 12:24:28.396202  6501 raft_consensus.cc:359] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:28.396742  6425 leader_election.cc:343] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.396958  6501 raft_consensus.cc:385] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.397269  6501 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.397987  6501 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.399565  6499 raft_consensus.cc:359] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.400346  6500 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.400377  6499 raft_consensus.cc:385] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.400858  6499 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.401089  6500 ts_tablet_manager.cc:1403] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.000s
I20260709 12:24:28.401816  6499 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.402699  6530 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.404426  6500 raft_consensus.cc:359] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.405211  6500 raft_consensus.cc:385] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.405431  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.405579  6500 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
W20260709 12:24:28.406870  6425 leader_election.cc:343] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.406594  6500 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.408828  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.409274  6501 ts_tablet_manager.cc:1434] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:24:28.409400  6384 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.409446  6500 ts_tablet_manager.cc:1434] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:24:28.410741  6500 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.412824  6499 ts_tablet_manager.cc:1434] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:24:28.416739  6384 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:28.417099  6501 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.417971  6500 tablet_bootstrap.cc:654] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.417918  6424 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:28.419073  6530 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.419245  6531 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.419812  6531 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:28.420769  6499 tablet_bootstrap.cc:492] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.429291  6500 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.420698  6531 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.429648  6500 ts_tablet_manager.cc:1403] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.006s
I20260709 12:24:28.432137  6500 raft_consensus.cc:359] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.432505  6501 tablet_bootstrap.cc:654] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.432845  6500 raft_consensus.cc:385] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.433241  6500 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.433916  6500 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.436131  6500 ts_tablet_manager.cc:1434] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:28.436305  6499 tablet_bootstrap.cc:654] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.437017  6500 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.444486  6500 tablet_bootstrap.cc:654] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.446924  6501 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.447557  6501 ts_tablet_manager.cc:1403] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.005s
I20260709 12:24:28.450955  6206 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.452064  6501 raft_consensus.cc:359] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.452752  6501 raft_consensus.cc:385] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.453051  6501 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.453969  6499 tablet_bootstrap.cc:492] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.453774  6501 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.453948  6500 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.454527  6499 ts_tablet_manager.cc:1403] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.008s
I20260709 12:24:28.455184  6500 ts_tablet_manager.cc:1403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 12:24:28.458091  6500 raft_consensus.cc:359] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.458922  6501 ts_tablet_manager.cc:1434] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:24:28.458807  6499 raft_consensus.cc:359] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.459728  6499 raft_consensus.cc:385] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.460115  6501 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.460058  6499 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.460893  6499 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.463686  6499 ts_tablet_manager.cc:1434] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:28.464816  6499 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.458796  6500 raft_consensus.cc:385] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.466962  6500 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.467674  6500 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.469766  6500 ts_tablet_manager.cc:1434] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.014s	user 0.003s	sys 0.000s
I20260709 12:24:28.470708  6500 tablet_bootstrap.cc:492] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.475967  6499 tablet_bootstrap.cc:654] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.476545  6500 tablet_bootstrap.cc:654] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.483825  6501 tablet_bootstrap.cc:654] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.486084  6500 tablet_bootstrap.cc:492] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.486330  6499 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.486624  6500 ts_tablet_manager.cc:1403] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.016s	user 0.012s	sys 0.003s
I20260709 12:24:28.486934  6499 ts_tablet_manager.cc:1403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 12:24:28.489023  6500 raft_consensus.cc:359] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.489538  6501 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.489547  6500 raft_consensus.cc:385] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.490214  6500 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.490248  6501 ts_tablet_manager.cc:1403] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.004s
I20260709 12:24:28.489786  6499 raft_consensus.cc:359] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.490710  6499 raft_consensus.cc:385] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.491029  6499 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.490895  6500 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.491947  6499 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.493156  6500 ts_tablet_manager.cc:1434] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:24:28.493403  6501 raft_consensus.cc:359] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.494067  6501 raft_consensus.cc:385] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.494266  6500 tablet_bootstrap.cc:492] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.494346  6501 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.495445  6501 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.498078  6499 ts_tablet_manager.cc:1434] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.002s
I20260709 12:24:28.499045  6499 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.500550  6500 tablet_bootstrap.cc:654] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.508971  6501 ts_tablet_manager.cc:1434] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 12:24:28.512382  6499 tablet_bootstrap.cc:654] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.515583  6501 tablet_bootstrap.cc:492] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.519868  6499 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.520305  6499 ts_tablet_manager.cc:1403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.001s
I20260709 12:24:28.521075  6500 tablet_bootstrap.cc:492] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.521595  6500 ts_tablet_manager.cc:1403] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.027s	user 0.022s	sys 0.003s
I20260709 12:24:28.522730  6505 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.523414  6501 tablet_bootstrap.cc:654] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.523216  6505 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:28.524243  6500 raft_consensus.cc:359] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.525063  6500 raft_consensus.cc:385] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.525388  6500 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.525578  6505 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.526443  6499 raft_consensus.cc:359] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.526645  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.527377  6499 raft_consensus.cc:385] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.527275  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.527863  6499 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.528177  6384 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:28.528730  6499 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.529515  6274 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.529101  6500 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.530861  6505 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.531136  6499 ts_tablet_manager.cc:1434] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:24:28.532195  6499 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.538337  6499 tablet_bootstrap.cc:654] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.542272  6500 ts_tablet_manager.cc:1434] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.002s
I20260709 12:24:28.543555  6524 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.544062  6524 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.546205  6524 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.547024  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.547781  6531 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.548089  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.548918  6459 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.548689  6531 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.550411  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.551403  6524 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.551558  6531 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.547899  6310 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.551862  6524 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.552428  6524 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.552663  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.553351  6384 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.554006  6500 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.553905  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.554859  6310 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.554997  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.555845  6531 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.556200  6531 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.556519  6531 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.559787  6505 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.560537  6500 tablet_bootstrap.cc:654] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.561214  6524 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.563947  6499 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.564464  6499 ts_tablet_manager.cc:1403] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.000s
I20260709 12:24:28.564281  6531 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.565415  6500 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.565852  6500 ts_tablet_manager.cc:1403] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.003s
I20260709 12:24:28.567389  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.567960  6310 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.568502  6500 raft_consensus.cc:359] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.569144  6501 tablet_bootstrap.cc:492] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.569149  6500 raft_consensus.cc:385] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.569583  6500 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.569942  6531 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.569643  6501 ts_tablet_manager.cc:1403] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.022s
I20260709 12:24:28.570474  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:28.570389  6500 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.570967  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.571138  6459 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.571557  6309 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.572362  6500 ts_tablet_manager.cc:1434] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:24:28.572635  6501 raft_consensus.cc:359] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.573256  6501 raft_consensus.cc:385] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.573511  6501 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.574283  6501 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.567169  6499 raft_consensus.cc:359] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.576272  6501 ts_tablet_manager.cc:1434] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:28.576247  6500 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.576567  6499 raft_consensus.cc:385] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.576872  6499 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.577487  6499 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.578856  6459 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:28.579389  6499 ts_tablet_manager.cc:1434] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:24:28.579926  6524 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.580157  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.581210  6524 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.581889  6524 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:28.582947  6500 tablet_bootstrap.cc:654] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.582927  6524 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.567400  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.594894  6384 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.597214  6500 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.597852  6500 ts_tablet_manager.cc:1403] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.002s
I20260709 12:24:28.598654  6499 tablet_bootstrap.cc:492] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.600330  6500 raft_consensus.cc:359] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.600983  6500 raft_consensus.cc:385] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.601248  6500 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.604636  6499 tablet_bootstrap.cc:654] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.604557  6500 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.604979  6309 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:28.608521  6384 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:28.609443  6500 ts_tablet_manager.cc:1434] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:24:28.609978  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.610352  6500 tablet_bootstrap.cc:492] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.613078  6531 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.613700  6531 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:28.611862  6501 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.614653  6531 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.616436  6500 tablet_bootstrap.cc:654] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.622102  6501 tablet_bootstrap.cc:654] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.612933  6310 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:28.610816  6499 tablet_bootstrap.cc:492] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.626780  6499 ts_tablet_manager.cc:1403] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.000s
I20260709 12:24:28.629304  6205 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:28.635735  6499 raft_consensus.cc:359] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.636734  6499 raft_consensus.cc:385] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.637079  6499 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.638096  6499 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.651166  6499 ts_tablet_manager.cc:1434] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:24:28.651979  6499 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.655954  6499 tablet_bootstrap.cc:654] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.661026  6500 tablet_bootstrap.cc:492] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.667520  6501 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:28.669312  6505 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.658990  6524 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.676088  6500 ts_tablet_manager.cc:1403] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.066s	user 0.022s	sys 0.027s
I20260709 12:24:28.676192  6505 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.676783  6524 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.679560  6505 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.680152  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.657706  6206 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:28.680780  6384 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:28.681797  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.682475  6310 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.682030  6500 raft_consensus.cc:359] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.683084  6500 raft_consensus.cc:385] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.683357  6516 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.683476  6500 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.683997  6531 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:28.684933  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.683924  6516 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.684653  6500 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.685575  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:28.687448  6274 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:28.687659  6500 ts_tablet_manager.cc:1434] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:24:28.687809  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:28.688378  6516 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.688943  6516 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.689209  6500 tablet_bootstrap.cc:492] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.689302  6516 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.689620  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.694967  6516 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.695268  6310 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.693979  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.697443  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.694190  6531 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.698797  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.698994  6531 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.700580  6524 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.701138  6524 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.701505  6524 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.702432  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.702641  6500 tablet_bootstrap.cc:654] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.702839  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.676156  6501 ts_tablet_manager.cc:1403] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.034s
I20260709 12:24:28.701869  6524 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.703596  6459 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:28.706017  6531 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.706302  6505 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.706910  6505 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.707356  6505 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.707952  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.708561  6310 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.709836  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.710731  6531 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.711099  6531 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.711440  6531 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.716666  6528 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:28.724328  6531 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.726521  6531 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:28.729125  6424 consensus_peers.cc:597] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:28.730172  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.734282  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.735064  6310 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.783147  6531 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.784168  6568 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.783856  6531 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.785197  6499 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.785487  6500 tablet_bootstrap.cc:492] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:28.785754  6499 ts_tablet_manager.cc:1403] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.134s	user 0.033s	sys 0.070s
I20260709 12:24:28.786075  6500 ts_tablet_manager.cc:1403] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.097s	user 0.052s	sys 0.023s
I20260709 12:24:28.788323  6504 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.788532  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.788897  6504 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.789156  6499 raft_consensus.cc:359] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.789832  6499 raft_consensus.cc:385] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.790169  6499 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.790521  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.791178  6309 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.791316  6504 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.790961  6499 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:28.790185  6424 leader_election.cc:343] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.792698  6425 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:28.793105  6500 raft_consensus.cc:359] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.793857  6504 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.793959  6499 ts_tablet_manager.cc:1434] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.005s
I20260709 12:24:28.794010  6500 raft_consensus.cc:385] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.794549  6504 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.794715  6500 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.794999  6504 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.795509  6499 tablet_bootstrap.cc:492] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.795588  6500 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.794034  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:28.799369  6424 leader_election.cc:343] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.803666  6499 tablet_bootstrap.cc:654] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.804610  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.805775  6309 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:28.807422  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:28.786154  6531 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.810110  6531 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:28.810581  6531 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.811043  6531 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.787777  6516 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.823339  6384 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 1: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:24:28.804925  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 117215us
I20260709 12:24:28.831530  6501 raft_consensus.cc:359] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.832612  6501 raft_consensus.cc:385] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.832959  6501 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.831920  6568 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.836480  6568 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.837333  6580 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.837926  6580 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.842913  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.843952  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:28.845070  6458 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:28.850639  6384 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.851269  6499 tablet_bootstrap.cc:492] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.851802  6499 ts_tablet_manager.cc:1403] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.057s	user 0.014s	sys 0.007s
I20260709 12:24:28.854804  6499 raft_consensus.cc:359] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.855535  6499 raft_consensus.cc:385] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.855837  6499 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.858355  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.859309  6459 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:28.860738  6580 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.861526  6499 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.864267  6499 ts_tablet_manager.cc:1434] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:24:28.865548  6499 tablet_bootstrap.cc:492] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:28.870394  6530 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.871124  6530 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.873819  6499 tablet_bootstrap.cc:654] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.878127  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.846112  6579 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 12:24:28.881631  6501 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.886847  6531 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.888559  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:28.889720  6309 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:28.889714  6531 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.890974  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.893364  6505 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.898670  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:28.933388  6424 leader_election.cc:343] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.901121  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.934036  6307 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.904433  6528 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.913822  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:28.934891  6425 leader_election.cc:343] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.935043  6528 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.936110  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.937223  6499 tablet_bootstrap.cc:492] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:28.937294  6580 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.937626  6305 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.938162  6579 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.916126  6504 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.940805  6504 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:28.917439  6582 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.943940  6582 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.946408  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.946647  6582 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.885357  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:28.947069  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:28.878131  6516 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.947072  6309 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.937170  6528 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.950599  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.951373  6307 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
W20260709 12:24:28.953411  6272 leader_election.cc:343] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.953711  6505 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.954851  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:28.956298  6531 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.957304  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:28.957952  6531 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:28.957975  6457 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.928831  6500 ts_tablet_manager.cc:1434] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.140s	user 0.043s	sys 0.083s
I20260709 12:24:28.959004  6309 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:28.929354  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.960093  6500 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:28.929958  6530 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:28.960887  6424 leader_election.cc:343] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:28.961414  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:28.962708  6530 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.963200  6425 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.964299  6530 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.964807  6530 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:28.965613  6530 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.966558  6500 tablet_bootstrap.cc:654] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.959188  6531 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.925796  6310 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:28.937844  6499 ts_tablet_manager.cc:1403] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.073s	user 0.033s	sys 0.028s
I20260709 12:24:28.944355  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:28.947741  6385 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.969399  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:28.970369  6499 raft_consensus.cc:359] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.971195  6499 raft_consensus.cc:385] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:28.971179  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.971796  6499 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:28.972325  6516 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:28.929261  6524 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.972600  6499 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.973078  6516 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:28.915252  6578 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:28.925261  6501 ts_tablet_manager.cc:1434] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.221s	user 0.006s	sys 0.002s
W20260709 12:24:28.966945  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:28.975044  6524 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:28.975597  6501 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:28.979890  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:28.981189  6499 ts_tablet_manager.cc:1434] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.001s
I20260709 12:24:28.981791  6504 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:28.982378  6504 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:28.988758  6579 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.007023  6574 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.006973  6504 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.007721  6574 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:28.989545  6524 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:24:28.995594  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.011497  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.012537  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:29.001585  6576 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:24:29.014793  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:28.994355  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:28.997975  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:29.016227  6309 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:24:29.016533  6274 leader_election.cc:343] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.017066  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:29.017774  6576 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.018323  6576 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.018656  6578 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.018877  6576 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:28.996224  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:29.020599  6382 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:29.021275  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.993767  6385 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:29.023248  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:28.989143  6457 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:29.016295  6500 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.026298  6500 ts_tablet_manager.cc:1403] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.020s
I20260709 12:24:29.029639  6272 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:29.022372  6579 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.034641  6578 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:29.035287  6578 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:29.036509  6578 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.039350  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.039429  6500 raft_consensus.cc:359] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.040586  6500 raft_consensus.cc:385] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.040992  6500 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
W20260709 12:24:29.041069  6424 leader_election.cc:343] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.042605  6574 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.046025  6501 tablet_bootstrap.cc:654] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:28.982298  6499 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.060151  6574 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:29.061745  6528 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.062525  6528 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.065042  6528 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.068312  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:29.070619  6272 leader_election.cc:343] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.070636  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:29.072527  6274 leader_election.cc:343] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.073082  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:29.061868  6500 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.046727  6576 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.085090  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:29.085812  6305 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.089643  6505 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.094676  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:29.097936  6500 ts_tablet_manager.cc:1434] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.008s
I20260709 12:24:29.096565  6501 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.103206  6501 ts_tablet_manager.cc:1403] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.128s	user 0.018s	sys 0.001s
I20260709 12:24:29.101267  6499 tablet_bootstrap.cc:654] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.077559  6578 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.114920  6500 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:29.117941  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.114223  6576 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.122825  6596 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:29.121193  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:29.123601  6596 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:29.123819  6530 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:29.127094  6524 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:29.128374  6424 leader_election.cc:343] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.130637  6205 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:29.139062  6206 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:29.126868  6596 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.137261  6205 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:29.144075  6530 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.145730  6530 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.147333  6501 raft_consensus.cc:359] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.148247  6501 raft_consensus.cc:385] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.148767  6501 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.148301  6530 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.152321  6500 tablet_bootstrap.cc:654] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.149829  6501 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.154783  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.156545  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:29.157022  6425 leader_election.cc:343] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.159333  6501 ts_tablet_manager.cc:1434] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.000s
I20260709 12:24:29.159811  6530 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:29.159196  6424 leader_election.cc:343] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.161911  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:29.162847  6530 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.170045  6501 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.180233  6530 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.180783  6530 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.196301  6524 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.196887  6524 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.199010  6524 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.200546  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.204999  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.206545  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:29.207665  6524 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.212110  6530 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.213294  6499 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.213794  6499 ts_tablet_manager.cc:1403] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.232s	user 0.008s	sys 0.009s
I20260709 12:24:29.213303  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:29.215826  6425 leader_election.cc:343] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.216531  6499 raft_consensus.cc:359] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.216763  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.217298  6499 raft_consensus.cc:385] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.217657  6499 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
W20260709 12:24:29.218741  6424 leader_election.cc:343] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.195755  6501 tablet_bootstrap.cc:654] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.219347  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:29.220314  6530 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.227376  6499 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.228700  6500 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.229219  6500 ts_tablet_manager.cc:1403] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.125s	user 0.016s	sys 0.012s
I20260709 12:24:29.229434  6499 ts_tablet_manager.cc:1434] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 12:24:29.230391  6499 tablet_bootstrap.cc:492] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.232113  6500 raft_consensus.cc:359] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.232795  6500 raft_consensus.cc:385] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.233084  6500 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.234098  6500 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.236816  6500 ts_tablet_manager.cc:1434] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:29.237732  6500 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:29.239238  6499 tablet_bootstrap.cc:654] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.242703  6501 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.243180  6501 ts_tablet_manager.cc:1403] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.076s	user 0.013s	sys 0.000s
I20260709 12:24:29.247269  6501 raft_consensus.cc:359] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.247917  6501 raft_consensus.cc:385] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.248139  6501 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.246301  6203 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:29.248839  6501 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.251019  6500 tablet_bootstrap.cc:654] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.250550  6203 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:29.251983  6501 ts_tablet_manager.cc:1434] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:29.252832  6501 tablet_bootstrap.cc:492] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.261035  6500 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.261499  6500 ts_tablet_manager.cc:1403] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.024s	user 0.005s	sys 0.009s
I20260709 12:24:29.261889  6501 tablet_bootstrap.cc:654] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.263645  6596 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.264180  6596 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.264569  6500 raft_consensus.cc:359] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.265388  6500 raft_consensus.cc:385] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.265723  6500 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.266546  6596 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.266781  6500 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.268714  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.269655  6500 ts_tablet_manager.cc:1434] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:24:29.270877  6500 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:29.270922  6424 leader_election.cc:343] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.271412  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.272168  6596 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:29.272922  6425 leader_election.cc:343] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.272738  6596 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.273362  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:29.274477  6504 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.275076  6596 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.276183  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.276494  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:29.277805  6424 leader_election.cc:343] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:29.278826  6425 leader_election.cc:343] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.279201  6500 tablet_bootstrap.cc:654] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.279410  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:29.281917  6596 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.283720  6499 tablet_bootstrap.cc:492] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.284296  6499 ts_tablet_manager.cc:1403] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.054s	user 0.008s	sys 0.011s
I20260709 12:24:29.287220  6499 raft_consensus.cc:359] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.288079  6499 raft_consensus.cc:385] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.288419  6499 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.289294  6499 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.300905  6596 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.301595  6499 ts_tablet_manager.cc:1434] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.002s
I20260709 12:24:29.301440  6596 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.304774  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:29.306268  6424 leader_election.cc:343] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.307523  6500 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.308092  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.308799  6596 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.308069  6500 ts_tablet_manager.cc:1403] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.003s
I20260709 12:24:29.309073  6381 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:29.309865  6425 leader_election.cc:343] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.310341  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:29.311216  6524 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.311429  6499 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.311705  6524 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.313820  6524 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.316430  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.316475  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.317438  6307 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:29.318586  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:29.319211  6499 tablet_bootstrap.cc:654] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.321951  6500 raft_consensus.cc:359] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.322929  6500 raft_consensus.cc:385] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.323275  6500 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.324676  6500 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.326831  6579 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.330958  6499 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.331525  6499 ts_tablet_manager.cc:1403] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 12:24:29.332801  6501 tablet_bootstrap.cc:492] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.333282  6501 ts_tablet_manager.cc:1403] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.081s	user 0.040s	sys 0.030s
I20260709 12:24:29.333758  6499 raft_consensus.cc:359] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.334296  6499 raft_consensus.cc:385] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.334553  6499 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.335016  6579 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.335371  6499 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.335914  6501 raft_consensus.cc:359] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.336261  6530 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:29.336602  6501 raft_consensus.cc:385] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.336908  6501 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.337935  6500 ts_tablet_manager.cc:1434] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:24:29.337610  6501 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.339547  6500 tablet_bootstrap.cc:492] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:29.339800  6501 ts_tablet_manager.cc:1434] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:24:29.340705  6501 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.342303  6499 ts_tablet_manager.cc:1434] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:24:29.343353  6499 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.353735  6499 tablet_bootstrap.cc:654] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.354158  6501 tablet_bootstrap.cc:654] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.366819  6500 tablet_bootstrap.cc:654] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.368940  6501 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.369457  6501 ts_tablet_manager.cc:1403] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.005s
I20260709 12:24:29.372408  6501 raft_consensus.cc:359] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.373047  6499 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.373102  6501 raft_consensus.cc:385] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.373512  6501 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.374202  6501 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.374758  6499 ts_tablet_manager.cc:1403] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.007s
I20260709 12:24:29.378149  6501 ts_tablet_manager.cc:1434] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:29.379146  6501 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.380074  6499 raft_consensus.cc:359] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.381109  6499 raft_consensus.cc:385] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.381666  6499 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.382592  6499 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.385157  6499 ts_tablet_manager.cc:1434] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:29.386709  6499 tablet_bootstrap.cc:492] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.387372  6501 tablet_bootstrap.cc:654] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.392905  6500 tablet_bootstrap.cc:492] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.393038  6499 tablet_bootstrap.cc:654] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.393427  6500 ts_tablet_manager.cc:1403] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.014s
I20260709 12:24:29.400123  6500 raft_consensus.cc:359] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.400940  6500 raft_consensus.cc:385] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.401283  6500 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.402117  6500 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.408298  6596 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:29.420886  6501 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.421435  6501 ts_tablet_manager.cc:1403] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.019s
I20260709 12:24:29.422773  6500 ts_tablet_manager.cc:1434] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 12:24:29.423539  6499 tablet_bootstrap.cc:492] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.424007  6499 ts_tablet_manager.cc:1403] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.011s
I20260709 12:24:29.424417  6501 raft_consensus.cc:359] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.425338  6501 raft_consensus.cc:385] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.425595  6501 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.426729  6499 raft_consensus.cc:359] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.426594  6501 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.427580  6499 raft_consensus.cc:385] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.428195  6500 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:29.429652  6501 ts_tablet_manager.cc:1434] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:24:29.431090  6578 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.431315  6501 tablet_bootstrap.cc:492] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.431643  6578 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.435941  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.436673  6457 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:29.428020  6499 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.438756  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.438534  6499 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.440508  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.441493  6505 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.441746  6499 ts_tablet_manager.cc:1434] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.000s
I20260709 12:24:29.441798  6596 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:29.442625  6274 leader_election.cc:343] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.441895  6505 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.443310  6578 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.443867  6505 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.443190  6499 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.456073  6500 tablet_bootstrap.cc:654] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.462144  6505 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.464092  6499 tablet_bootstrap.cc:654] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.470394  6596 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:29.471019  6501 tablet_bootstrap.cc:654] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.486861  6505 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.488938  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:29.489116  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:29.490065  6457 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:29.493263  6274 leader_election.cc:343] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.494006  6500 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.494779  6500 ts_tablet_manager.cc:1403] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.067s	user 0.021s	sys 0.009s
I20260709 12:24:29.496712  6505 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.497243  6505 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.499540  6505 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.500631  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.501760  6500 raft_consensus.cc:359] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.502545  6500 raft_consensus.cc:385] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.502825  6500 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.503444  6500 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.504261  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.505156  6459 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:29.505481  6500 ts_tablet_manager.cc:1434] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 12:24:29.506558  6500 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:29.507830  6274 leader_election.cc:343] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.508852  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:29.513718  6505 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.514168  6505 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.514225  6457 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:29.514605  6505 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.515568  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:29.516713  6578 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:29.517239  6578 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:29.518090  6578 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.522384  6500 tablet_bootstrap.cc:654] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.525405  6499 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.525929  6499 ts_tablet_manager.cc:1403] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.083s	user 0.022s	sys 0.004s
I20260709 12:24:29.528095  6501 tablet_bootstrap.cc:492] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.528640  6501 ts_tablet_manager.cc:1403] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.098s	user 0.009s	sys 0.014s
I20260709 12:24:29.529239  6500 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.529671  6500 ts_tablet_manager.cc:1403] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.003s
I20260709 12:24:29.531368  6501 raft_consensus.cc:359] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.532092  6501 raft_consensus.cc:385] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.532044  6500 raft_consensus.cc:359] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.532508  6501 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.532802  6500 raft_consensus.cc:385] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.533247  6500 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.533347  6501 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.534008  6500 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.522176  6505 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.536545  6500 ts_tablet_manager.cc:1434] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:24:29.539077  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:29.539350  6501 ts_tablet_manager.cc:1434] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:24:29.540858  6501 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.541711  6574 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:29.544499  6499 raft_consensus.cc:359] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.545289  6499 raft_consensus.cc:385] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.545579  6499 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.537580  6500 tablet_bootstrap.cc:492] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:29.532708  6530 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:24:29.546255  6499 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:29.550408  6274 leader_election.cc:343] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.553238  6579 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.553735  6579 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.555838  6579 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.556609  6505 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.557391  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.558035  6307 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:29.558321  6574 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:29.558468  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:29.559202  6457 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.559554  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:29.561884  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.562666  6459 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:29.562137  6499 ts_tablet_manager.cc:1434] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.004s
I20260709 12:24:29.564692  6499 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:29.564169  6579 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.576596  6579 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.577618  6501 tablet_bootstrap.cc:654] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.577196  6579 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.579321  6457 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:29.599160  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:29.600389  6505 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:29.600912  6505 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:29.601704  6505 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.605190  6500 tablet_bootstrap.cc:654] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.604558  6582 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:29.629287  6499 tablet_bootstrap.cc:654] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:29.630005  6530 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.630048  6574 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.630795  6530 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.633013  6530 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.636114  6382 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.630842  6574 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.634990  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.637096  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.670228  6205 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:29.648172  6578 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:24:29.671573  6582 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.674885  6310 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:29.682773  6574 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.677889  6582 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.688606  6582 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.696714  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.698122  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.709077  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.710049  6459 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
W20260709 12:24:29.691882  6424 leader_election.cc:343] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:29.712472  6274 leader_election.cc:343] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.705166  6582 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.747103  6579 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.738185  6574 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.750447  6574 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.754105  6574 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.756918  6501 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:29.757457  6501 ts_tablet_manager.cc:1403] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.217s	user 0.036s	sys 0.039s
I20260709 12:24:29.733089  6582 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:29.733469  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53158us
I20260709 12:24:29.766047  6580 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.755156  6579 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.766532  6580 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.756376  6578 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.768246  6505 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.780107  6307 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:29.783583  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:29.794857  6500 tablet_bootstrap.cc:492] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:29.781186  6578 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.795440  6500 ts_tablet_manager.cc:1403] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.258s	user 0.014s	sys 0.042s
I20260709 12:24:29.795539  6501 raft_consensus.cc:359] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.795575  6205 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:29.797665  6501 raft_consensus.cc:385] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.791006  6579 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.799939  6579 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.800510  6582 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.799261  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:29.798113  6501 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.803926  6580 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.793156  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:24:29.807055  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 88004us
I20260709 12:24:29.808825  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.809489  6384 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:29.810539  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.811094  6578 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.811617  6578 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.810946  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.811956  6578 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.812302  6458 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:29.812587  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.813460  6456 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:29.812302  6578 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.784377  6505 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.817130  6501 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.797307  6459 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.815382  6579 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.822985  6348 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.816681  6310 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.823901  6505 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.826138  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.827310  6501 ts_tablet_manager.cc:1434] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.069s	user 0.006s	sys 0.003s
I20260709 12:24:29.830667  6501 tablet_bootstrap.cc:492] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:29.801476  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:24:29.834270  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 147817us
I20260709 12:24:29.835179  6530 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.835698  6530 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.836172  6530 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.830881  6579 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.836447  6630 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.879979  6579 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.883261  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.890192  6456 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:29.890352  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.891364  6307 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:29.893332  6459 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:29.883296  6635 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.876085  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:29.897220  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60252us
I20260709 12:24:29.897634  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.899777  6633 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:29.900384  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.900991  6307 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:29.899624  6582 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:29.901823  6633 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:29.902166  6582 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.902633  6582 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:29.886514  6576 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:24:29.902954  6633 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.888149  6579 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.906729  6504 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:29.907640  6499 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:29.910086  6499 ts_tablet_manager.cc:1403] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.346s	user 0.015s	sys 0.008s
I20260709 12:24:29.881681  6630 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.899315  6635 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.920686  6635 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.898232  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.922960  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:29.923794  6635 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:29.924124  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.925362  6459 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:29.924158  6635 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.926890  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.926911  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.926966  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.930538  6310 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:29.926234  6530 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.931978  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.932925  6499 raft_consensus.cc:359] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.933626  6499 raft_consensus.cc:385] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.933383  6580 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.933766  6505 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:29.934149  6580 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.927745  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:29.934892  6580 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.934245  6505 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.935722  6385 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:29.936055  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:29.933279  6530 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:29.933995  6499 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.935731  6505 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.942718  6459 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:29.945376  6500 raft_consensus.cc:359] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.945884  6500 raft_consensus.cc:385] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:29.946113  6500 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:29.929459  6635 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:29.950903  6499 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.948347  6500 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.918589  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:29.959445  6500 ts_tablet_manager.cc:1434] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.162s	user 0.008s	sys 0.000s
I20260709 12:24:29.960165  6500 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:29.949647  6351 leader_election.cc:343] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.964985  6499 ts_tablet_manager.cc:1434] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.055s	user 0.003s	sys 0.003s
I20260709 12:24:29.965711  6499 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
W20260709 12:24:29.961951  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51550.4us
I20260709 12:24:29.972903  6630 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.974525  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:29.978645  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:29.978735  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:29.977209  6505 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.966706  6578 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.979660  6310 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
W20260709 12:24:29.974685  6424 leader_election.cc:343] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:29.981366  6578 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.985834  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:29.986454  6310 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:29.987044  6505 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:29.984838  6635 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.976107  6582 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:29.981978  6638 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:29.995345  6638 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.997061  6638 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:29.978958  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:30.001174  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:29.999697  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.005098  6309 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:30.001828  6582 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.006466  6382 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:24:30.029117  6501 tablet_bootstrap.cc:654] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:30.018323  6274 leader_election.cc:343] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.021447  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:24:30.020512  6424 leader_election.cc:343] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.023316  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.022734  6348 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:30.036783  6456 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.037856  6633 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:30.033054  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.038359  6633 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.038703  6459 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.038571  6425 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.035720  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.039070  6633 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.038008  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.037235  6635 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.040366  6504 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:30.045506  6504 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.045943  6504 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.048877  6500 tablet_bootstrap.cc:654] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.039284  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.024228  6580 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.052731  6272 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.053478  6580 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.051692  6579 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.055955  6641 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:30.056571  6641 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.056967  6641 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.056923  6638 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.062717  6310 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:30.064682  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.065272  6381 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:30.066172  6458 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:30.064019  6582 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:24:30.068266  6504 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.069269  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.071545  6576 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.072029  6576 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:24:30.072291  6274 leader_election.cc:343] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.072371  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.073057  6457 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.072883  6576 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.079871  6504 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.078209  6638 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.087505  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:30.087514  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:30.089037  6307 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:30.089318  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:30.092320  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:24:30.093639  6274 leader_election.cc:343] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.093664  6641 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.098388  6309 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:24:30.096601  6274 leader_election.cc:343] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.104557  6579 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.114297  6641 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.114256  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.115098  6454 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.115763  6505 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:30.106607  6456 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:30.117530  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.119699  6458 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:30.107081  6381 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:30.131012  6500 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.131516  6500 ts_tablet_manager.cc:1403] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.171s	user 0.015s	sys 0.018s
I20260709 12:24:30.134477  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.124611  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.135887  6499 tablet_bootstrap.cc:654] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.136816  6579 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:24:30.137403  6351 leader_election.cc:343] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.140372  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:30.141081  6309 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.136541  6459 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:30.142184  6504 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.143352  6530 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:30.143713  6530 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:30.144527  6530 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.145153  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.145429  6530 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:30.132855  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.150256  6528 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.151026  6528 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:30.151708  6633 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.157603  6382 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:30.151979  6528 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.159406  6457 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:30.154826  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.160097  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.160365  6505 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.153801  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.160825  6505 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:30.160691  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.161513  6582 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:30.161646  6580 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.162061  6582 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:30.162437  6580 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:30.161787  6505 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.163185  6582 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.163458  6580 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.166320  6206 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.173280  6633 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:30.173906  6424 leader_election.cc:343] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.175239  6500 raft_consensus.cc:359] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.176285  6500 raft_consensus.cc:385] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.156927  6454 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:30.176654  6500 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.183571  6500 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:30.188665  6274 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:30.191972  6348 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:30.193061  6579 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.175278  6424 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.193572  6579 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:30.194373  6579 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.194844  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.196156  6638 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.196826  6574 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:30.170771  6309 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:30.202286  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.203013  6205 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:30.214681  6500 ts_tablet_manager.cc:1434] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.001s
I20260709 12:24:30.215910  6500 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.228421  6501 tablet_bootstrap.cc:492] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:30.229123  6501 ts_tablet_manager.cc:1403] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.399s	user 0.010s	sys 0.014s
W20260709 12:24:30.220745  6274 leader_election.cc:343] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.235977  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:30.236621  6309 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.229733  6454 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.242944  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.279104  6500 tablet_bootstrap.cc:654] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.268522  6505 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.283902  6309 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:30.278865  6576 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.289870  6501 raft_consensus.cc:359] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.290330  6501 raft_consensus.cc:385] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.290544  6501 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.293138  6501 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.295497  6501 ts_tablet_manager.cc:1434] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.002s
W20260709 12:24:30.281190  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61557us
I20260709 12:24:30.296479  6501 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
W20260709 12:24:30.297413  6274 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:30.297744  6454 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:30.298877  6348 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:30.299957  6580 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.300060  6205 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.301090  6580 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:30.301949  6580 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.301677  6205 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.303500  6205 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 2, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.307525  6576 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.309748  6576 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.310356  6206 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.311751  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.312153  6206 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:30.313012  6424 leader_election.cc:343] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.317718  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.324397  6499 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:30.325057  6499 ts_tablet_manager.cc:1403] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.359s	user 0.022s	sys 0.017s
W20260709 12:24:30.335790  6425 leader_election.cc:343] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.336275  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.337980  6576 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.339604  6499 raft_consensus.cc:359] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.340369  6499 raft_consensus.cc:385] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.340629  6499 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.341324  6499 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.343751  6499 ts_tablet_manager.cc:1434] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:24:30.344676  6499 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:30.351743  6499 tablet_bootstrap.cc:654] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.356698  6500 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.357191  6500 ts_tablet_manager.cc:1403] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.142s	user 0.025s	sys 0.033s
I20260709 12:24:30.358111  6501 tablet_bootstrap.cc:654] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.359954  6500 raft_consensus.cc:359] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.360648  6500 raft_consensus.cc:385] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.361014  6500 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.361776  6500 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.363672  6500 ts_tablet_manager.cc:1434] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:24:30.364580  6500 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.370613  6500 tablet_bootstrap.cc:654] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.371225  6580 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:30.386572  6206 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.391314  6501 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:30.391862  6501 ts_tablet_manager.cc:1403] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.096s	user 0.009s	sys 0.004s
I20260709 12:24:30.397774  6501 raft_consensus.cc:359] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.398501  6501 raft_consensus.cc:385] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.398780  6501 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.399420  6501 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.402814  6501 ts_tablet_manager.cc:1434] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:24:30.403890  6501 tablet_bootstrap.cc:492] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:30.405556  6499 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:30.406107  6499 ts_tablet_manager.cc:1403] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.007s
I20260709 12:24:30.406272  6635 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:30.408733  6499 raft_consensus.cc:359] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.409351  6499 raft_consensus.cc:385] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.409626  6499 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.410305  6499 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.412673  6499 ts_tablet_manager.cc:1434] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:30.413693  6499 tablet_bootstrap.cc:492] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:30.414649  6500 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.415158  6500 ts_tablet_manager.cc:1403] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.051s	user 0.029s	sys 0.015s
I20260709 12:24:30.415560  6501 tablet_bootstrap.cc:654] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.420318  6500 raft_consensus.cc:359] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.421053  6500 raft_consensus.cc:385] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.421356  6500 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.422765  6499 tablet_bootstrap.cc:654] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.422704  6500 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.425457  6500 ts_tablet_manager.cc:1434] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:30.426421  6500 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.432965  6500 tablet_bootstrap.cc:654] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.443446  6499 tablet_bootstrap.cc:492] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:30.443977  6499 ts_tablet_manager.cc:1403] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.008s
I20260709 12:24:30.447048  6499 raft_consensus.cc:359] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.447427  6576 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.447742  6499 raft_consensus.cc:385] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.448184  6499 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.448004  6576 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.449141  6499 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.450618  6576 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.450879  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.451648  6310 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:30.452545  6641 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.453431  6499 ts_tablet_manager.cc:1434] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 12:24:30.453210  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:30.453217  6641 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.454741  6576 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:30.455255  6576 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.455705  6576 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.457343  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.457496  6641 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.459053  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:30.460041  6457 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:30.462723  6272 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.464347  6641 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.466841  6524 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.467537  6524 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.467900  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.470398  6524 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:30.473526  6424 leader_election.cc:343] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.475345  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.477059  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:30.478626  6641 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.482650  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.483566  6524 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.484599  6499 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:30.484653  6501 tablet_bootstrap.cc:492] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:30.485357  6501 ts_tablet_manager.cc:1403] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.082s	user 0.043s	sys 0.014s
I20260709 12:24:30.486126  6500 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.486652  6500 ts_tablet_manager.cc:1403] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.060s	user 0.022s	sys 0.027s
I20260709 12:24:30.487759  6641 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.471835  6576 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.488267  6641 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.490370  6501 raft_consensus.cc:359] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.491200  6501 raft_consensus.cc:385] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.491539  6501 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.492089  6500 raft_consensus.cc:359] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.492424  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.492817  6500 raft_consensus.cc:385] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.493098  6500 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.493196  6501 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.494007  6500 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.495390  6499 tablet_bootstrap.cc:654] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.496178  6501 ts_tablet_manager.cc:1434] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:24:30.496320  6500 ts_tablet_manager.cc:1434] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:30.497262  6500 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.497394  6501 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:30.497789  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:24:30.493870  6274 leader_election.cc:343] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.498397  6310 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.502919  6500 tablet_bootstrap.cc:654] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.506013  6310 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:30.508373  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:30.509673  6424 leader_election.cc:343] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.510444  6576 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.510730  6501 tablet_bootstrap.cc:654] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.511785  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.513626  6576 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.514256  6576 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:30.515352  6576 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.539672  6576 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.540359  6576 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.542553  6574 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.543252  6574 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.545014  6576 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.547962  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.549002  6499 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:30.549687  6499 ts_tablet_manager.cc:1403] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.065s	user 0.010s	sys 0.003s
I20260709 12:24:30.549557  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.551729  6574 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.552759  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:30.553305  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.553583  6457 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
W20260709 12:24:30.555383  6424 leader_election.cc:343] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.556447  6272 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:24:30.557400  6425 leader_election.cc:343] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:30.558105  6425 leader_election.cc:343] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.558533  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.559633  6582 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:30.560019  6582 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.560403  6582 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.563138  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.563794  6504 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.563875  6576 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:30.564481  6424 leader_election.cc:343] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.564949  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.566569  6504 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.540689  6206 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:30.558667  6499 raft_consensus.cc:359] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.570210  6499 raft_consensus.cc:385] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.570641  6499 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.571549  6499 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.580469  6499 ts_tablet_manager.cc:1434] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.002s
I20260709 12:24:30.581772  6499 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:30.584678  6641 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.589175  6582 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.593397  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.594785  6582 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.595822  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:24:30.596387  6274 leader_election.cc:343] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.596590  6457 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.599247  6500 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.599604  6501 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:30.599782  6500 ts_tablet_manager.cc:1403] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.103s	user 0.021s	sys 0.066s
I20260709 12:24:30.600255  6501 ts_tablet_manager.cc:1403] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.103s	user 0.036s	sys 0.046s
I20260709 12:24:30.603363  6500 raft_consensus.cc:359] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.604031  6500 raft_consensus.cc:385] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.603624  6501 raft_consensus.cc:359] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.604255  6500 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.604593  6501 raft_consensus.cc:385] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.604970  6501 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.605063  6500 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.605823  6501 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.607074  6500 ts_tablet_manager.cc:1434] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:30.608080  6500 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.608280  6384 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.613720  6500 tablet_bootstrap.cc:654] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.614068  6457 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:30.614825  6501 ts_tablet_manager.cc:1434] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.003s
I20260709 12:24:30.615666  6272 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:30.615936  6501 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:30.616876  6582 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.617404  6582 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:30.618244  6582 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.624708  6500 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.625193  6500 ts_tablet_manager.cc:1403] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.017s	user 0.006s	sys 0.008s
I20260709 12:24:30.625300  6381 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:30.627099  6641 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.627887  6500 raft_consensus.cc:359] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.628275  6576 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.628569  6500 raft_consensus.cc:385] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.628829  6500 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.629664  6500 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.631090  6500 ts_tablet_manager.cc:1434] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:24:30.638072  6635 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.642630  6500 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.642959  6457 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:30.644658  6505 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.653012  6501 tablet_bootstrap.cc:654] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.655975  6500 tablet_bootstrap.cc:654] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:30.665108  6274 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:30.671149  6635 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.672423  6205 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.677992  6499 tablet_bootstrap.cc:654] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.678236  6635 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.686983  6382 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.688203  6635 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.707195  6500 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.700188  6635 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.707684  6500 ts_tablet_manager.cc:1403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.065s	user 0.011s	sys 0.005s
I20260709 12:24:30.709630  6635 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.711153  6500 raft_consensus.cc:359] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.711905  6500 raft_consensus.cc:385] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.712178  6500 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.712437  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:30.712827  6500 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.713840  6641 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.714949  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.715241  6500 ts_tablet_manager.cc:1434] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:24:30.716308  6500 tablet_bootstrap.cc:492] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.718778  6505 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.723024  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.724375  6580 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.724713  6633 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:30.730397  6384 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.741109  6633 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.742501  6501 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:30.743160  6501 ts_tablet_manager.cc:1403] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.127s	user 0.014s	sys 0.014s
I20260709 12:24:30.755081  6500 tablet_bootstrap.cc:654] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.760886  6582 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.750643  6524 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.762535  6524 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.764974  6635 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:30.770080  6382 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.771363  6524 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.775173  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:30.783049  6499 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:30.784648  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:30.785109  6499 ts_tablet_manager.cc:1403] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.204s	user 0.012s	sys 0.013s
I20260709 12:24:30.795387  6351 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:30.798388  6580 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:30.814765  6640 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.814522  6501 raft_consensus.cc:359] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.815212  6640 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.817269  6640 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.818841  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.821507  6501 raft_consensus.cc:385] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.821841  6501 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.822631  6501 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.824666  6501 ts_tablet_manager.cc:1434] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.000s
I20260709 12:24:30.824546  6385 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:30.825578  6501 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:30.818763  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:30.826949  6274 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:30.827370  6456 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:30.828008  6640 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:30.831231  6640 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.832674  6640 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:30.839553  6679 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:24:30.845053  6640 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.860416  6499 raft_consensus.cc:359] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.861243  6499 raft_consensus.cc:385] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.861541  6499 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.862021  6499 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.863130  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:30.863792  6381 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.864537  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:30.865067  6454 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:30.866889  6499 ts_tablet_manager.cc:1434] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.000s
I20260709 12:24:30.867996  6499 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:30.873435  6454 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:30.875547  6501 tablet_bootstrap.cc:654] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.878470  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:30.879448  6582 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:30.879963  6582 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:30.880867  6582 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.883793  6500 tablet_bootstrap.cc:492] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.884330  6500 ts_tablet_manager.cc:1403] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.168s	user 0.016s	sys 0.003s
I20260709 12:24:30.869884  6640 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:30.888388  6500 raft_consensus.cc:359] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.890897  6500 raft_consensus.cc:385] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.891222  6500 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.891834  6500 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.902793  6500 ts_tablet_manager.cc:1434] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:24:30.903926  6500 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.925992  6381 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:30.928373  6499 tablet_bootstrap.cc:654] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.936650  6205 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:30.937309  6500 tablet_bootstrap.cc:654] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.947227  6504 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.947842  6504 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.951932  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:30.953363  6424 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:30.956434  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:30.958140  6424 leader_election.cc:343] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:30.959142  6425 leader_election.cc:343] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.964141  6500 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:30.964389  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.959856  6504 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.965674  6576 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.964615  6500 ts_tablet_manager.cc:1403] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.061s	user 0.003s	sys 0.014s
I20260709 12:24:30.967609  6499 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:30.968079  6499 ts_tablet_manager.cc:1403] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.011s
I20260709 12:24:30.969584  6500 raft_consensus.cc:359] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.970178  6500 raft_consensus.cc:385] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.970422  6500 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.970606  6499 raft_consensus.cc:359] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.971372  6499 raft_consensus.cc:385] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.971144  6500 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.971666  6499 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:30.971902  6504 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.972491  6504 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.972543  6499 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.974785  6499 ts_tablet_manager.cc:1434] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:24:30.974819  6504 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:30.973495  6500 ts_tablet_manager.cc:1434] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:24:30.975988  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:30.976429  6500 tablet_bootstrap.cc:492] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:30.976653  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:30.977787  6424 leader_election.cc:343] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.979264  6499 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
W20260709 12:24:30.979255  6425 leader_election.cc:343] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:30.983619  6501 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:30.984099  6501 ts_tablet_manager.cc:1403] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.159s	user 0.008s	sys 0.008s
I20260709 12:24:30.986847  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:30.988410  6501 raft_consensus.cc:359] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:30.988914  6500 tablet_bootstrap.cc:654] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:30.989231  6501 raft_consensus.cc:385] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:30.989356  6504 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:30.993733  6499 tablet_bootstrap.cc:654] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.007623  6500 tablet_bootstrap.cc:492] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.008186  6500 ts_tablet_manager.cc:1403] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.032s	user 0.009s	sys 0.009s
I20260709 12:24:31.011847  6504 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:31.013638  6500 raft_consensus.cc:359] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.014305  6500 raft_consensus.cc:385] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.014664  6500 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.015336  6500 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:31.014302  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:31.017656  6499 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.018143  6499 ts_tablet_manager.cc:1403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.021s
I20260709 12:24:31.018699  6500 ts_tablet_manager.cc:1434] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:31.019985  6500 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.020771  6499 raft_consensus.cc:359] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.021462  6499 raft_consensus.cc:385] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.021739  6499 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.022423  6499 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.024082  6582 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.989605  6501 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.024946  6582 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.026021  6501 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.026980  6499 ts_tablet_manager.cc:1434] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:24:31.028024  6499 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:31.027973  6582 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.028344  6501 ts_tablet_manager.cc:1434] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:24:31.029425  6501 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.029666  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.030287  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:31.033169  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.032760  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.037528  6582 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.037935  6582 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.038241  6582 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:31.038552  6274 leader_election.cc:343] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.042623  6500 tablet_bootstrap.cc:654] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.044402  6499 tablet_bootstrap.cc:654] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.044685  6582 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.052629  6574 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.053138  6574 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.053933  6501 tablet_bootstrap.cc:654] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.055368  6574 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.057407  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.058130  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.058308  6582 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:31.058773  6424 leader_election.cc:343] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.061118  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:31.062418  6274 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:31.064788  6274 leader_election.cc:343] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.062618  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.067353  6457 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:31.063287  6425 leader_election.cc:343] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.071460  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.073364  6574 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.075961  6582 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.078002  6640 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:31.078853  6381 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.076467  6582 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.099689  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.101610  6500 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.102313  6500 ts_tablet_manager.cc:1403] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.001s
W20260709 12:24:31.104516  6274 leader_election.cc:343] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.105474  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.106396  6454 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:31.097687  6582 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.108211  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.109310  6641 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.109818  6641 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.110247  6641 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.118288  6574 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.119036  6574 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.122620  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.123099  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:31.124349  6424 leader_election.cc:343] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.125130  6574 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:31.126315  6425 leader_election.cc:343] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.127030  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.128546  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:31.140308  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.142127  6582 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.142839  6582 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:31.144200  6582 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.145426  6504 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.152674  6501 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.153151  6501 ts_tablet_manager.cc:1403] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.124s	user 0.035s	sys 0.058s
I20260709 12:24:31.154138  6641 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.158768  6501 raft_consensus.cc:359] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.159755  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:31.155922  6500 raft_consensus.cc:359] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.159463  6501 raft_consensus.cc:385] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.160542  6500 raft_consensus.cc:385] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.161144  6501 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.158768  6499 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.161435  6500 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.164103  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.164678  6457 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.161883  6499 ts_tablet_manager.cc:1403] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.134s	user 0.038s	sys 0.049s
I20260709 12:24:31.161897  6501 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.166841  6574 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.167354  6382 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.167402  6574 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.168172  6499 raft_consensus.cc:359] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.168813  6499 raft_consensus.cc:385] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.169086  6499 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.169749  6499 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.171613  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:31.173322  6424 leader_election.cc:343] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.173944  6574 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.162098  6641 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.180266  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:31.182294  6425 leader_election.cc:343] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.182775  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.163882  6500 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.179879  6382 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:31.226444  6500 ts_tablet_manager.cc:1434] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.124s	user 0.026s	sys 0.021s
I20260709 12:24:31.228039  6500 tablet_bootstrap.cc:492] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.185871  6574 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.185599  6499 ts_tablet_manager.cc:1434] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:24:31.237119  6500 tablet_bootstrap.cc:654] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.240963  6501 ts_tablet_manager.cc:1434] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.088s	user 0.012s	sys 0.000s
I20260709 12:24:31.241837  6501 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.248418  6501 tablet_bootstrap.cc:654] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.252954  6499 tablet_bootstrap.cc:492] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:31.178241  6457 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:31.258921  6499 tablet_bootstrap.cc:654] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.259819  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.267170  6582 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.267864  6582 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:31.269006  6582 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.272478  6501 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.273048  6501 ts_tablet_manager.cc:1403] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.000s
I20260709 12:24:31.275992  6501 raft_consensus.cc:359] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.276872  6501 raft_consensus.cc:385] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.277197  6501 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.277980  6501 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.281944  6501 ts_tablet_manager.cc:1434] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:24:31.281553  6203 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.283157  6501 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.281615  6500 tablet_bootstrap.cc:492] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.285318  6500 ts_tablet_manager.cc:1403] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.029s
I20260709 12:24:31.288146  6500 raft_consensus.cc:359] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.290174  6501 tablet_bootstrap.cc:654] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.290374  6499 tablet_bootstrap.cc:492] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.288805  6500 raft_consensus.cc:385] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.291944  6500 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.290886  6499 ts_tablet_manager.cc:1403] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.007s
I20260709 12:24:31.294788  6499 raft_consensus.cc:359] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.295465  6499 raft_consensus.cc:385] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.295766  6499 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.296332  6499 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.298317  6499 ts_tablet_manager.cc:1434] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.000s
I20260709 12:24:31.299144  6499 tablet_bootstrap.cc:492] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:31.302769  6500 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.305259  6500 ts_tablet_manager.cc:1434] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:24:31.306133  6500 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.307523  6501 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.307731  6641 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:31.308101  6501 ts_tablet_manager.cc:1403] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 12:24:31.310937  6501 raft_consensus.cc:359] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.311693  6500 tablet_bootstrap.cc:654] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.311707  6501 raft_consensus.cc:385] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.314559  6501 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.315254  6501 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.317409  6501 ts_tablet_manager.cc:1434] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.000s
I20260709 12:24:31.318207  6501 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.321316  6499 tablet_bootstrap.cc:654] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.323611  6501 tablet_bootstrap.cc:654] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.323496  6500 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.324960  6500 ts_tablet_manager.cc:1403] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 12:24:31.327800  6500 raft_consensus.cc:359] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.328604  6500 raft_consensus.cc:385] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.328953  6500 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.329838  6501 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.329644  6500 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.330264  6501 ts_tablet_manager.cc:1403] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:24:31.331983  6500 ts_tablet_manager.cc:1434] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:24:31.332922  6501 raft_consensus.cc:359] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.333312  6500 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.333642  6501 raft_consensus.cc:385] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.334122  6501 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.334970  6501 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.336302  6582 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.336771  6582 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.337332  6501 ts_tablet_manager.cc:1434] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.001s
I20260709 12:24:31.337173  6641 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.337857  6641 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.339238  6582 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.339811  6641 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.340902  6501 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.340970  6500 tablet_bootstrap.cc:654] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.345361  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.345997  6457 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:31.347033  6501 tablet_bootstrap.cc:654] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.346871  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.347481  6454 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:31.358345  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.358702  6501 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.359131  6641 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.359292  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.359500  6641 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.359860  6641 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.361502  6582 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.361933  6582 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.362358  6582 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.365248  6501 ts_tablet_manager.cc:1403] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.004s
I20260709 12:24:31.368160  6641 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.371925  6501 raft_consensus.cc:359] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.372571  6501 raft_consensus.cc:385] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.372841  6501 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.373449  6501 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.375440  6501 ts_tablet_manager.cc:1434] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:24:31.376323  6501 tablet_bootstrap.cc:492] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.377213  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.377660  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:31.381851  6274 leader_election.cc:343] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.382443  6641 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.384028  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.384652  6454 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.386986  6501 tablet_bootstrap.cc:654] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.392381  6454 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
W20260709 12:24:31.393128  6274 leader_election.cc:343] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.395385  6499 tablet_bootstrap.cc:492] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.395828  6499 ts_tablet_manager.cc:1403] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.097s	user 0.003s	sys 0.007s
I20260709 12:24:31.396065  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:31.397490  6274 leader_election.cc:343] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.398751  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.399183  6582 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.405284  6641 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.405769  6641 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:31.405740  6582 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.406643  6641 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.398344  6499 raft_consensus.cc:359] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.407675  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.410661  6578 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:31.406857  6205 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.411217  6499 raft_consensus.cc:385] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.418993  6499 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.419692  6499 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.421115  6499 ts_tablet_manager.cc:1434] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.001s
I20260709 12:24:31.411414  6457 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.411067  6501 tablet_bootstrap.cc:492] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.424074  6501 ts_tablet_manager.cc:1403] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.002s
I20260709 12:24:31.414093  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:31.428831  6499 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
W20260709 12:24:31.431558  6274 leader_election.cc:343] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.441272  6457 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:31.444677  6381 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.444502  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.447520  6500 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.447968  6578 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.448074  6500 ts_tablet_manager.cc:1403] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.115s	user 0.025s	sys 0.047s
I20260709 12:24:31.448768  6578 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:31.456106  6500 raft_consensus.cc:359] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.456951  6500 raft_consensus.cc:385] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.457235  6500 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.449654  6578 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.460656  6205 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.457883  6500 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.466210  6500 ts_tablet_manager.cc:1434] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:24:31.467450  6500 tablet_bootstrap.cc:492] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.472697  6501 raft_consensus.cc:359] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.473569  6501 raft_consensus.cc:385] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.473919  6501 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.474745  6501 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.477135  6501 ts_tablet_manager.cc:1434] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.000s
I20260709 12:24:31.478130  6501 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.481292  6499 tablet_bootstrap.cc:654] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.485674  6501 tablet_bootstrap.cc:654] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.486050  6500 tablet_bootstrap.cc:654] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.495779  6500 tablet_bootstrap.cc:492] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.496393  6500 ts_tablet_manager.cc:1403] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 12:24:31.498437  6500 raft_consensus.cc:359] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.499157  6500 raft_consensus.cc:385] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.499420  6500 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.500069  6633 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.500087  6500 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.500953  6633 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.502674  6500 ts_tablet_manager.cc:1434] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:24:31.504032  6500 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.508219  6630 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.508353  6574 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.508561  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.508764  6630 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.508927  6574 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.511137  6630 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.511286  6574 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.512283  6578 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:31.512821  6578 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.513971  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.514531  6500 tablet_bootstrap.cc:654] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.515844  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:31.517277  6351 leader_election.cc:343] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.518402  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:31.519127  6425 leader_election.cc:343] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.519160  6381 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:31.519732  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.527078  6500 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.527637  6500 ts_tablet_manager.cc:1403] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.012s
I20260709 12:24:31.528473  6499 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.529007  6499 ts_tablet_manager.cc:1403] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.100s	user 0.011s	sys 0.017s
W20260709 12:24:31.529376  6424 leader_election.cc:343] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.529891  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.530601  6500 raft_consensus.cc:359] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.531328  6500 raft_consensus.cc:385] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.531678  6500 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.531831  6504 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.532449  6500 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.533046  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.532793  6499 raft_consensus.cc:359] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.534214  6500 ts_tablet_manager.cc:1434] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:31.534713  6499 raft_consensus.cc:385] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.535313  6499 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.535642  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:31.536705  6630 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.537987  6578 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.538967  6633 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.539037  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.539690  6457 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:31.540270  6500 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.540819  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.540836  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.542332  6457 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:31.536554  6499 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.544480  6641 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:31.544862  6641 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:31.545208  6641 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:31.547868  6499 ts_tablet_manager.cc:1434] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:24:31.549381  6499 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:31.550771  6500 tablet_bootstrap.cc:654] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.554704  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.556330  6633 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.556715  6633 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.557060  6633 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.563184  6500 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.563854  6500 ts_tablet_manager.cc:1403] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.012s
I20260709 12:24:31.566126  6582 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:31.566469  6499 tablet_bootstrap.cc:654] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.574851  6574 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.575459  6574 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.575806  6630 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.575928  6500 raft_consensus.cc:359] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.576282  6630 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.576907  6500 raft_consensus.cc:385] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.577294  6500 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.578256  6630 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.578171  6500 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.580451  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.580835  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.581436  6457 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:31.581687  6500 ts_tablet_manager.cc:1434] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:24:31.582886  6500 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:31.584173  6351 leader_election.cc:343] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.553666  6641 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.590265  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:31.591136  6381 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:31.591409  6500 tablet_bootstrap.cc:654] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.591779  6205 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.596719  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.585760  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.599272  6630 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.599685  6630 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.600064  6630 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.601564  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.603068  6576 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.603595  6576 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.605870  6576 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.607889  6630 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.608551  6499 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.609277  6499 ts_tablet_manager.cc:1403] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.009s
I20260709 12:24:31.610034  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:31.611021  6425 leader_election.cc:343] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.613008  6582 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.613507  6582 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.613286  6499 raft_consensus.cc:359] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.614643  6499 raft_consensus.cc:385] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.615095  6499 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.616223  6582 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.618161  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.619040  6455 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:31.621347  6641 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.623443  6574 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.624070  6633 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.624596  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.625638  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:31.625816  6505 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.626819  6505 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.627230  6505 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.616139  6499 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.632191  6499 ts_tablet_manager.cc:1434] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:24:31.633894  6499 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:31.634859  6505 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.626282  6633 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.610200  6630 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.641323  6505 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.637274  6305 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:31.653439  6501 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.653919  6501 ts_tablet_manager.cc:1403] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.176s	user 0.024s	sys 0.038s
I20260709 12:24:31.654778  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.655817  6455 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.649636  6457 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:31.660953  6500 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.661437  6500 ts_tablet_manager.cc:1403] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.079s	user 0.022s	sys 0.008s
I20260709 12:24:31.663233  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:31.665071  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.665280  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.669333  6501 raft_consensus.cc:359] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.666268  6576 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:31.670089  6501 raft_consensus.cc:385] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.670177  6576 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.670393  6501 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.670583  6576 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.658371  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.671237  6454 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.671094  6501 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.672888  6455 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:31.658463  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.675107  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:31.675755  6456 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.675983  6500 raft_consensus.cc:359] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.676699  6500 raft_consensus.cc:385] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.678367  6576 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.679997  6501 ts_tablet_manager.cc:1434] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:24:31.676966  6500 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.681927  6505 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.682147  6501 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.682771  6505 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.686790  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:31.687420  6305 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.690637  6456 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:31.681712  6500 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.693279  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:31.694262  6351 leader_election.cc:343] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:31.695653  6274 leader_election.cc:343] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.696631  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.700773  6641 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:31.701231  6641 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:31.702062  6641 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.703886  6576 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.698408  6454 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:31.700196  6499 tablet_bootstrap.cc:654] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:31.707245  6274 leader_election.cc:343] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:31.711541  6424 leader_election.cc:343] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.708312  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:31.695665  6505 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.714958  6500 ts_tablet_manager.cc:1434] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.001s
I20260709 12:24:31.715978  6500 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:31.716528  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.718653  6505 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.719367  6505 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:31.720366  6505 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:31.728626  6424 leader_election.cc:343] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.729049  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:31.729151  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.729802  6310 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:31.731233  6305 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
W20260709 12:24:31.731384  6274 consensus_peers.cc:597] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:31.732105  6274 leader_election.cc:343] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.734658  6574 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.736084  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.734289  6582 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:31.737682  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.737949  6633 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.738469  6633 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:31.739058  6630 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.739357  6633 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.743635  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.745206  6501 tablet_bootstrap.cc:654] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.745757  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:31.745884  6272 leader_election.cc:343] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.748755  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:31.742403  6310 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:31.753309  6630 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:31.753952  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:31.755133  6576 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:31.751479  6578 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.759809  6576 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:31.773298  6500 tablet_bootstrap.cc:654] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:31.775977  6205 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:31.775206  6576 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.755610  6630 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:31.794912  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 59291.7us
W20260709 12:24:31.797158  6424 leader_election.cc:343] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.797395  6578 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:31.803563  6504 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.804158  6504 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.807621  6504 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:31.879308  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82195.3us
I20260709 12:24:31.795440  6730 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.888819  6574 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.889391  6574 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.891746  6574 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.897109  6574 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.897528  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.897624  6574 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:31.899147  6424 leader_election.cc:343] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.899768  6574 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.900190  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.885993  6730 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.904109  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.904814  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.906436  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.904799  6206 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:31.907588  6501 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:31.908282  6501 ts_tablet_manager.cc:1403] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.226s	user 0.016s	sys 0.007s
I20260709 12:24:31.910604  6499 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:31.911037  6499 ts_tablet_manager.cc:1403] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.277s	user 0.017s	sys 0.010s
I20260709 12:24:31.922044  6501 raft_consensus.cc:359] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:31.913599  6424 leader_election.cc:343] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.922605  6501 raft_consensus.cc:385] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.915486  6633 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.923092  6501 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.925985  6501 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.924376  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:31.923409  6574 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.921572  6730 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:31.929144  6425 leader_election.cc:343] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.944813  6501 ts_tablet_manager.cc:1434] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.036s	user 0.016s	sys 0.002s
I20260709 12:24:31.946192  6501 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:31.950452  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.951324  6457 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:31.940886  6574 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.954766  6574 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:31.928915  6633 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:31.938467  6424 leader_election.cc:343] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.957948  6500 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:31.958788  6500 ts_tablet_manager.cc:1403] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.243s	user 0.029s	sys 0.004s
I20260709 12:24:31.958722  6499 raft_consensus.cc:359] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.962672  6499 raft_consensus.cc:385] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:31.963152  6499 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:31.964993  6633 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:31.948690  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.969133  6576 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:31.969235  6425 leader_election.cc:343] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.969715  6425 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.971328  6576 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:31.976166  6499 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.976233  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:31.977841  6499 ts_tablet_manager.cc:1434] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
W20260709 12:24:31.979516  6425 leader_election.cc:343] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:31.980227  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:31.984445  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:31.985294  6456 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:31.987691  6499 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:31.987581  6206 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:31.989254  6424 leader_election.cc:343] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.006506  6385 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.010986  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.013679  6500 raft_consensus.cc:359] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:31.997622  6729 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.015743  6500 raft_consensus.cc:385] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.016069  6500 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.018116  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.019923  6310 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:32.021536  6740 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.025583  6500 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.028774  6740 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:24:32.033505  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.037793  6580 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.038291  6580 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:32.039557  6425 leader_election.cc:343] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.040033  6425 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:32.042589  6580 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.048089  6500 ts_tablet_manager.cc:1434] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.089s	user 0.004s	sys 0.002s
I20260709 12:24:32.049113  6500 tablet_bootstrap.cc:492] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:32.053416  6501 tablet_bootstrap.cc:654] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.059617  6206 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:32.067618  6206 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:32.057879  6729 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.081082  6380 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.093992  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.095360  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:32.096206  6274 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:32.097206  6580 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.118873  6499 tablet_bootstrap.cc:654] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.126168  6351 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:32.126878  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.127628  6457 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
W20260709 12:24:32.127687  6351 leader_election.cc:343] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.130836  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:32.134256  6640 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.135769  6738 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:32.135908  6580 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.136197  6738 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.136539  6738 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.138865  6630 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.148906  6580 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.150960  6580 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.153254  6633 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.153416  6580 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.208106  6500 tablet_bootstrap.cc:654] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.212548  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.213424  6454 raft_consensus.cc:2410] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260709 12:24:32.214720  6505 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:24:32.219521  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:32.221525  6499 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:32.222289  6499 ts_tablet_manager.cc:1403] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.235s	user 0.005s	sys 0.021s
I20260709 12:24:32.222385  6738 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.225910  6582 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.225817  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.226680  6582 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.227845  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:32.228647  6459 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.269454  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:32.271813  6500 tablet_bootstrap.cc:492] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:32.272053  6501 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:32.272770  6501 ts_tablet_manager.cc:1403] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.327s	user 0.008s	sys 0.028s
I20260709 12:24:32.274297  6500 ts_tablet_manager.cc:1403] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.225s	user 0.014s	sys 0.016s
I20260709 12:24:32.281980  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.278134  6580 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:32.282788  6454 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:32.236565  6582 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.284135  6499 raft_consensus.cc:359] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.284830  6499 raft_consensus.cc:385] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.285122  6499 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.285588  6459 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:32.285815  6499 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.286825  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:32.289343  6580 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:32.289856  6580 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:32.291105  6580 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.240581  6738 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.299432  6501 raft_consensus.cc:359] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.300877  6501 raft_consensus.cc:385] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.301172  6501 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.301239  6499 ts_tablet_manager.cc:1434] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.079s	user 0.000s	sys 0.006s
I20260709 12:24:32.302434  6499 tablet_bootstrap.cc:492] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:32.301887  6501 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:32.311030  6747 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:24:32.316872  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:24:32.333633  6351 leader_election.cc:343] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.336149  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.336907  6380 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:32.340164  6641 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.340735  6641 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.345147  6641 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:32.364553  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90229.3us
I20260709 12:24:32.366472  6380 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:32.367880  6456 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:32.369865  6630 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:24:32.384608  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.385494  6456 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:32.392063  6272 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:32.392235  6501 ts_tablet_manager.cc:1434] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.119s	user 0.003s	sys 0.004s
I20260709 12:24:32.393631  6501 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:32.394412  6641 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:32.394872  6641 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.395195  6641 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:32.396891  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.396885  6500 raft_consensus.cc:359] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.400059  6500 raft_consensus.cc:385] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.400350  6500 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.401091  6500 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.415838  6500 ts_tablet_manager.cc:1434] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.141s	user 0.008s	sys 0.006s
I20260709 12:24:32.416790  6500 tablet_bootstrap.cc:492] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:32.414039  6499 tablet_bootstrap.cc:654] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:32.419540  6274 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:32.423223  6738 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.426048  6641 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.428171  6641 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.430027  6740 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.430675  6501 tablet_bootstrap.cc:654] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.437496  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:32.438136  6383 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:32.438812  6500 tablet_bootstrap.cc:654] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.458428  6499 tablet_bootstrap.cc:492] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:32.459342  6499 ts_tablet_manager.cc:1403] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.157s	user 0.019s	sys 0.043s
W20260709 12:24:32.461341  6424 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:32.462556  6499 raft_consensus.cc:359] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.463387  6499 raft_consensus.cc:385] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.463663  6499 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.464358  6499 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.464856  6500 tablet_bootstrap.cc:492] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:32.466324  6500 ts_tablet_manager.cc:1403] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.050s	user 0.006s	sys 0.007s
I20260709 12:24:32.466764  6499 ts_tablet_manager.cc:1434] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:24:32.467725  6499 tablet_bootstrap.cc:492] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:32.468907  6383 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:32.469071  6500 raft_consensus.cc:359] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.469806  6500 raft_consensus.cc:385] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.470098  6500 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.473249  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:32.475788  6499 tablet_bootstrap.cc:654] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.475666  6500 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.480077  6500 ts_tablet_manager.cc:1434] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:24:32.481099  6500 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:32.480418  6203 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:32.484804  6744 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.485325  6753 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:32.485410  6744 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.485864  6753 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:32.486692  6753 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.489203  6744 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:32.492028  6305 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.497764  6500 tablet_bootstrap.cc:654] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.509037  6499 tablet_bootstrap.cc:492] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:32.509582  6499 ts_tablet_manager.cc:1403] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.009s
W20260709 12:24:32.505070  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 111965us
I20260709 12:24:32.512533  6499 raft_consensus.cc:359] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.513244  6499 raft_consensus.cc:385] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.513540  6499 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.514875  6744 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.515468  6744 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.516371  6640 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.517468  6744 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:32.514320  6499 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.520391  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.520465  6500 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:32.521848  6500 ts_tablet_manager.cc:1403] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.009s
W20260709 12:24:32.522709  6424 leader_election.cc:343] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.521126  6499 ts_tablet_manager.cc:1434] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:24:32.523581  6753 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.524358  6499 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:32.524101  6753 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.527493  6753 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.528857  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.529819  6500 raft_consensus.cc:359] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.530589  6500 raft_consensus.cc:385] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.531006  6500 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.531960  6500 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.533217  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:32.534624  6424 leader_election.cc:343] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.532306  6272 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:32.530174  6274 leader_election.cc:343] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.535425  6500 ts_tablet_manager.cc:1434] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 12:24:32.535528  6640 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.536772  6500 tablet_bootstrap.cc:492] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:32.538141  6499 tablet_bootstrap.cc:654] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.543706  6576 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.545341  6500 tablet_bootstrap.cc:654] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.546787  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:32.547403  6455 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:32.553193  6499 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:32.553908  6499 ts_tablet_manager.cc:1403] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.000s
I20260709 12:24:32.557188  6455 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:32.562463  6499 raft_consensus.cc:359] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.563591  6499 raft_consensus.cc:385] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.563963  6499 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.564853  6499 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.568368  6576 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.569089  6576 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.570123  6500 tablet_bootstrap.cc:492] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:32.570813  6500 ts_tablet_manager.cc:1403] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.034s	user 0.020s	sys 0.010s
I20260709 12:24:32.571887  6576 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:32.572992  6499 ts_tablet_manager.cc:1434] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 12:24:32.573580  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.574081  6499 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:32.574188  6500 raft_consensus.cc:359] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.574903  6500 raft_consensus.cc:385] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:24:32.575126  6424 leader_election.cc:343] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.575165  6500 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.576368  6500 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.581669  6500 ts_tablet_manager.cc:1434] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:24:32.583900  6641 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.584594  6641 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.587379  6641 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.589259  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.590508  6500 tablet_bootstrap.cc:492] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:32.592767  6499 tablet_bootstrap.cc:654] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:32.594801  6274 leader_election.cc:343] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.603704  6500 tablet_bootstrap.cc:654] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.603760  6499 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:32.605507  6499 ts_tablet_manager.cc:1403] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.002s
I20260709 12:24:32.607987  6580 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:32.608912  6499 raft_consensus.cc:359] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.609856  6499 raft_consensus.cc:385] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.610103  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.610214  6499 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.610860  6457 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:32.611264  6499 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.614068  6499 ts_tablet_manager.cc:1434] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:24:32.616591  6500 tablet_bootstrap.cc:492] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:32.617198  6580 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.617225  6500 ts_tablet_manager.cc:1403] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.001s
I20260709 12:24:32.617744  6580 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.619778  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.621151  6500 raft_consensus.cc:359] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.621809  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.622104  6500 raft_consensus.cc:385] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.622473  6310 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:32.624681  6499 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:32.632714  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.624521  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.633427  6307 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:32.627557  6383 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.623229  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.622550  6500 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.635010  6580 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.645073  6500 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.647178  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:32.552634  6206 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 2, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:32.657634  6500 ts_tablet_manager.cc:1434] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.000s
I20260709 12:24:32.658195  6501 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:32.658831  6501 ts_tablet_manager.cc:1403] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.265s	user 0.040s	sys 0.060s
I20260709 12:24:32.677594  6272 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:32.678855  6582 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:32.679229  6582 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.679562  6582 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.678731  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.682439  6459 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:32.684465  6499 tablet_bootstrap.cc:654] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.692414  6500 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:32.692857  6351 consensus_peers.cc:597] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:32.695029  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:32.696419  6641 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:32.696777  6641 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.697116  6641 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:32.700440  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45640.5us
W20260709 12:24:32.703081  6425 leader_election.cc:343] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.703699  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:32.718989  6501 raft_consensus.cc:359] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.720587  6501 raft_consensus.cc:385] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.720880  6501 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.724871  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:32.724496  6501 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.724470  6745 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.728127  6744 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:32.744009  6744 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.744359  6744 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.749116  6501 ts_tablet_manager.cc:1434] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.004s
I20260709 12:24:32.750571  6501 tablet_bootstrap.cc:492] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:32.750716  6499 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:32.751606  6499 ts_tablet_manager.cc:1403] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.127s	user 0.030s	sys 0.012s
I20260709 12:24:32.755088  6499 raft_consensus.cc:359] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.755957  6499 raft_consensus.cc:385] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:32.756584  6499 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:32.726133  6776 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.757588  6499 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.766772  6501 tablet_bootstrap.cc:654] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:32.777297  6500 tablet_bootstrap.cc:654] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:32.779632  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52912.5us
I20260709 12:24:32.780745  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:32.767091  6756 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.788807  6499 ts_tablet_manager.cc:1434] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.001s
I20260709 12:24:32.791687  6499 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:32.738014  6745 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.812237  6776 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:32.812685  6776 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.813006  6776 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.799643  6756 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.835013  6580 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.816673  6738 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:24:32.838738  6641 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.843056  6745 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:32.844976  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.845276  6582 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.847289  6582 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.859670  6756 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.852187  6641 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.860247  6576 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.865659  6424 leader_election.cc:343] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.866130  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.860769  6576 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.868503  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.869657  6576 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:32.870787  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:24:32.872347  6425 leader_election.cc:343] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.873443  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:32.871413  6454 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.886950  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:32.895640  6424 leader_election.cc:343] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.882812  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:32.887904  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:32.907066  6457 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:32.913039  6425 leader_election.cc:343] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:32.913790  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:32.915053  6641 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.915782  6641 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.920089  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:32.920950  6458 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:32.918401  6641 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:32.926381  6454 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:32.931691  6779 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.932399  6779 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.942386  6580 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.953583  6776 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.950213  6580 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.955133  6744 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:32.960289  6744 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:32.942430  6745 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.976989  6272 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:32.978060  6641 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:32.978446  6641 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:32.979663  6641 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.938565  6783 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:32.992229  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:32.995714  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:32.996625  6309 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:32.998636  6455 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:32.992312  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:32.999436  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:32.994189  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:33.000757  6272 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:33.001211  6424 leader_election.cc:343] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.003340  6580 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.003690  6753 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.004314  6753 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:33.005352  6753 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.006067  6779 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.007221  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.007723  6740 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:33.009078  6641 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.010866  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.011584  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:33.008738  6776 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.016242  6641 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.026707  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.027413  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:33.029740  6456 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.030150  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:33.031126  6310 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:33.031957  6424 leader_election.cc:343] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:33.032773  6424 leader_election.cc:343] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.051321  6310 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:33.054191  6499 tablet_bootstrap.cc:654] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.045451  6783 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.054311  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.060302  6783 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.046839  6308 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:33.069659  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.064704  6782 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.071171  6501 tablet_bootstrap.cc:492] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:33.071686  6501 ts_tablet_manager.cc:1403] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.321s	user 0.015s	sys 0.007s
I20260709 12:24:33.072312  6456 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:33.079778  6500 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.080250  6500 ts_tablet_manager.cc:1403] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.388s	user 0.028s	sys 0.013s
I20260709 12:24:33.083176  6425 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.074802  6745 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.085446  6501 raft_consensus.cc:359] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.086427  6501 raft_consensus.cc:385] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.086757  6501 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.088169  6457 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:33.077070  6782 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:33.101025  6272 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:33.100862  6501 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.092085  6782 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.112444  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:33.103545  6740 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.116302  6501 ts_tablet_manager.cc:1434] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.044s	user 0.012s	sys 0.002s
I20260709 12:24:33.114274  6783 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.089920  6745 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.105279  6576 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.120303  6501 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:33.108072  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:33.122741  6576 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.127092  6576 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.114712  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.124289  6740 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:33.131551  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.119293  6640 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.133251  6740 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.154371  6499 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.154989  6499 ts_tablet_manager.cc:1403] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.364s	user 0.015s	sys 0.009s
I20260709 12:24:33.154608  6500 raft_consensus.cc:359] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.160104  6500 raft_consensus.cc:385] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.160400  6500 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.142572  6640 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:33.163335  6640 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.164618  6383 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.167675  6500 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.169358  6745 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.153937  6309 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:33.170449  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:33.145710  6274 leader_election.cc:343] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.184273  6500 ts_tablet_manager.cc:1434] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.001s
I20260709 12:24:33.187803  6500 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.175740  6274 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:33.183573  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:33.238406  6425 leader_election.cc:343] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.239167  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.258781  6745 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.259352  6745 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:33.260262  6745 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.213012  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.214535  6753 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.267446  6305 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:33.271771  6424 leader_election.cc:343] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.261812  6576 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.288975  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:33.290787  6424 leader_election.cc:343] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:33.297823  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58719.8us
W20260709 12:24:33.298720  6424 leader_election.cc:343] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.299290  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.300400  6744 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:33.301792  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 126121us
W20260709 12:24:33.303093  6274 leader_election.cc:343] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.306139  6576 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.310453  6501 tablet_bootstrap.cc:700] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent opening tablet: real 0.119s	user 0.010s	sys 0.009s
I20260709 12:24:33.311285  6501 tablet_bootstrap.cc:654] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:33.312000  6274 leader_election.cc:343] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:33.313242  6274 leader_election.cc:343] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.316582  6500 tablet_bootstrap.cc:654] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.321352  6203 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.325459  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:33.326138  6304 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.338848  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.339931  6783 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.340482  6783 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.342511  6783 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.342856  6739 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.346145  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:33.352999  6425 leader_election.cc:343] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.353454  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.354182  6384 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:33.355049  6783 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.354988  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.360584  6753 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:33.314100  6206 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.362355  6499 raft_consensus.cc:359] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.363130  6499 raft_consensus.cc:385] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.363421  6499 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.364051  6499 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:33.364856  6424 leader_election.cc:343] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.365365  6304 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
W20260709 12:24:33.367957  6425 leader_election.cc:343] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.368381  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.367862  6500 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.372288  6500 ts_tablet_manager.cc:1403] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.185s	user 0.018s	sys 0.027s
I20260709 12:24:33.372706  6783 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.373945  6425 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.374729  6783 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.375526  6783 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:33.376601  6499 ts_tablet_manager.cc:1434] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.221s	user 0.033s	sys 0.025s
I20260709 12:24:33.376458  6783 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.377980  6499 tablet_bootstrap.cc:492] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.378087  6500 raft_consensus.cc:359] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.380020  6500 raft_consensus.cc:385] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.380676  6500 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.381620  6500 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.386353  6500 ts_tablet_manager.cc:1434] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:24:33.388773  6500 tablet_bootstrap.cc:492] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.401129  6499 tablet_bootstrap.cc:654] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.402104  6500 tablet_bootstrap.cc:654] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.407956  6501 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:33.408519  6501 ts_tablet_manager.cc:1403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.291s	user 0.022s	sys 0.028s
I20260709 12:24:33.410643  6633 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.412189  6633 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.413156  6501 raft_consensus.cc:359] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.414386  6501 raft_consensus.cc:385] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.414836  6501 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.415560  6500 tablet_bootstrap.cc:492] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.415712  6633 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.416034  6500 ts_tablet_manager.cc:1403] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.001s
I20260709 12:24:33.417358  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.418053  6457 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:33.418951  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.421334  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:33.422056  6499 tablet_bootstrap.cc:492] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.422221  6633 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.415761  6501 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.422554  6499 ts_tablet_manager.cc:1403] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.000s
I20260709 12:24:33.426736  6500 raft_consensus.cc:359] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.427377  6500 raft_consensus.cc:385] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.427573  6501 ts_tablet_manager.cc:1434] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.002s
I20260709 12:24:33.427623  6500 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.428490  6501 tablet_bootstrap.cc:492] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:33.429544  6500 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.430100  6499 raft_consensus.cc:359] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.431156  6499 raft_consensus.cc:385] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.431473  6499 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.432335  6499 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.434556  6499 ts_tablet_manager.cc:1434] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:24:33.435494  6499 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.437456  6500 ts_tablet_manager.cc:1434] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 12:24:33.441406  6633 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.442754  6500 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.448238  6500 tablet_bootstrap.cc:654] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.441226  6501 tablet_bootstrap.cc:654] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.457995  6499 tablet_bootstrap.cc:654] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:33.472930  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:33.473562  6500 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.474532  6500 ts_tablet_manager.cc:1403] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.000s
I20260709 12:24:33.472615  6499 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.475805  6499 ts_tablet_manager.cc:1403] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.000s
I20260709 12:24:33.476835  6206 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.478454  6499 raft_consensus.cc:359] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.481266  6499 raft_consensus.cc:385] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.481803  6499 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.480218  6206 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:33.482456  6499 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:33.483489  6274 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:33.484582  6499 ts_tablet_manager.cc:1434] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:24:33.480472  6500 raft_consensus.cc:359] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.485248  6500 raft_consensus.cc:385] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.485526  6500 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.485621  6499 tablet_bootstrap.cc:492] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.486220  6500 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.488624  6500 ts_tablet_manager.cc:1434] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:24:33.490906  6501 tablet_bootstrap.cc:492] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:33.491617  6501 ts_tablet_manager.cc:1403] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.063s	user 0.007s	sys 0.018s
I20260709 12:24:33.494411  6499 tablet_bootstrap.cc:654] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.494338  6501 raft_consensus.cc:359] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.495434  6501 raft_consensus.cc:385] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.495774  6501 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.496574  6501 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.498080  6205 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.502360  6500 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.502655  6205 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.509433  6500 tablet_bootstrap.cc:654] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.505229  6501 ts_tablet_manager.cc:1434] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:24:33.507330  6499 tablet_bootstrap.cc:492] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.510586  6499 ts_tablet_manager.cc:1403] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.007s
I20260709 12:24:33.513603  6501 tablet_bootstrap.cc:492] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:33.516597  6499 raft_consensus.cc:359] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.517134  6499 raft_consensus.cc:385] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.517377  6499 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.518021  6499 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.520042  6499 ts_tablet_manager.cc:1434] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:24:33.520871  6499 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.520938  6500 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.521538  6500 ts_tablet_manager.cc:1403] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 12:24:33.522629  6783 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.524251  6500 raft_consensus.cc:359] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.524911  6500 raft_consensus.cc:385] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.525221  6500 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.523145  6783 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.525933  6500 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.528159  6500 ts_tablet_manager.cc:1434] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:24:33.528513  6501 tablet_bootstrap.cc:654] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.529469  6500 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.536028  6500 tablet_bootstrap.cc:654] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.536276  6499 tablet_bootstrap.cc:654] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.538415  6753 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.539000  6753 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.544430  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.544513  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.545336  6457 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
W20260709 12:24:33.546933  6274 leader_election.cc:343] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.547310  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.547534  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.548400  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.548609  6578 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.549166  6578 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.549518  6578 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.548548  6500 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
W20260709 12:24:33.549813  6424 leader_election.cc:343] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.550168  6500 ts_tablet_manager.cc:1403] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
W20260709 12:24:33.550850  6425 leader_election.cc:343] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.551236  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.549751  6753 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.552736  6500 raft_consensus.cc:359] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.553382  6500 raft_consensus.cc:385] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.553647  6500 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.554273  6500 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.552013  6782 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.556571  6499 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.557036  6499 ts_tablet_manager.cc:1403] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.008s
I20260709 12:24:33.559448  6499 raft_consensus.cc:359] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.560065  6499 raft_consensus.cc:385] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.560339  6499 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.562254  6500 ts_tablet_manager.cc:1434] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:24:33.563184  6500 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.546959  6783 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.562744  6578 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.567617  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:33.568956  6500 tablet_bootstrap.cc:654] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:33.569418  6274 leader_election.cc:343] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.570897  6578 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.561489  6499 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.572571  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:33.573160  6457 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.573417  6499 ts_tablet_manager.cc:1434] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.003s
I20260709 12:24:33.574409  6499 tablet_bootstrap.cc:492] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.579901  6457 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:33.581084  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.581406  6499 tablet_bootstrap.cc:654] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.591687  6633 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.592499  6633 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.596709  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.598789  6578 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.603416  6633 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.604826  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.610796  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:33.612385  6630 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.613270  6578 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:33.614446  6499 tablet_bootstrap.cc:492] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.615096  6499 ts_tablet_manager.cc:1403] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.003s
I20260709 12:24:33.614535  6578 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.617679  6499 raft_consensus.cc:359] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.618341  6499 raft_consensus.cc:385] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.620357  6499 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.623726  6753 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.624218  6753 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.625017  6756 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.625495  6756 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.626276  6753 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.627591  6756 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.628304  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.628899  6457 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:33.629166  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.630215  6272 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:33.630359  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.630554  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.631140  6457 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
W20260709 12:24:33.631875  6274 leader_election.cc:343] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.632357  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:33.632905  6274 leader_election.cc:343] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.633136  6753 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.633622  6753 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.633872  6756 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.634032  6753 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.634305  6756 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.634742  6756 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.620958  6499 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.636485  6499 ts_tablet_manager.cc:1434] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 12:24:33.637385  6499 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.635995  6501 tablet_bootstrap.cc:492] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:33.640854  6501 ts_tablet_manager.cc:1403] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.128s	user 0.018s	sys 0.045s
I20260709 12:24:33.641362  6753 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.642874  6779 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.643314  6779 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.646400  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.647167  6499 tablet_bootstrap.cc:654] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.647851  6582 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:33.647930  6779 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.649610  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.650097  6582 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:24:33.647847  6424 leader_election.cc:343] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:33.652105  6425 leader_election.cc:343] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.652070  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:33.652719  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.652971  6455 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.653898  6779 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.650249  6500 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.656773  6500 ts_tablet_manager.cc:1403] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.023s
I20260709 12:24:33.658087  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:33.660224  6274 leader_election.cc:343] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.666885  6455 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:33.667280  6501 raft_consensus.cc:359] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.668059  6500 raft_consensus.cc:359] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.670723  6500 raft_consensus.cc:385] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.671190  6500 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.650177  6753 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.669905  6272 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.672096  6500 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.676977  6500 ts_tablet_manager.cc:1434] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.003s
I20260709 12:24:33.673214  6753 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.677117  6756 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.679582  6500 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.677874  6753 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:33.680581  6753 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.681103  6501 raft_consensus.cc:385] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.682425  6501 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.682720  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:33.684100  6501 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.686558  6501 ts_tablet_manager.cc:1434] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 12:24:33.687614  6501 tablet_bootstrap.cc:492] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
W20260709 12:24:33.705542  6424 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:33.707304  6274 leader_election.cc:343] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.716538  6783 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:33.717978  6206 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.716596  6500 tablet_bootstrap.cc:654] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.723522  6578 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:24:33.729003  6756 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.743850  6630 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.745158  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:33.745813  6457 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.744496  6630 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.752537  6630 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.756094  6578 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:33.758097  6782 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.758684  6782 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.761089  6782 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.763420  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:33.765077  6425 leader_election.cc:343] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.778255  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.779259  6304 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:33.781543  6501 tablet_bootstrap.cc:654] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.785406  6457 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:33.786834  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:24:33.787081  6274 consensus_peers.cc:597] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:33.790059  6756 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:33.791282  6756 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:33.791929  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.792552  6499 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:33.792213  6756 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.793128  6499 ts_tablet_manager.cc:1403] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.156s	user 0.027s	sys 0.047s
W20260709 12:24:33.794539  6424 leader_election.cc:343] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.795061  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.796327  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.800048  6205 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.798583  6745 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.806370  6641 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.806891  6641 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.808768  6641 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:33.813195  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:33.815543  6738 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.824512  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:33.828629  6274 leader_election.cc:343] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.830343  6499 raft_consensus.cc:359] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.831188  6499 raft_consensus.cc:385] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.831467  6499 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.832170  6499 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.834151  6782 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:33.830265  6745 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.835798  6745 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.850293  6499 ts_tablet_manager.cc:1434] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.001s
I20260709 12:24:33.851468  6499 tablet_bootstrap.cc:492] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:33.838161  6745 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.854534  6783 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.857471  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.857867  6783 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.859179  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.860464  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.862365  6578 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.876366  6500 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:33.876942  6500 ts_tablet_manager.cc:1403] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.198s	user 0.021s	sys 0.021s
I20260709 12:24:33.875976  6578 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.882213  6578 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:33.882990  6425 leader_election.cc:343] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.874233  6455 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:33.890785  6782 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:33.891399  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:33.892652  6738 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.894160  6384 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:33.894804  6424 leader_election.cc:343] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.895318  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:33.896821  6745 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:33.901510  6782 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:33.901203  6205 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:33.905274  6783 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:33.905160  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:33.906975  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:33.905982  6274 leader_election.cc:343] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.911589  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.913056  6500 raft_consensus.cc:359] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.913833  6500 raft_consensus.cc:385] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.914258  6500 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:33.911993  6459 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:33.920321  6500 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.924679  6501 tablet_bootstrap.cc:492] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:33.925165  6501 ts_tablet_manager.cc:1403] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.238s	user 0.016s	sys 0.000s
W20260709 12:24:33.926815  6424 leader_election.cc:343] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:33.919185  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:33.928164  6500 ts_tablet_manager.cc:1434] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:24:33.930020  6500 tablet_bootstrap.cc:492] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:33.921416  6753 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.940076  6753 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.940439  6753 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.957712  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:33.961301  6641 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:33.961830  6641 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.962212  6641 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:33.968894  6499 tablet_bootstrap.cc:654] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:33.975852  6501 raft_consensus.cc:359] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:33.976886  6501 raft_consensus.cc:385] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:33.977241  6501 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
W20260709 12:24:34.028317  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66780.3us
I20260709 12:24:34.031040  6745 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.031592  6745 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.034578  6738 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.062566  6425 leader_election.cc:343] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.076946  6738 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.078116  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:34.079408  6641 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.085664  6641 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.131199  6501 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.087004  6578 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.121337  6738 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.132300  6578 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.133100  6753 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.134730  6501 ts_tablet_manager.cc:1434] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.209s	user 0.009s	sys 0.000s
I20260709 12:24:34.085767  6782 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.135396  6753 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.136929  6500 tablet_bootstrap.cc:654] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.134758  6578 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.152879  6501 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:34.190405  6578 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.203776  6833 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:24:34.203959  6578 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.205746  6745 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:34.206089  6578 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.217581  6499 tablet_bootstrap.cc:492] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:34.218189  6499 ts_tablet_manager.cc:1403] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.367s	user 0.018s	sys 0.036s
I20260709 12:24:34.196112  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:34.196293  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46427.9us
W20260709 12:24:34.223474  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 77243.2us
I20260709 12:24:34.223054  6305 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:34.233410  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.234087  6457 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 0.
I20260709 12:24:34.234265  6499 raft_consensus.cc:359] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.236326  6499 raft_consensus.cc:385] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.237650  6499 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.233310  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.251551  6500 tablet_bootstrap.cc:492] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:34.252722  6500 ts_tablet_manager.cc:1403] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.323s	user 0.022s	sys 0.017s
W20260709 12:24:34.253417  6425 leader_election.cc:343] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.257506  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:34.258675  6739 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.259086  6739 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.259438  6739 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.259676  6501 tablet_bootstrap.cc:654] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.241304  6499 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.267379  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:34.274358  6500 raft_consensus.cc:359] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.276404  6500 raft_consensus.cc:385] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.276695  6500 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
W20260709 12:24:34.278250  6424 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:34.280328  6499 ts_tablet_manager.cc:1434] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
W20260709 12:24:34.280745  6424 leader_election.cc:343] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.281131  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:34.281286  6499 tablet_bootstrap.cc:492] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:34.277395  6500 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.284813  6779 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.287748  6500 ts_tablet_manager.cc:1434] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.001s
W20260709 12:24:34.295969  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 217086us
I20260709 12:24:34.303488  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:34.305965  6739 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.309286  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:34.309844  6456 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.312983  6500 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:34.316448  6456 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:34.317587  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:34.318267  6633 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.318771  6633 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:34.319537  6633 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.331126  6739 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.331485  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:34.335395  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:34.338174  6501 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:34.340749  6501 ts_tablet_manager.cc:1403] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.188s	user 0.023s	sys 0.020s
I20260709 12:24:34.342211  6739 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.342793  6739 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.346426  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.388798  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:34.389472  6455 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.391754  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.392500  6458 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:34.393313  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:34.393898  6452 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.394721  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:34.362123  6739 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.418370  6452 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:34.418355  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.419557  6457 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:34.422264  6501 raft_consensus.cc:359] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.423053  6501 raft_consensus.cc:385] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.423334  6501 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.424063  6501 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.428100  6455 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:34.428805  6500 tablet_bootstrap.cc:654] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.428807  6499 tablet_bootstrap.cc:654] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.431787  6501 ts_tablet_manager.cc:1434] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.091s	user 0.018s	sys 0.006s
I20260709 12:24:34.434541  6501 tablet_bootstrap.cc:492] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
W20260709 12:24:34.443118  6274 leader_election.cc:343] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:34.444887  6274 leader_election.cc:343] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:34.445772  6274 leader_election.cc:343] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:34.446709  6274 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:34.450069  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:34.450802  6307 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.452755  6206 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.453635  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.459486  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:34.461527  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.462255  6641 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:34.463519  6582 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.465104  6582 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.465384  6582 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.464385  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.465816  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:34.467178  6839 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.468731  6839 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.469353  6840 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.469906  6840 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:24:34.473603  6274 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:34.474423  6274 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:34.475682  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.477798  6582 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.479312  6745 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.479859  6745 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.470794  6840 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.486784  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:34.486960  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:34.470813  6839 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:34.487995  6499 tablet_bootstrap.cc:492] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:34.488481  6499 ts_tablet_manager.cc:1403] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.207s	user 0.006s	sys 0.013s
W20260709 12:24:34.489282  6424 leader_election.cc:343] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:34.490067  6424 leader_election.cc:343] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.489585  6630 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.482102  6745 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:34.499198  6307 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:34.500905  6582 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.501561  6640 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.510828  6500 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:34.511379  6500 ts_tablet_manager.cc:1403] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.199s	user 0.019s	sys 0.017s
I20260709 12:24:34.501554  6835 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.501569  6753 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:24:34.515412  6835 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:34.534843  6835 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.543752  6500 raft_consensus.cc:359] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.544966  6500 raft_consensus.cc:385] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.545526  6500 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.513268  6633 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.546411  6500 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.550284  6501 tablet_bootstrap.cc:654] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.549885  6499 raft_consensus.cc:359] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:34.551059  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48984.9us
I20260709 12:24:34.550839  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:34.554937  6499 raft_consensus.cc:385] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.556818  6381 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.558022  6500 ts_tablet_manager.cc:1434] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.001s
I20260709 12:24:34.557652  6499 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.559782  6499 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.561390  6500 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
W20260709 12:24:34.569569  6274 leader_election.cc:343] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.569986  6582 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:34.550153  6633 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.579567  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.580349  6499 ts_tablet_manager.cc:1434] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.004s
I20260709 12:24:34.583580  6499 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:34.585880  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.582000  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.578209  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:34.581210  6633 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.592860  6456 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.591398  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.597150  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:34.603024  6630 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:34.606312  6425 leader_election.cc:343] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.606767  6425 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:34.609429  6425 leader_election.cc:343] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.610236  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:34.599977  6203 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.620569  6745 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.622368  6839 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.640136  6383 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.649652  6456 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:34.652441  6500 tablet_bootstrap.cc:654] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.667725  6499 tablet_bootstrap.cc:654] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.657555  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.678884  6500 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:34.678884  6641 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.678845  6501 tablet_bootstrap.cc:492] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:34.680871  6641 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:34.681025  6501 ts_tablet_manager.cc:1403] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.247s	user 0.014s	sys 0.005s
I20260709 12:24:34.680480  6500 ts_tablet_manager.cc:1403] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.002s
I20260709 12:24:34.681710  6641 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.686565  6205 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.690378  6500 raft_consensus.cc:359] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.691824  6500 raft_consensus.cc:385] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.692327  6500 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.691504  6499 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:34.694796  6499 ts_tablet_manager.cc:1403] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.112s	user 0.013s	sys 0.002s
I20260709 12:24:34.703946  6501 raft_consensus.cc:359] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.704762  6501 raft_consensus.cc:385] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.705066  6501 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.705806  6501 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.711616  6500 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.720187  6500 ts_tablet_manager.cc:1434] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:24:34.721062  6500 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Bootstrap starting.
I20260709 12:24:34.723526  6501 ts_tablet_manager.cc:1434] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.042s	user 0.006s	sys 0.000s
I20260709 12:24:34.730969  6499 raft_consensus.cc:359] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.731715  6499 raft_consensus.cc:385] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.732002  6499 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.732611  6499 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.736531  6499 ts_tablet_manager.cc:1434] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.000s
I20260709 12:24:34.737404  6499 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:34.746541  6834 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.747100  6834 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.749315  6834 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.750617  6500 tablet_bootstrap.cc:654] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.753258  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:34.755167  6274 leader_election.cc:343] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.758087  6501 tablet_bootstrap.cc:492] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:34.774019  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.774782  6457 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:34.776046  6206 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:34.786531  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.788059  6641 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.788512  6641 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.788908  6641 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.794415  6630 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:34.799279  6501 tablet_bootstrap.cc:654] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.817950  6501 tablet_bootstrap.cc:492] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:34.818432  6501 ts_tablet_manager.cc:1403] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.061s	user 0.010s	sys 0.003s
I20260709 12:24:34.818127  6630 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:34.819819  6834 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.820302  6834 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.822268  6834 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.823650  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:34.825217  6274 leader_election.cc:343] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.825798  6499 tablet_bootstrap.cc:654] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.827946  6501 raft_consensus.cc:359] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.832957  6501 raft_consensus.cc:385] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.833274  6501 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.834561  6501 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.836864  6501 ts_tablet_manager.cc:1434] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:24:34.836557  6745 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.837739  6501 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:34.837823  6745 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.847364  6745 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:34.849822  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:34.851277  6424 leader_election.cc:343] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.859243  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.859911  6455 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:34.862242  6501 tablet_bootstrap.cc:654] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:34.882933  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:34.833685  6641 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.889736  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:34.891186  6274 leader_election.cc:343] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.893934  6641 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.895444  6501 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:34.897276  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.898458  6500 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: No bootstrap required, opened a new log
I20260709 12:24:34.901383  6500 ts_tablet_manager.cc:1403] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent bootstrapping tablet: real 0.180s	user 0.040s	sys 0.056s
I20260709 12:24:34.912242  6872 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.912724  6872 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.914564  6501 ts_tablet_manager.cc:1403] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.003s
I20260709 12:24:34.916009  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:34.917732  6424 leader_election.cc:343] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.914728  6872 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:34.920138  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:34.920822  6425 leader_election.cc:343] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.921238  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:34.923624  6837 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:34.921868  6501 raft_consensus.cc:359] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.924852  6501 raft_consensus.cc:385] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.925267  6500 raft_consensus.cc:359] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.926030  6500 raft_consensus.cc:385] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.926258  6501 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.927275  6501 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.929937  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:34.930341  6501 ts_tablet_manager.cc:1434] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:24:34.930609  6455 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.931396  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.931896  6501 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:34.926303  6500 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.939157  6500 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.940388  6501 tablet_bootstrap.cc:654] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.941972  6499 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:34.942857  6499 ts_tablet_manager.cc:1403] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.206s	user 0.015s	sys 0.016s
I20260709 12:24:34.940505  6455 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:34.945719  6835 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:34.946134  6835 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.946530  6835 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:34.951030  6499 raft_consensus.cc:359] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.951853  6499 raft_consensus.cc:385] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:34.952179  6499 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:34.953045  6499 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.954070  6835 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.955935  6499 ts_tablet_manager.cc:1434] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:24:34.958215  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:34.958582  6499 tablet_bootstrap.cc:492] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
W20260709 12:24:34.959674  6274 leader_election.cc:343] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.960306  6500 ts_tablet_manager.cc:1434] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent starting tablet: real 0.059s	user 0.007s	sys 0.000s
I20260709 12:24:34.967007  6499 tablet_bootstrap.cc:654] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:34.971165  6835 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.976785  6501 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:34.977278  6501 ts_tablet_manager.cc:1403] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.046s	user 0.017s	sys 0.001s
I20260709 12:24:34.977310  6499 tablet_bootstrap.cc:492] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:34.977958  6499 ts_tablet_manager.cc:1403] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.006s
I20260709 12:24:34.983253  6633 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.983812  6633 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.986137  6633 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:34.987327  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:34.987327  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:34.988382  6853 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:34.988862  6853 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:34.989741  6853 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:34.992885  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:34.994725  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:34.996116  6633 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:34.997748  6425 leader_election.cc:343] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:34.998456  6425 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:34.999584  6837 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.000325  6499 raft_consensus.cc:359] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.000504  6501 raft_consensus.cc:359] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.001128  6499 raft_consensus.cc:385] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.001327  6381 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.001294  6501 raft_consensus.cc:385] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.001494  6499 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.002034  6501 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.002766  6499 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.002978  6501 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.004628  6633 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.005110  6499 ts_tablet_manager.cc:1434] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.005s
I20260709 12:24:35.005131  6633 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.006472  6499 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.007843  6501 ts_tablet_manager.cc:1434] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:24:35.008667  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.008728  6501 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:35.012399  6630 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.013548  6630 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.019238  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:35.019790  6457 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.022723  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.024367  6348 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:35.025880  6457 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:35.025309  6739 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.022759  6633 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.027824  6630 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.030570  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.029760  6854 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:24:35.031095  6206 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.033118  6499 tablet_bootstrap.cc:654] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.034298  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.035220  6854 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.035423  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.035710  6854 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:35.035668  6853 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:35.036794  6837 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.036751  6854 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.037307  6837 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.038769  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:35.040115  6837 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:35.041174  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:35.043449  6424 leader_election.cc:343] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.053818  6630 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.056118  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:35.065846  6425 leader_election.cc:343] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.066354  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:35.076112  6837 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:35.078423  6274 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:35.081277  6274 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:35.099208  6501 tablet_bootstrap.cc:654] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.100404  6380 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.112862  6641 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.113595  6641 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.117242  6641 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.117532  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:35.119331  6274 leader_election.cc:343] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.127256  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.135991  6457 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:35.137058  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.142911  6578 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.143337  6578 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.143739  6578 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.143359  6499 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:35.144522  6499 ts_tablet_manager.cc:1403] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.138s	user 0.012s	sys 0.004s
I20260709 12:24:35.148401  6499 raft_consensus.cc:359] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.149220  6499 raft_consensus.cc:385] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.149518  6499 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.150142  6499 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.161173  6501 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:35.161756  6501 ts_tablet_manager.cc:1403] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.153s	user 0.024s	sys 0.014s
I20260709 12:24:35.162266  6206 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.164535  6501 raft_consensus.cc:359] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.165652  6501 raft_consensus.cc:385] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.165925  6501 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.172299  6853 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.172911  6853 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:35.175312  6274 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:35.175597  6501 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.159677  6499 ts_tablet_manager.cc:1434] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:24:35.177095  6853 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.178786  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:35.179792  6501 ts_tablet_manager.cc:1434] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.018s	user 0.002s	sys 0.005s
I20260709 12:24:35.156633  6578 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:35.180328  6274 leader_election.cc:343] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.181178  6499 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.184214  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:35.180786  6501 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:35.191745  6840 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.192234  6840 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:35.186206  6274 leader_election.cc:343] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.195827  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.196610  6455 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:35.198247  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:35.200444  6835 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.200942  6835 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.201470  6835 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.202558  6840 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.203583  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:35.207360  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.208334  6875 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.225669  6875 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.212365  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:35.228802  6875 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:35.209429  6274 leader_election.cc:343] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.240005  6499 tablet_bootstrap.cc:654] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.240440  6501 tablet_bootstrap.cc:654] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.245195  6835 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.246656  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:35.249159  6835 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.249379  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:35.250070  6459 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.256083  6499 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:35.256721  6499 ts_tablet_manager.cc:1403] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.079s	user 0.013s	sys 0.000s
I20260709 12:24:35.257318  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:35.257768  6424 leader_election.cc:343] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.258003  6307 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:35.260721  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:35.261349  6499 raft_consensus.cc:359] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.262097  6499 raft_consensus.cc:385] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.262367  6499 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.262753  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.263069  6499 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.265283  6499 ts_tablet_manager.cc:1434] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:24:35.265462  6853 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.265506  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:35.266242  6853 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.266342  6455 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.266822  6853 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.267252  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.268530  6875 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.268898  6875 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.269210  6875 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.262638  6578 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.274315  6455 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
W20260709 12:24:35.275338  6274 leader_election.cc:343] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.266268  6499 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.281627  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.284986  6835 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.286092  6835 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:35.287058  6835 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.288152  6499 tablet_bootstrap.cc:654] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.291824  6854 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:35.265286  6459 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:35.297681  6381 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.313887  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.317904  6578 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.323192  6578 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:35.283178  6875 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.334295  6578 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.364706  6875 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:35.388967  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:35.354552  6630 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.386245  6206 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.399423  6630 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.401595  6630 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.408063  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.411104  6385 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.459470  6499 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:35.460031  6499 ts_tablet_manager.cc:1403] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.194s	user 0.049s	sys 0.029s
W20260709 12:24:35.466434  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64982.3us
W20260709 12:24:35.471020  6424 leader_election.cc:343] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.469950  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.480662  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:35.474846  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:35.489540  6501 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:35.490123  6501 ts_tablet_manager.cc:1403] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.310s	user 0.035s	sys 0.062s
I20260709 12:24:35.491250  6641 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:35.492069  6499 raft_consensus.cc:359] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.492734  6499 raft_consensus.cc:385] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.493028  6499 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.487938  6304 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.493851  6499 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.507701  6853 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.526757  6853 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.530602  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:35.533550  6274 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:35.537428  6501 raft_consensus.cc:359] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.541662  6304 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:35.549909  6499 ts_tablet_manager.cc:1434] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.090s	user 0.007s	sys 0.002s
W20260709 12:24:35.550549  6274 leader_election.cc:343] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.552074  6499 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.553313  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.538319  6501 raft_consensus.cc:385] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.554896  6501 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.555821  6501 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.560197  6839 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.570665  6839 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:35.571590  6839 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.573175  6499 tablet_bootstrap.cc:654] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.574321  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:35.576090  6205 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.577453  6835 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:35.580241  6840 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.580700  6840 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.581816  6453 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.592587  6633 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.603372  6840 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.610574  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.611418  6459 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:35.612923  6272 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:35.617220  6835 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.617660  6835 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.618057  6835 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.623308  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:35.626914  6274 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:35.627661  6501 ts_tablet_manager.cc:1434] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.137s	user 0.022s	sys 0.011s
W20260709 12:24:35.627815  6274 leader_election.cc:343] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.628796  6501 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:35.644085  6385 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.681247  6205 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:35.702796  6501 tablet_bootstrap.cc:654] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
W20260709 12:24:35.718456  6892 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:24:35.727579  6274 consensus_peers.cc:597] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:24:35.751425  6274 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:35.753824  6839 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.753715  6835 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.754556  6839 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.756067  6499 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:35.756907  6499 ts_tablet_manager.cc:1403] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.205s	user 0.011s	sys 0.009s
I20260709 12:24:35.756382  6835 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.758805  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:35.759590  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.770272  6839 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:35.771190  6307 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:35.772128  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:35.773721  6424 leader_election.cc:343] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.775060  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.759606  6452 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.781630  6893 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:35.789184  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:35.789433  6499 raft_consensus.cc:359] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.790172  6499 raft_consensus.cc:385] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.790407  6499 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.787364  6893 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.791234  6893 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:24:35.792985  6274 leader_election.cc:343] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.797430  6897 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:35.800611  6897 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.803386  6453 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:35.807930  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.822011  6499 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.820815  6900 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.828122  6900 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:35.828012  6893 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.829013  6900 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.830516  6897 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.832028  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.835788  6452 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:35.837083  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.841846  6893 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:35.843061  6499 ts_tablet_manager.cc:1434] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.086s	user 0.010s	sys 0.013s
I20260709 12:24:35.844048  6499 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.844494  6351 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:35.845507  6897 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:35.847951  6272 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.850143  6578 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.850625  6578 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:35.851768  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:35.852900  6424 leader_election.cc:343] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:35.851821  6835 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:24:35.851372  6578 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.862131  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:35.862901  6307 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.863123  6501 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:35.863663  6501 ts_tablet_manager.cc:1403] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.235s	user 0.012s	sys 0.012s
I20260709 12:24:35.865015  6499 tablet_bootstrap.cc:654] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.866561  6501 raft_consensus.cc:359] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.867270  6501 raft_consensus.cc:385] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.867558  6501 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.868363  6501 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.871511  6501 ts_tablet_manager.cc:1434] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:24:35.872320  6501 tablet_bootstrap.cc:492] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:35.877691  6501 tablet_bootstrap.cc:654] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.880106  6897 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.880600  6897 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.883880  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.886406  6307 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:35.889546  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:35.889621  6499 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:35.890524  6499 ts_tablet_manager.cc:1403] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.047s	user 0.009s	sys 0.003s
I20260709 12:24:35.890625  6893 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:35.890936  6739 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.891119  6893 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:35.891476  6739 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.892023  6893 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.892839  6897 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.893811  6739 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.895987  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.896826  6499 raft_consensus.cc:359] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.897495  6499 raft_consensus.cc:385] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.897313  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:35.897855  6499 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.899662  6348 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:35.902253  6897 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.902976  6499 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.909157  6499 ts_tablet_manager.cc:1434] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:24:35.909885  6578 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:35.910677  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:35.914202  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:35.915199  6739 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:35.911783  6499 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.917433  6501 tablet_bootstrap.cc:492] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:35.918033  6501 ts_tablet_manager.cc:1403] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.001s
I20260709 12:24:35.918726  6204 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.920902  6501 raft_consensus.cc:359] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.922920  6501 raft_consensus.cc:385] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.923183  6501 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.923138  6203 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:35.933490  6499 tablet_bootstrap.cc:654] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.924391  6501 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.940475  6501 ts_tablet_manager.cc:1434] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 12:24:35.940521  6382 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:35.943858  6501 tablet_bootstrap.cc:492] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:35.952207  6501 tablet_bootstrap.cc:654] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.935710  6204 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:35.966734  6499 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
I20260709 12:24:35.967312  6499 ts_tablet_manager.cc:1403] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.017s
I20260709 12:24:35.970088  6499 raft_consensus.cc:359] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.970801  6499 raft_consensus.cc:385] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.971076  6499 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.971711  6499 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.977118  6499 ts_tablet_manager.cc:1434] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:24:35.977975  6499 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Bootstrap starting.
I20260709 12:24:35.978411  6501 tablet_bootstrap.cc:492] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:35.978976  6501 ts_tablet_manager.cc:1403] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.010s
I20260709 12:24:35.982853  6501 raft_consensus.cc:359] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.983552  6501 raft_consensus.cc:385] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:35.983594  6499 tablet_bootstrap.cc:654] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:35.983923  6501 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:35.984804  6501 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.987154  6501 ts_tablet_manager.cc:1434] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:24:35.988085  6501 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:35.991745  6834 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.992266  6834 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:35.994386  6834 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:35.997586  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:36.000438  6274 leader_election.cc:343] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.003031  6893 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:36.007663  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:36.008364  6453 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:36.009583  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:36.010782  6854 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.011183  6854 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.011546  6854 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.024608  6854 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.027951  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:36.029206  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:24:36.029323  6274 leader_election.cc:343] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.030153  6453 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.031193  6854 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:36.032159  6501 tablet_bootstrap.cc:654] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.039636  6499 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: No bootstrap required, opened a new log
W20260709 12:24:36.039659  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:36.040292  6499 ts_tablet_manager.cc:1403] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent bootstrapping tablet: real 0.062s	user 0.015s	sys 0.015s
I20260709 12:24:36.043313  6499 raft_consensus.cc:359] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.043983  6499 raft_consensus.cc:385] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.043759  6453 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:36.044317  6499 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.045514  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:36.047324  6854 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.047868  6854 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:36.048774  6854 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.049823  6501 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:36.050580  6501 ts_tablet_manager.cc:1403] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.016s
I20260709 12:24:36.048478  6499 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:36.053321  6424 consensus_peers.cc:597] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:36.053088  6501 raft_consensus.cc:359] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.053642  6499 ts_tablet_manager.cc:1434] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:24:36.053833  6501 raft_consensus.cc:385] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.054188  6501 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.054979  6834 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.055438  6834 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.057476  6834 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:36.057651  6501 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.060056  6501 ts_tablet_manager.cc:1434] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:24:36.060191  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:36.061076  6501 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:36.061034  6452 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:36.062671  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:36.063503  6834 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:36.063866  6834 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.064191  6834 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.060182  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:36.071957  6274 leader_election.cc:343] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.075513  6501 tablet_bootstrap.cc:654] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.080078  6834 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.082917  6501 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:36.083555  6501 ts_tablet_manager.cc:1403] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.000s
I20260709 12:24:36.083998  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:36.085490  6274 leader_election.cc:343] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:36.084702  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:36.086443  6501 raft_consensus.cc:359] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.087179  6501 raft_consensus.cc:385] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.087491  6501 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.088172  6501 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.088727  6834 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:36.086556  6453 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.089743  6897 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.090199  6897 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.092273  6897 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:36.092440  6501 ts_tablet_manager.cc:1434] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:24:36.094470  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:36.094911  6501 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:36.096312  6453 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:36.099455  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:36.101333  6501 tablet_bootstrap.cc:654] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.101320  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:36.103188  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:36.103335  6854 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:24:36.104782  6739 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:36.106665  6854 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:36.108100  6854 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.110922  6501 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:36.111936  6501 ts_tablet_manager.cc:1403] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 12:24:36.114622  6501 raft_consensus.cc:359] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.115245  6501 raft_consensus.cc:385] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:36.115510  6501 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:36.116139  6501 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.118253  6501 ts_tablet_manager.cc:1434] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:36.119263  6501 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
W20260709 12:24:36.163635  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55377.2us
I20260709 12:24:36.214116  6501 tablet_bootstrap.cc:654] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:36.219648  6385 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.237560  6382 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.270303  6853 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:36.425103  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 159247us
I20260709 12:24:36.448843  6739 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.450129  6739 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.500685  6853 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:36.528028  6384 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.536375  6378 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:36.580173  6739 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:36.581118  6853 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:36.652480  6501 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:36.653374  6501 ts_tablet_manager.cc:1403] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.534s	user 0.018s	sys 0.020s
W20260709 12:24:37.557693  6378 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.993s	user 0.011s	sys 0.004s
W20260709 12:24:37.948794  5142 debug-util.cc:398] Leaking SignalData structure 0x7b080000fa20 after lost signal to thread 6351
W20260709 12:24:37.949678  5142 kernel_stack_watchdog.cc:198] Thread 6351 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 415ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:24:38.263707  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 305) took 1776 ms. Trace:
I20260709 12:24:38.264254  6378 rpcz_store.cc:276] 0709 12:24:36.487852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.491913 (+  4061us) service_pool.cc:224] Handling call
0709 12:24:36.517954 (+ 26041us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.611648 (+1093694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.611722 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.612851 (+  1129us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.891708 (+278857us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:38.248385 (+356677us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.249436 (+  1051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.249589 (+   153us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.263369 (+ 13780us) raft_consensus.cc:1685] finished
0709 12:24:38.263401 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.263597 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176,"prepare.queue_time_us":353901,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":53226,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":1785,"spinlock_wait_cycles":23936,"thread_start_us":633376,"threads_started":2,"wal-append.queue_time_us":632774}
W20260709 12:24:38.331884  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:38.339661  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 295) took 2121 ms. Trace:
I20260709 12:24:38.340166  6381 rpcz_store.cc:276] 0709 12:24:36.217914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.218609 (+   695us) service_pool.cc:224] Handling call
0709 12:24:38.338515 (+2119906us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:24:38.338813 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.339136 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.339153 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.339186 (+    33us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:38.339221 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.339262 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.339418 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29932416}
W20260709 12:24:38.359470  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.821s	user 0.008s	sys 0.026s
W20260709 12:24:38.360209  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.822s	user 0.008s	sys 0.026s
I20260709 12:24:38.369606  6947 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.370241  6947 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.372334  6947 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.374382  6948 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.375267  6948 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.380445  6948 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:38.393443  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800091de0 after lost signal to thread 5140
I20260709 12:24:38.406278  6950 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.407133  6950 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.413715  6950 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.418787  6949 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.414327  6950 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:24:38.420513  6382 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.871s	user 0.018s	sys 0.021s
W20260709 12:24:38.421293  6382 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.872s	user 0.018s	sys 0.021s
I20260709 12:24:38.420727  6949 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.422364  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 292) took 2231 ms. Trace:
I20260709 12:24:38.420030  6950 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.422876  6382 rpcz_store.cc:276] 0709 12:24:36.190710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.191412 (+   702us) service_pool.cc:224] Handling call
0709 12:24:36.236783 (+ 45371us) spinlock_profiling.cc:243] Waited 2.88 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182e9c 00007f6fa0182a5b 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:24:36.237047 (+   264us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.477216 (+240169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.478293 (+  1077us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.534947 (+ 56654us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.546654 (+ 11707us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:36.549170 (+  2516us) log.cc:844] Serialized 14 byte log entry
0709 12:24:38.421778 (+1872608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.421889 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:38.421952 (+    63us) raft_consensus.cc:1685] finished
0709 12:24:38.422068 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.422288 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14462,"prepare.queue_time_us":90010,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":85,"raft.queue_time_us":237637,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":139,"spinlock_wait_cycles":16577920,"thread_start_us":1885379,"threads_started":3,"wal-append.queue_time_us":108181,"wal-append.run_cpu_time_us":3657,"wal-append.run_wall_time_us":1009074}
I20260709 12:24:38.425336  6950 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.427975  6950 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.428634  6950 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.431777  6949 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.433082  6950 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.444015  6925 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.444725  6925 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.448344  6745 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.449074  6745 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.450196  6925 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.450774  6925 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.451258  6745 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.451339  6925 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.459224  6948 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:24:38.460157  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.945s	user 0.036s	sys 0.018s
I20260709 12:24:38.459314  6947 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:24:38.460856  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.947s	user 0.037s	sys 0.018s
I20260709 12:24:38.461227  6947 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.463490  6947 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.463579  6944 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.463786  6953 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.464042  6947 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.464646  6953 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.464121  6944 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.465682  6952 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.466303  6952 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.460183  6948 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.469002  6951 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.469638  6951 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.471028  6948 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.471753  6951 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.473726  6501 raft_consensus.cc:359] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.468397  6952 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:38.476838  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 123791us
I20260709 12:24:38.474148  6953 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.483096  6942 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.474427  6501 raft_consensus.cc:385] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:38.485826  6501 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:38.474531  6925 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.493111  6745 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.493095  6501 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.499490  6928 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.499948  6944 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.494251  6745 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.496460  6942 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.500419  6928 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.464670  6947 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.502298  6745 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.503779  6925 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.513008  6956 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.513808  6956 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.513685  6925 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.518469  6956 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.519050  6931 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.516868  6950 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.512956  6944 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.520359  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 335) took 2009 ms. Trace:
I20260709 12:24:38.521528  6931 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.523682  6952 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.523125  6745 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.525095  6925 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.525892  6949 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.619205  6958 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:24:38.609313  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54346.4us
I20260709 12:24:38.566105  6947 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.574332  6501 ts_tablet_manager.cc:1434] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 1.919s	user 0.019s	sys 0.006s
W20260709 12:24:38.613685  6384 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 2.085s	user 0.014s	sys 0.035s
I20260709 12:24:38.638656  6952 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:38.642213  6237 debug-util.cc:398] Leaking SignalData structure 0x7b080003eaa0 after lost signal to thread 6240
I20260709 12:24:38.628036  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:38.527237  6944 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.642869  6897 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:38.526784  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:38.643436  6959 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.601869  6928 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:38.644963  6501 ts_tablet_manager.cc:1467] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet startup took 1989ms
I20260709 12:24:38.564662  6942 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.525630  6456 rpcz_store.cc:276] 0709 12:24:36.512592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.515573 (+  2981us) service_pool.cc:224] Handling call
0709 12:24:38.518687 (+2003114us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:24:38.519078 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.519565 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.519623 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:38.519713 (+    90us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:38.519774 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.519876 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:38.520073 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27322112}
I20260709 12:24:38.647009  6847 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.650306  6745 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.654877  6897 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.655166  6847 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.526988  6950 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.651397  6959 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.650763  6960 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.657320  6924 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.647563  6949 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.704957  6950 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.673658  6928 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.699606  6924 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.657936  6962 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:24:38.652972  6501 ts_tablet_manager.cc:1469] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Trace:
0709 12:24:28.048124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.048245 (+   121us) service_pool.cc:224] Handling call
0709 12:24:28.050081 (+  1836us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:24:28.050244 (+   163us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:24:28.072148 (+ 21904us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:28.168535 (+ 96387us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:28.168983 (+   448us) inbound_call.cc:177] Queueing success response
0709 12:24:36.118940 (+7949957us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:36.655546 (+536606us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:38.471447 (+1815901us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:38.472031 (+   584us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":48310,"raft.queue_time_us":28506,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":35044,"spinlock_wait_cycles":28544,"tablet-open.queue_time_us":7950021}
I20260709 12:24:38.708276  6949 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.712757  6962 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:38.714304  6385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.142s	user 0.013s	sys 0.039s
W20260709 12:24:38.714762  6385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.142s	user 0.014s	sys 0.039s
I20260709 12:24:38.714021  6745 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.715276  6955 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.708266  6928 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.714082  6944 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.699688  6960 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.715493  6952 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.649005  6958 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.707744  6307 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:38.708276  6897 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:38.745849  6931 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.753943  6967 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.746542  6942 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.759770  6964 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.760066  6501 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:38.762434  6745 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.763518  6968 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.765450  6963 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:38.760583  6964 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.758769  6944 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.766080  6963 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.044672  6963 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.761394  6897 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.763900  6962 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:38.784387  6925 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.766435  6206 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:38.767704  6931 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.763967  6745 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:38.766618  6942 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.069620  6896 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:38.752969  6955 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.232928  6971 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:39.233860  6971 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.236537  6971 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.126223  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 296) took 2900 ms. Trace:
I20260709 12:24:39.242200  6970 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:24:39.247843  6491 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.IsCreateTableDone -> {remote=127.5.4.126:35813, user_credentials={real_user=slave}} blocked reactor thread for 59944.8us
I20260709 12:24:38.786654  6967 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.216166  6963 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.215329  6959 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.152885  6966 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.104795  6897 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.240031  6847 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.104660  6944 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.126444  6931 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.262368  6381 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:38.764187  6968 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.245041  6974 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:39.237433  6925 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.280169  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 307) took 2791 ms. Trace:
W20260709 12:24:39.283110  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.010s	sys 0.003s
I20260709 12:24:39.257237  6897 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.254817  6973 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:24:39.283560  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.010s	sys 0.003s
I20260709 12:24:39.244709  6896 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.269415  6959 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:39.324430  6893 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.331017  6962 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.280535  6384 rpcz_store.cc:276] 0709 12:24:36.488659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.490194 (+  1535us) service_pool.cc:224] Handling call
0709 12:24:36.498684 (+  8490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.647843 (+2149159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:38.647877 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:38.650010 (+  2133us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.746849 (+ 96839us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:39.143344 (+396495us) log.cc:844] Serialized 14 byte log entry
0709 12:24:39.279785 (+136441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.279881 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.279926 (+    45us) raft_consensus.cc:1685] finished
0709 12:24:39.279941 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.280102 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96378,"prepare.queue_time_us":11273,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":98517,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":2727,"spinlock_wait_cycles":3840,"thread_start_us":473355,"threads_started":3,"wal-append.queue_time_us":95016}
I20260709 12:24:39.277923  6847 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:39.258391  6963 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.260593  6970 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.331310  6893 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.287252  6975 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.287729  6924 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.331310  6959 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.260895  6966 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.282720  6974 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.359817  6930 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.360632  6970 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.369561  6979 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.331729  6962 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.373799  6965 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.362599  6969 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.373396  6930 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.353296  6971 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.377728  6963 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.378176  6977 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.378398  6893 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.396684  6745 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.378692  6924 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.364820  6958 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.345009  6847 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.352035  6960 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.396495  6974 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.348011  6942 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.288262  6973 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.397198  6971 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.395284  6979 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.401676  6959 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.396498  6962 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.394747  6965 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.401335  6977 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.420773  6924 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.244346  6385 rpcz_store.cc:276] 0709 12:24:36.218338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.218531 (+   193us) service_pool.cc:224] Handling call
0709 12:24:36.219076 (+   545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.510582 (+291506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.511565 (+   983us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.572320 (+ 60755us) log.cc:844] Serialized 26 byte log entry
0709 12:24:38.715121 (+2142801us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:38.715462 (+   341us) log.cc:844] Serialized 14 byte log entry
0709 12:24:39.015190 (+299728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.067385 (+ 52195us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:39.071316 (+  3931us) raft_consensus.cc:1685] finished
0709 12:24:39.074235 (+  2919us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.118265 (+ 44030us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192591,"prepare.queue_time_us":1374,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":961,"raft.queue_time_us":274914,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":12215,"thread_start_us":2344848,"threads_started":3,"wal-append.queue_time_us":1677032}
I20260709 12:24:39.397143  6893 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.395131  6964 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.410502  6963 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:39.459640  6977 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.395704  6975 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.460911  6924 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.469082  6847 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.462057  6931 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.469962  6501 tablet_bootstrap.cc:654] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.470829  6944 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.471623  6925 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.472555  6966 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.466363  6893 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.485818  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.486143  6979 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.487795  6893 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.477746  6967 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.490229  6975 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.476198  6963 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.480805  6925 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.403287  6969 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.488400  6971 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:39.494936  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.962s	user 0.039s	sys 0.079s
W20260709 12:24:39.497466  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.997s	user 0.039s	sys 0.080s
I20260709 12:24:39.501024  6928 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.504171  6982 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.506913  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.503449  6979 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.511005  6982 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.504421  6893 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.515084  6971 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.515089  6893 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.518285  6931 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:39.518826  6931 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.519277  6847 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.516250  6971 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.519881  6847 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.509246  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.521349  6962 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.521765  6376 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.530519  6973 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.512857  6979 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.527875  6745 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.534873  6745 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.537418  6745 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.521937  6962 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.546205  6955 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.547705  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.548786  6453 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:39.513226  6967 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:39.504595  6925 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.563203  6896 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.609163  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.661512  6454 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.644496  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.657531  6930 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.645107  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.596673  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:39.587106  6350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.055s	user 0.031s	sys 0.037s
W20260709 12:24:39.670329  6350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.224s	user 0.034s	sys 0.039s
I20260709 12:24:39.671249  6973 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.671510  6928 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.671865  6973 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.672079  6928 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.607331  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.586879  6983 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.685976  6928 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.687935  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.688655  6452 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.690236  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.691637  6967 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.692431  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.693053  6452 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:39.694655  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.695238  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.696301  6384 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.696764  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.697510  6376 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.700517  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.702112  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.523792  6975 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.702804  6381 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.703008  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.703096  6975 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.704288  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.701293  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.704909  6381 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.701203  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.563493  6377 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.600739  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.708500  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.683202  6983 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:39.713286  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf6a0 after lost signal to thread 6334
I20260709 12:24:39.714982  6896 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.715716  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.715572  6896 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.716368  6456 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.717715  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.719590  6928 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.720043  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.720177  6928 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.720757  6380 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.721921  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.722450  6928 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.722675  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:39.722994  6928 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:24:39.713676  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08002cf6c0 after lost signal to thread 6335
I20260709 12:24:39.723563  6928 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:39.724481  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08002cfee0 after lost signal to thread 6409
W20260709 12:24:39.724804  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08002cff00 after lost signal to thread 6410
I20260709 12:24:39.725544  6928 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:39.725915  6237 debug-util.cc:398] Leaking SignalData structure 0x7b080020dba0 after lost signal to thread 6483
W20260709 12:24:39.726305  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800213040 after lost signal to thread 6499
I20260709 12:24:39.726691  6966 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:39.744120  6966 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
W20260709 12:24:39.744148  6237 debug-util.cc:398] Leaking SignalData structure 0x7b080019dc00 after lost signal to thread 6840
I20260709 12:24:39.750689  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.751763  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.752868  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.754593  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.755338  6458 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.755848  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.756319  6931 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.757475  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.757700  6979 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.758903  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.730307  6973 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.733518  6963 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.736744  6968 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.759824  6458 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:39.737658  6501 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:39.737938  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.761876  6501 ts_tablet_manager.cc:1403] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 1.002s	user 0.015s	sys 0.006s
I20260709 12:24:39.738197  6847 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.762009  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.739521  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.739825  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.763634  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.764289  6376 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:39.741142  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.741124  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.765198  6452 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.742561  6896 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.765583  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.742813  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.743286  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.746963  6965 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.746292  6966 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.766264  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.743541  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.766978  6982 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.767169  6459 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.767298  6305 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.747895  6971 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.768113  6969 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.748137  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.768989  6975 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.748448  6893 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.737584  6962 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.749184  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 308) took 3260 ms. Trace:
I20260709 12:24:39.749325  6983 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.758994  6967 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.733074  6985 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.769862  6377 rpcz_store.cc:276] 0709 12:24:36.489039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.491964 (+  2925us) service_pool.cc:224] Handling call
0709 12:24:39.547959 (+3055995us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:24:39.548194 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.748887 (+200693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.749120 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3776384}
I20260709 12:24:39.742082  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.771590  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.770536  6985 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.772815  6206 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 115) took 3614 ms. Trace:
I20260709 12:24:39.729429  6930 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.773702  6930 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.774418  6985 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.776347  6925 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.778705  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.779255  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.779500  6501 raft_consensus.cc:359] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.782955  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.783154  6501 raft_consensus.cc:385] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.783792  6501 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.783906  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.784673  6501 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.782968  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.786597  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.787695  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.787968  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.788322  6385 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:39.788245  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.776260  6930 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.789640  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.790128  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.789062  6501 ts_tablet_manager.cc:1434] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.004s
I20260709 12:24:39.790323  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.791590  6501 tablet_bootstrap.cc:492] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:39.791628  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.792335  6377 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.794009  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.794893  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.795604  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.798167  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.796345  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.798561  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.797873  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.798220  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.800264  6501 tablet_bootstrap.cc:654] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:39.800130  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.800132  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:39.801362  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.802521  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.803049  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.803668  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.803788  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.805979  6310 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.819080  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.819169  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.820451  6925 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.821120  6925 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.822011  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.823609  6925 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:39.823841  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.825567  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.826002  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.826242  6309 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.827411  6376 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:39.773268  6206 rpcz_store.cc:276] 0709 12:24:36.159285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.170926 (+ 11641us) service_pool.cc:224] Handling call
0709 12:24:39.772736 (+3601810us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:39.351352 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:39.392867 (+ 41515us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:39.392921 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:24:39.392957 (+    36us) tablet.cc:662] Decoding operations
0709 12:24:39.393430 (+   473us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:39.394419 (+   989us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:39.394457 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:39.396171 (+  1714us) tablet.cc:701] Row locks acquired
0709 12:24:39.396208 (+    37us) write_op.cc:260] PREPARE: finished
0709 12:24:39.396393 (+   185us) write_op.cc:270] Start()
0709 12:24:39.396553 (+   160us) write_op.cc:276] Timestamp: P: 1783599879396335 usec, L: 0
0709 12:24:39.396595 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:24:39.463886 (+ 67291us) log.cc:844] Serialized 531 byte log entry
0709 12:24:39.754549 (+290663us) op_driver.cc:464] REPLICATION: finished
0709 12:24:39.759505 (+  4956us) write_op.cc:301] APPLY: starting
0709 12:24:39.759592 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:39.759701 (+   109us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:39.759836 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:39.760151 (+   315us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:39.760397 (+   246us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:39.760446 (+    49us) write_op.cc:312] APPLY: finished
0709 12:24:39.760708 (+   262us) log.cc:844] Serialized 22 byte log entry
0709 12:24:39.760769 (+    61us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b180026dac0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa3224190 00007f6fa321dae7 00007f6fa321cdd9
0709 12:24:39.760881 (+   112us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:39.760975 (+    94us) write_op.cc:454] Released schema lock
0709 12:24:39.761019 (+    44us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":4670,"apply.run_cpu_time_us":1826,"apply.run_wall_time_us":1831,"num_ops":1,"prepare.queue_time_us":102580,"prepare.run_cpu_time_us":7642,"prepare.run_wall_time_us":138196,"raft.queue_time_us":73211,"replication_time_us":330773,"spinlock_wait_cycles":8277376,"thread_start_us":105557,"threads_started":4,"wal-append.queue_time_us":28546}]]}
I20260709 12:24:39.845860  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.846061  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.846889  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.847859  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.848659  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.849257  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.849962  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.852059  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:39.853904  6274 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:39.855986  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.857403  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.857784  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.859318  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.860436  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.861581  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.863564  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.864661  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.864832  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.864871  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.864173  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.863564  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.867625  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.875560  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.881091  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.882362  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.882943  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.883453  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.883684  6457 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:39.883515  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.882270  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.885615  6381 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.891400  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.894639  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.895630  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.896458  6457 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 0.
I20260709 12:24:39.898718  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.899252  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.903129  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.903456  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.904999  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:39.906500  6274 leader_election.cc:343] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.906569  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:39.907958  6274 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:39.908347  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.910050  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.910183  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.910338  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.910830  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.911042  6385 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.911137  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.912091  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.912354  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.914041  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.914729  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.915417  6376 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.918773  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.919003  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.918370  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.919430  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.918994  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.921458  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.921978  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.922605  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.924680  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.924772  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.922751  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.927175  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.928077  6377 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:39.930709  6501 tablet_bootstrap.cc:492] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:39.931267  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.931516  6501 ts_tablet_manager.cc:1403] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.140s	user 0.052s	sys 0.040s
I20260709 12:24:39.932890  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:39.933776  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.936105  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.936746  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.936976  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:39.937342  6424 consensus_peers.cc:597] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:39.937453  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.937302  6455 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.938378  6458 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.939741  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.940987  6953 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:39.941493  6953 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.941856  6459 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.942689  6983 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:39.943687  6953 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:39.947808  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:39.949326  6274 leader_election.cc:343] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.950886  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.951495  6455 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.951841  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:39.946811  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.953855  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.954998  6953 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:39.955057  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.956143  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:39.955547  6953 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:39.953876  6459 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.959774  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.960069  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.950587  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.960872  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.959815  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.961843  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:39.955853  6454 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:39.963435  6424 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:39.968537  6425 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:39.969081  6424 leader_election.cc:343] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:39.970122  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:39.970767  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:39.962783  6953 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:39.969833  6971 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:39.972970  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:39.973925  6962 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:39.974653  6985 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:39.975052  6985 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:39.975227  6975 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:39.975411  6985 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:39.975713  6975 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:39.979547  6274 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:39.980484  6348 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:39.981835  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:39.983392  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:39.983193  6501 raft_consensus.cc:359] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.983855  6501 raft_consensus.cc:385] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:39.984166  6501 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:39.986903  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:39.987528  6501 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.996306  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:39.977978  6975 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.001350  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:40.002714  6983 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.003201  6983 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.003556  6983 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.002816  6966 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:39.999910  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:40.005034  6424 leader_election.cc:343] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.010838  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:40.011644  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:40.012634  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:40.014542  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:40.006470  6971 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.017518  6971 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:40.018013  6971 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.018205  6953 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:39.997578  6962 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.020785  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.021929  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.022248  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.009656  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.023187  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.022464  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.024866  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.025751  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.006734  6966 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.026219  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.026350  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.026517  6966 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.029968  6424 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:40.030990  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50616.9us
I20260709 12:24:40.032189  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.024142  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.031792  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.052188  6348 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.058044  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.061388  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.033656  6982 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.068825  6348 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.034657  6962 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.065075  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.034192  6953 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.058398  6949 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.063184  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.058310  6973 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:40.058926  6893 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.072423  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:40.072795  6896 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.093382  6348 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.098978  6993 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:40.100140  6992 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.067826  6925 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.068593  6930 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:40.110522  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45513.8us
I20260709 12:24:40.116572  6424 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.113072  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.130849  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.133965  6965 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:40.134300  6962 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.117632  6930 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.129446  6925 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.134148  6203 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:40.136592  6996 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:40.143255  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50021.7us
W20260709 12:24:40.132915  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69852.4us
I20260709 12:24:40.101895  6949 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.109174  6973 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.112283  6924 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.115513  6896 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.115743  6993 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.147708  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.189057  6973 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:40.193048  6985 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.188807  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.196187  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.189121  6896 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.145704  6965 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.204034  6971 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.204702  6965 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.206147  6971 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.206570  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.207551  6983 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.115239  6992 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.208459  6992 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.203239  6925 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:40.147806  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81656.3us
I20260709 12:24:40.209591  6925 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.145280  6928 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.209960  6983 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.210090  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.210356  6928 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.210860  6928 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.211050  6953 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.197332  6924 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.203727  6962 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.189235  6993 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:40.148382  6930 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.147226  6847 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:40.210984  6966 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.189167  6949 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.225569  6501 ts_tablet_manager.cc:1434] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.293s	user 0.053s	sys 0.054s
I20260709 12:24:40.211553  6953 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.227545  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.228350  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.210033  6925 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:24:40.195967  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48365.1us
I20260709 12:24:40.231873  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.232577  6348 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.234519  6424 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.235698  6274 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.235733  6925 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.207820  6973 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.227531  6996 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.230894  6995 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.236234  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.228965  6962 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:40.235932  6997 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.243604  7001 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:40.231508  6953 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.246253  6424 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.227450  6975 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.248629  6424 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.247802  6847 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:40.265653  6994 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.265950  6983 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.263906  7001 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.266254  6994 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.266707  6994 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.238973  6924 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.267328  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.267817  6501 tablet_bootstrap.cc:492] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:40.247020  6962 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.242241  6985 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.269600  7001 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.246997  6998 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:24:40.275629  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43833.2us
I20260709 12:24:40.276221  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.247100  6997 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.276424  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.245292  6996 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.245234  6995 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.278615  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.279757  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.248425  6971 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:40.281899  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.280835  6996 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.284737  6424 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.277828  6982 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.282611  6924 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.278790  6997 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.286995  6847 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.288342  6983 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.281471  6985 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.281728  6999 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.280619  6893 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.281533  6998 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.280658  6995 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.294689  7003 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.287199  6982 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.294093  6985 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.298967  6966 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.302014  6982 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.326177  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:40.328594  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 96084.2us
I20260709 12:24:40.309696  6999 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.329761  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.329413  6424 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.319494  6985 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.331342  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.332335  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:40.332291  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49173.5us
I20260709 12:24:40.348712  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.350165  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.350183  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.354069  6348 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.332540  6983 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.339975  6973 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.355374  6893 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.355786  6893 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.355906  6999 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.356200  6893 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.357515  6983 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.338538  7005 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.358016  7005 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.358381  7005 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.358579  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.328523  6969 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.359786  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.361806  6424 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.356316  6999 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.364734  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.356498  6847 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.368422  6424 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.368557  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.370394  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:40.374866  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 139226us
I20260709 12:24:40.375039  6982 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.377254  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.377525  6982 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.378018  6947 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.378343  6947 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:40.378705  7004 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.378754  6947 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.360426  6924 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.384215  6424 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.363339  6975 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.392343  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:40.386983  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.399467  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:40.400599  6424 leader_election.cc:343] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.401052  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.366786  6999 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.368294  6973 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.386157  7007 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.364355  7010 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.415397  7007 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:40.415473  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60172.1us
I20260709 12:24:40.416170  7007 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.398880  6982 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.395877  6971 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.385200  6983 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.416548  7007 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.403152  6969 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.404163  6973 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.383179  7006 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.417083  6924 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.404223  6967 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.421561  6983 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.392112  7012 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:24:40.431896  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63466.4us
I20260709 12:24:40.435035  6274 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:40.437027  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.428649  6973 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.439375  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.441246  6351 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.436551  6896 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.430402  7007 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.451438  6896 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.452065  6896 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.435683  6983 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.437510  6998 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.456024  6966 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.454355  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.462462  7011 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.440546  7010 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.470038  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.472885  6953 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.477432  6896 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.438130  7012 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:24:40.492969  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57827.5us
I20260709 12:24:40.471441  6966 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:24:40.498412  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97844.5us
W20260709 12:24:40.499871  6424 leader_election.cc:343] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:40.501194  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 108914us
I20260709 12:24:40.501075  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.502290  6348 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.480502  7011 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.508983  6501 tablet_bootstrap.cc:654] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:40.513907  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.487903  7012 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.522517  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.524796  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.525713  6992 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.526065  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.527989  6992 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.531945  6953 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.516108  7003 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.536980  6997 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.515148  7011 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.516355  6924 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.542158  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:40.543576  6424 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:40.544189  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64890.4us
I20260709 12:24:40.545593  6847 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.545118  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:40.552556  6424 leader_election.cc:343] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:24:40.553092  6424 leader_election.cc:343] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.552814  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.521296  6928 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.562152  7018 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.562340  6999 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.563110  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.563665  6985 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.564414  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.565513  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.565688  6985 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.562990  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.567174  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:40.568053  6377 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.568383  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.570086  6384 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.564599  6999 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.551683  6930 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.576218  6930 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.577461  6928 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.547267  6847 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.577812  6997 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.559595  6966 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:40.590696  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 174183us
I20260709 12:24:40.591570  6381 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.581234  6847 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.598963  6272 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:40.569514  6949 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.606165  7008 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:40.606616  7008 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.606961  7008 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.598541  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:40.598573  6379 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:40.617295  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 103143us
I20260709 12:24:40.617980  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:40.621624  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:40.609061  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.627292  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.628111  6456 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:40.628577  6994 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.629197  6925 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.629674  6378 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.631029  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.625546  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.631347  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.635092  7001 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:24:40.635500  7001 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.635845  7001 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.602361  7004 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:24:40.648787  6994 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:40.665786  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45953.5us
I20260709 12:24:40.609545  6962 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.631397  6993 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.660802  6893 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.684386  6993 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.684751  6893 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.686196  6454 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:40.686947  6925 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.689206  6453 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.662750  6385 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.687819  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.664453  7017 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.693526  7004 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.643298  6947 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.681615  6995 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.696051  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:40.696802  6383 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 0.
I20260709 12:24:40.695240  6376 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:40.698033  6996 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.698350  7000 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.700068  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.665140  6965 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.686235  6962 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.697649  6995 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.702236  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.802852  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.804550  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.803129  6384 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.816601  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.816004  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.817900  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.818373  6378 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.808893  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.818699  6994 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:40.800882  6983 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.819415  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.819347  6995 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.822706  7020 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:40.821391  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:40.839670  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 145263us
I20260709 12:24:40.821331  6949 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.811164  6973 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.825656  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.849032  6384 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:40.813544  7001 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.836790  6383 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:40.812088  7007 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.839449  7017 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.845053  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.852632  6983 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.813736  7005 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.816861  7012 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.858537  7004 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:40.864470  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.834316  7008 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.869242  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.869493  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.872812  6379 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.873450  6953 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:40.873821  6953 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:40.874742  6953 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:40.876223  6953 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.878684  6380 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.880857  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.881523  6973 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.881712  6379 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:40.859189  6965 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:40.859027  6947 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.883479  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.885128  6382 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.887684  6377 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:40.888762  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.889477  6847 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:40.887779  7022 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:24:40.891233  6996 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.901351  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.902596  6382 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:40.916905  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.919322  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.928932  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.929772  6457 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:40.939152  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.941149  6458 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.941803  6973 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.942970  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:40.946367  6348 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.950511  6384 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:40.948786  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:40.952593  6424 leader_election.cc:343] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:40.959990  6348 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.961405  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.962810  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.963456  7008 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:40.964212  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:40.958377  6457 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:40.979178  6348 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.956126  6962 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:40.982187  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:24:40.984771  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 143081us
I20260709 12:24:40.985491  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.969192  6993 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.981600  6995 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.992827  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:40.988562  6348 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.989635  7024 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:40.994853  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:40.982633  6962 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:40.986994  6996 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.998314  7010 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:40.998581  6930 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:41.003865  6847 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:41.004029  7012 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.004914  7000 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:40.985920  7017 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.006340  7001 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.007339  7007 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.012591  6272 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.010061  6962 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.015799  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.005292  7000 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.014994  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.016238  6953 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.019534  6348 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.020718  6272 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.021718  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.018632  7007 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:41.026567  7001 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.026906  6996 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.018110  7024 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.026093  7000 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.029182  7000 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.030910  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.033733  7005 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.028270  6501 tablet_bootstrap.cc:492] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
W20260709 12:24:41.035156  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 182555us
I20260709 12:24:41.035423  6501 ts_tablet_manager.cc:1403] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.793s	user 0.044s	sys 0.046s
I20260709 12:24:41.037089  6949 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.038179  6953 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.034535  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.049086  6995 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:41.049335  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:41.049680  6995 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:41.039824  7007 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:40.986670  7011 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.052678  6949 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.050941  7007 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:41.054046  6949 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.054824  6949 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.051095  7020 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:41.050750  6995 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.055886  6949 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.033867  6962 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.050025  6953 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.058538  6953 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.059687  6996 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.059131  6962 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:41.057404  6949 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.054296  7007 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.060051  6953 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.060070  6996 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:41.061738  6996 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.063015  7025 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.035156  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.064146  7024 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.064462  7000 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.064532  7024 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.065100  7007 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.065632  7024 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.066938  7024 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.067709  6272 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.070847  7025 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.071965  7025 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.072332  7025 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.072587  7011 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.073199  7025 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.074164  7025 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.076956  7031 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.079309  6996 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.080560  6272 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.087745  7001 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.092964  7008 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.093159  6962 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.094079  7008 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:41.090559  6453 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.093951  7000 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.094288  6967 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.094548  7008 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.094774  6967 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.095222  7024 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.095657  7024 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.095494  7008 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.096994  7008 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:41.096578  7024 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.099218  7001 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.099624  7001 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.100418  7001 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.101632  7001 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.095528  6967 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.103899  6967 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:41.110126  6424 leader_election.cc:343] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.119761  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.122567  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.123199  6453 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:41.125959  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45457.2us
I20260709 12:24:41.127576  6994 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:41.126588  7031 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.127941  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.129259  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.130378  6962 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.087962  6501 raft_consensus.cc:359] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.132292  6501 raft_consensus.cc:385] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.132712  6501 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.135499  6501 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.136492  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:41.136833  6975 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.136716  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:41.137776  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:41.136700  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.138019  6455 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.138792  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.139322  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:41.139567  6458 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:41.141289  6451 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.149204  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.130542  7024 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.150137  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.150439  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.150892  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.150199  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:41.152405  6458 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.153651  6459 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.153856  6457 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.157120  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.157853  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.159453  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.160254  6451 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:41.159631  6206 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.160640  6458 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:24:41.162436  6303 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:24:41.162678  6301 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:41.163331  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75510.2us
I20260709 12:24:41.164531  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.165354  6451 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:41.165069  6949 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.166366  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:41.164868  7008 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:41.168143  6301 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.170454  6949 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.170885  6949 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.171593  6949 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.172917  6949 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.175774  7008 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.176151  7008 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.176973  7008 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.177017  7024 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.178119  7008 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.178788  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:41.180666  7031 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:41.181085  7031 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.181324  6501 ts_tablet_manager.cc:1434] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.146s	user 0.004s	sys 0.003s
I20260709 12:24:41.181931  7031 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.183269  7031 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:41.189961  6501 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
W20260709 12:24:41.196826  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.198861  6383 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.218953  6453 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:41.219908  6452 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:41.220350  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 84728us
I20260709 12:24:41.220461  6995 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.221710  6995 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.222128  6995 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:41.222254  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.223219  7007 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.223085  6995 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.224509  6995 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.226826  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.227205  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:41.228087  6383 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.228444  7007 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.228001  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.229912  7007 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:41.230446  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:41.231007  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.230506  7007 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:41.233012  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:41.233632  7007 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.237635  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:41.245594  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:24:41.254652  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.269395  6303 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.271683  7024 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.272890  7024 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:41.272122  6274 leader_election.cc:343] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.273860  7024 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:41.274016  6274 leader_election.cc:343] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:24:41.275580  7000 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.275975  7000 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.276939  7000 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.278187  7000 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.274245  7024 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.283216  7024 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.285003  7024 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.286276  6383 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.287514  6303 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.290316  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.291493  6303 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:41.297338  6309 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.299547  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.301149  6309 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.303483  6309 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.281467  6501 tablet_bootstrap.cc:654] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.311436  6301 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:41.329496  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.331836  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.333685  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.334751  6308 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:41.338137  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.355082  6501 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:41.355721  6501 ts_tablet_manager.cc:1403] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.166s	user 0.013s	sys 0.000s
I20260709 12:24:41.357990  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.358532  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.359393  6847 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:41.360175  6847 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.360487  6847 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.361268  6847 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.365355  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:41.367368  6962 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.368389  6962 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.368744  6962 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.368665  6967 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.369725  6967 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.370101  6967 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.373221  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.373929  6971 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.375061  6975 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.376155  6924 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.376507  6924 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.376669  6272 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.377658  7008 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.377413  6924 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.378331  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.379256  7008 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.379639  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.380456  7008 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.381557  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:41.381824  6452 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.375257  6971 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.383780  6971 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.384171  6971 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.371423  6501 raft_consensus.cc:359] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.384727  6501 raft_consensus.cc:385] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.385047  6501 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.385106  6971 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.386047  6309 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:41.387107  6501 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.387689  6967 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.389175  6967 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.389748  6501 ts_tablet_manager.cc:1434] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:24:41.390770  7007 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.390539  6205 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:41.391232  7007 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.393553  7007 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.394279  6501 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:41.396723  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:41.399482  6272 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.399607  6924 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.400328  7008 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.401053  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:41.402734  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.369637  6962 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.404254  6971 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.403342  6847 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.376430  6425 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:24:41.411563  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.405014  6962 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.411908  6975 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.417726  6995 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.420004  6925 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.420531  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:41.420966  6965 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.421306  6925 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.421787  6925 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.421994  6965 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:41.422550  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.423182  6965 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.424170  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.420189  6983 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.424511  7018 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.425854  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.425962  7018 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.422712  6893 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.427090  6893 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.428298  6893 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.429975  6893 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.424212  6965 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.435354  6272 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:41.441028  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.434464  6925 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.441933  6925 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.449216  6272 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.451146  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.450688  7008 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.451964  7000 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.451406  7024 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.453150  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.453845  7024 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.437077  6847 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.477468  6501 tablet_bootstrap.cc:654] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.481638  6992 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.483028  6992 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.483393  6992 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.484191  6992 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.481874  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:41.485450  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.485687  7008 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.486330  6992 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.487140  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.489902  7024 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.499892  6272 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:41.502249  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.502351  7004 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.511631  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.512357  7018 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.513136  7018 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.509438  7008 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:24:41.516537  7024 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.520094  6272 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.520658  7024 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.514170  7018 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.521718  6949 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.531852  6205 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.535830  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.537448  6425 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.539790  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:41.541399  6983 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.539098  7045 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.542661  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:41.545575  6847 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:41.546025  6847 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.547271  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:41.530849  7024 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.552640  6847 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.553786  6452 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.540184  7031 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:24:41.559459  6949 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:41.559828  6949 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:41.548659  6965 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:41.560645  6949 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.546896  7018 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.562315  6995 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.564893  6383 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.550263  6983 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:41.575366  6983 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:41.576503  6983 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.585449  6949 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.654556  6459 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.658341  6996 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.661257  6308 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.664093  7020 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:41.648645  6309 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:41.672407  7024 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.673580  7024 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:41.676502  6501 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:41.676950  6501 ts_tablet_manager.cc:1403] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.286s	user 0.016s	sys 0.024s
I20260709 12:24:41.708362  6501 raft_consensus.cc:359] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.710865  6501 raft_consensus.cc:385] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.711215  6501 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.713603  6501 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.714898  6995 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.715469  6995 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.717690  6995 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:41.718767  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:41.721652  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:41.723580  6501 ts_tablet_manager.cc:1434] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.046s	user 0.001s	sys 0.005s
I20260709 12:24:41.724931  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:41.727550  6996 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:41.725746  6501 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:41.730299  6301 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.736270  6995 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:24:41.759123  6501 tablet_bootstrap.cc:654] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.780117  6204 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.785030  6204 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:41.804339  6379 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.820567  6377 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:41.840327  6501 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:41.840950  6501 ts_tablet_manager.cc:1403] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.116s	user 0.022s	sys 0.030s
I20260709 12:24:41.843761  6501 raft_consensus.cc:359] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.844478  6501 raft_consensus.cc:385] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.849128  6383 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.851948  6983 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.859606  6501 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.860404  6501 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.868005  6501 ts_tablet_manager.cc:1434] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.027s	user 0.003s	sys 0.003s
I20260709 12:24:41.868916  6501 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:41.879843  6303 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.886682  6847 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.888402  6501 tablet_bootstrap.cc:654] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.910051  6501 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:41.910588  6501 ts_tablet_manager.cc:1403] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.013s
I20260709 12:24:41.913038  6501 raft_consensus.cc:359] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.913735  6501 raft_consensus.cc:385] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.914068  6501 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.914780  6501 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.917160  6501 ts_tablet_manager.cc:1434] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:24:41.918120  6501 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Bootstrap starting.
I20260709 12:24:41.924316  6501 tablet_bootstrap.cc:654] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Neither blocks nor log segments found. Creating new log.
I20260709 12:24:41.933974  6501 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: No bootstrap required, opened a new log
I20260709 12:24:41.936159  6501 ts_tablet_manager.cc:1403] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.005s
I20260709 12:24:41.939174  6501 raft_consensus.cc:359] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.940127  6501 raft_consensus.cc:385] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:24:41.940519  6501 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Initialized, Role: FOLLOWER
I20260709 12:24:41.941309  6501 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:41.943887  6501 ts_tablet_manager.cc:1434] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:24:41.963038  6303 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.964583  7018 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:41.976001  6377 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:41.977561  6983 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.017007  6377 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.045075  6303 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.046667  6983 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.061225  6377 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.063148  6847 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.078938  6983 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.079499  6983 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.081611  6983 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.083601  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.084393  6383 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.085270  6995 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.085812  6995 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.086433  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.087080  6303 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.087971  6995 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.088548  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.091320  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.091761  6983 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.092149  6983 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.092541  6983 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.092252  6452 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.091390  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.099210  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.100430  6983 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.103229  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.107406  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.112131  6301 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.112658  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.113404  6383 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.114425  6996 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.114866  6996 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.115206  6996 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.117205  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.118542  6847 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.119073  6847 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:42.119951  6847 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.103360  6983 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.139169  6996 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.142802  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.143626  6452 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:24:42.145270  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:42.145710  6996 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.151901  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.161455  6301 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.162693  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.163466  6996 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.164028  6996 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.164991  6996 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.166555  6847 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.167168  6847 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.170701  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.171432  6301 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.171262  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.172340  6377 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.172081  6847 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.174525  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.175699  6983 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.176095  6983 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.176472  6983 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.180159  7007 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.180845  7007 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.168071  6205 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.183282  7007 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.184676  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.185588  6301 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.185461  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.186414  6452 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.187583  6348 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.188887  6995 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.199458  6983 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.202422  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.210537  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.211338  6377 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.212497  6983 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.215262  6301 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.216357  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.217322  6983 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.218410  7007 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.218992  7007 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.226111  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.227667  6458 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.229581  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.230379  7060 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.230518  7007 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.230806  7060 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.231180  7060 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.233606  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.219172  6983 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:42.236739  6983 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.239841  6995 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:42.243229  6204 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.245967  7060 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.248646  7060 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.249444  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.252698  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.253376  6452 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.253767  6992 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:42.254284  6992 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.256343  6992 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.259889  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.260566  6377 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.262017  6274 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:42.262456  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.263283  6458 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.263293  6992 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.264204  6992 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:42.264663  6992 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.264938  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:42.266260  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.262723  6206 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.268402  6996 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.268947  6996 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.269810  6996 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.282361  6992 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.286520  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.287398  6992 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.289276  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.290077  6452 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.295389  6206 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.303906  6377 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.305375  6274 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.306553  6992 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.307229  6992 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:42.308175  6992 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.325989  6206 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 2, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.327896  6996 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.328387  6996 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.331425  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.331562  6996 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.332334  6303 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.333617  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.334237  6452 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 1: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.335136  6348 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.335714  6996 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:24:42.345371  6996 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:24:42.369807  6996 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.370409  6996 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.372895  6996 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.373404  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.374193  6303 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.374125  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.375588  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:42.376312  6996 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.376642  6996 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.376925  6996 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.379632  6995 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.380127  6995 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.382282  6995 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.383505  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.384099  6458 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.384123  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.384999  6303 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.385820  6996 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.386584  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.387331  7060 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.387719  6995 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.388187  6995 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.387805  7060 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.388531  6996 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.389786  6995 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.390136  7060 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.390302  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.391268  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.391856  6303 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.392352  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.393136  6301 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.395721  7060 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.396378  7060 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.399075  7060 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.399966  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.400679  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.403997  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.409659  6301 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.407399  6458 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.409101  6303 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:42.411547  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.412119  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.412570  7060 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.412997  7060 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.413367  7060 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.413460  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.414353  7020 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.414778  7020 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.415136  7020 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.421237  7060 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.420600  6995 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.424053  7020 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.425113  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.426697  6996 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.427199  6995 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.428004  6996 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.429644  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.429004  6996 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.434639  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.434877  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.435647  6301 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.435863  6303 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.439234  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.439860  6303 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.440807  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.439932  7020 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.445988  7060 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.450819  6452 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.451221  6303 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:42.452024  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.452211  6206 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 3, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.452944  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.454097  7020 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.454929  6996 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.455407  6996 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.456233  6996 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.458662  7020 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.460570  7020 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.470304  6458 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.473487  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.474184  6995 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.474759  6995 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.475629  6995 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.478941  6983 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.479523  6983 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.483078  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.483855  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.484508  6983 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.484902  6992 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.485081  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.485769  6383 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.485421  6992 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.487573  6992 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.488543  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.489173  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.489136  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.490093  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:42.490352  6983 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.489204  6383 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.491003  6983 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.491479  6983 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.491917  6274 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:42.492657  7008 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.493017  7008 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.493345  7008 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.501048  6847 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.501614  6847 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.503768  6847 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.504724  6983 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.505463  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.506353  6377 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.506675  6983 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.507910  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.508490  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.508756  6377 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.509197  6303 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.510741  6425 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.511421  6983 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.511834  6983 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.511871  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.512413  6983 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.519207  6983 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.520962  7060 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.521587  7060 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.521998  6983 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.524145  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.524235  7008 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.526154  7008 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.527256  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.527271  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.527972  6458 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.528424  6309 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.531164  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.532382  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.532716  7025 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.533164  7025 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.535149  7025 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.536314  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.536501  6847 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.537153  7018 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.537569  7018 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.530433  7060 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.539531  7018 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.539855  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.540200  6847 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:42.540606  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.542577  6847 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.543154  7060 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.545475  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.546114  6385 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.548930  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.549934  6385 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.552440  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.543715  7060 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.561655  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.562903  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.548880  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.562301  6383 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.560945  6206 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.569442  6206 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 3, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.574151  6383 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:42.575670  6274 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:42.574673  7020 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.576567  7025 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.576615  7020 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.574939  7011 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.577430  7025 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:42.577389  7020 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.578471  7025 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.579838  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:42.577451  7011 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.583246  7011 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.586374  7060 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.586812  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.586702  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.587512  6309 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:42.587512  6303 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.589216  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:42.590144  7060 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:42.590320  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.590641  7060 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.591086  7060 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:42.591640  6983 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.592095  6983 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.591717  7011 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.592460  6983 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.592857  6301 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.575759  7008 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.594162  7008 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.588521  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.594696  6425 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.594797  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.595486  6377 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.596227  7018 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.596688  7008 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.597616  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.601917  6452 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.603631  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.607851  6383 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.609128  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.609776  6274 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.610649  7008 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.611016  7008 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.611357  7008 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.613526  7011 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.615554  6377 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:42.616662  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.617592  7025 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:42.618053  7025 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.620075  7008 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.620824  7025 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.621012  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.621364  7025 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.622414  6893 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.624356  6893 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.625439  6996 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.622603  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.626948  6379 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.627586  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.627882  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.628366  6452 raft_consensus.cc:2368] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:42.630827  6272 leader_election.cc:400] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:42.631217  6272 leader_election.cc:403] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:42.631814  7024 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.635649  6382 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:42.630342  6893 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.638134  7020 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.638885  6983 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.639631  6379 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:42.640630  7020 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.641345  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.648689  6308 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.644445  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.646059  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.644768  7025 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:42.649417  6309 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.651363  7025 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.652200  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.649632  6996 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.655189  6274 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:42.642419  7024 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:42.655660  6996 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.649976  6204 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.661029  6996 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.661273  7018 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:42.662245  7018 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.656257  7011 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.664368  7011 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:42.665144  7011 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.666534  6307 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.669677  6847 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.669859  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.671540  6308 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.672996  7007 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:42.650386  6893 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:24:42.675711  6893 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.675858  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.676268  6893 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.679298  6893 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.680203  7060 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.651130  6303 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.649003  6301 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.685278  7083 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.685880  7083 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.691283  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.697245  6303 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:42.693560  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.694746  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.694938  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.706430  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:42.696540  6309 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.622006  7008 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.682394  7060 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.686787  6983 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.704525  6206 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.715258  6307 raft_consensus.cc:2368] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:42.700171  7024 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:42.702955  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.743106  7018 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.743712  7018 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.711732  6304 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.752123  6456 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
W20260709 12:24:42.755446  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60595.3us
I20260709 12:24:42.758939  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.765381  6348 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.748284  7083 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.746616  6847 consensus_queue.cc:1243] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:42.771027  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.771895  6383 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:42.741575  7024 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.779147  6382 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.761351  7060 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.786578  7024 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.787083  7097 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.788956  6307 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:42.779623  6996 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.763813  7007 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.794190  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:42.799307  6456 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:42.800748  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.802295  6459 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.781100  6847 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.798883  7097 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.805799  7097 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.783612  7087 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.791114  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.814898  6425 leader_election.cc:400] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:42.791374  7011 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.820804  7011 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.822990  7011 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.818751  7087 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.825774  7087 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.798187  7060 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.803041  6996 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.802503  6995 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.828320  7060 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.828534  6995 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.828976  7011 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.829190  6996 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.829591  7011 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.802556  6983 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:42.831398  6983 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:42.833395  7018 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.834136  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.802667  7007 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.840216  6459 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.841620  7007 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.830204  6377 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.836988  7011 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.820217  7092 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:24:42.810235  6847 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.843828  6847 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.843951  6893 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:42.844427  6893 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:42.826275  7087 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.845530  7087 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.845296  6893 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.845916  7087 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.838905  6983 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.848074  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.842371  7007 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 12:24:42.849291  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.850286  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.852955  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:24:42.854004  6455 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:42.854692  7020 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.855268  7020 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.855901  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.856552  6452 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:42.857187  7020 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.857540  6847 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.860563  7083 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.860816  6377 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.861115  7083 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.819144  6425 leader_election.cc:403] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:42.862849  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51550.2us
I20260709 12:24:42.863533  6204 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.869489  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.866926  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:42.880378  6272 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.860159  6847 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.900022  7097 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.878531  6893 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.902184  7097 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.887279  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:42.891669  7096 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:42.903043  6309 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.891472  7024 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:42.903101  7096 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.905428  7096 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.906028  7091 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.906419  7091 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.906801  7091 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.906674  6206 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:42.909127  6206 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:42.898077  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97632.8us
I20260709 12:24:42.916828  6309 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:42.917832  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.921309  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.922504  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.922875  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.923246  6452 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:42.923413  6383 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:42.921893  6382 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.841651  6995 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.878684  7092 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.931375  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.932586  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.933662  7101 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.881649  6992 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:42.934616  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:42.934392  7101 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.936120  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.937031  7086 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.936968  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.937317  7101 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.937644  7086 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.937840  6308 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:42.938040  7086 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.939251  7060 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:42.939774  7060 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:42.940475  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:42.941027  6384 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.940650  7060 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.946610  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.947921  6893 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:42.948299  6893 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.948642  6893 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:42.950855  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:42.951910  6378 raft_consensus.cc:2368] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:42.952931  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.953111  7086 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.953949  6453 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:42.921605  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:42.977238  6384 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:24:42.978461  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56973.4us
I20260709 12:24:42.984249  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:42.985190  6309 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:42.981163  6893 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.987624  6893 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:42.958348  6992 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:42.989528  6992 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:42.996111  7091 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:42.998164  7091 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:42.999356  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:42.999784  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.000492  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.002055  7092 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.003170  7086 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:43.003507  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.004271  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.005954  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:24:43.009843  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61133.9us
I20260709 12:24:42.982946  7018 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:43.017599  6452 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:43.017805  6309 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.017130  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.019893  6424 leader_election.cc:400] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.026969  6424 leader_election.cc:403] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.020331  6995 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.020674  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.031318  6305 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.025220  6456 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.032311  6309 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:43.035838  7094 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.030028  6995 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.036337  7094 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.037873  6995 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.039677  7094 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.040263  6983 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.041229  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.041846  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:43.041877  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.044052  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.044973  7094 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:43.045372  7094 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.045717  7094 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.054452  6456 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:43.055967  6272 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.009001  7024 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:43.058957  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54754.4us
I20260709 12:24:43.066227  7105 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:43.066962  7102 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:43.066998  7105 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:43.067667  7102 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:43.068172  7102 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.068228  7105 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.070504  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.071285  6310 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:43.063002  7100 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.015265  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:24:43.073489  6986 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:43.077028  7018 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:43.018985  6450 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.078055  6378 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.078747  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.058106  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:43.083528  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.092311  6382 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.085568  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.097671  6376 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:43.088406  6893 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:43.104847  6893 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:43.105459  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.105685  6893 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.074414  7024 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.102027  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.108516  6456 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.109247  7024 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.109567  7018 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.111424  6983 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.112515  7087 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.069842  6992 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.087930  7100 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.115428  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.116024  6992 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.116709  7100 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.118564  6385 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.089574  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.120160  6384 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:43.123775  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.124478  6452 raft_consensus.cc:2368] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.125655  6307 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:43.126183  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.091708  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:43.126912  6310 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:43.114427  7087 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:43.106247  6303 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.129745  7102 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.130751  6383 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.131129  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.153497  6348 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.154791  6382 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:43.154961  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:43.155174  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.169322  7098 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:43.169735  7098 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:43.170094  7098 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.170418  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.170768  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.171271  6377 raft_consensus.cc:2368] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.171299  6456 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.173657  7083 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:43.175510  6272 leader_election.cc:400] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.175837  6272 leader_election.cc:403] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.143858  6302 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.177124  7098 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.177716  7025 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.178593  7100 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.179450  6383 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:43.180534  7101 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:43.181054  7101 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:43.184816  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.194913  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:43.202581  6459 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.131795  7102 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.204859  6452 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.184653  6992 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.212118  7098 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.211907  6992 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.200524  7024 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.213207  7024 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.213624  7101 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.214069  6992 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.215292  7024 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.203423  7097 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:43.215869  7097 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:43.216285  7097 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:43.217896  6457 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:43.201716  7100 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:43.229357  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.076355  7094 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:43.224123  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 195506us
I20260709 12:24:43.231033  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.230074  6459 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:43.235073  6452 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:43.232489  6456 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.234869  7102 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.224165  7100 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.291482  6456 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:24:43.306743  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70510.5us
I20260709 12:24:43.308772  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:43.310698  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 108321us
I20260709 12:24:43.232159  7008 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:24:43.232335  6992 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:24:43.302511  7100 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.311674  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.300459  7102 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.321914  7060 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:43.323807  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:43.316068  6893 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:43.326659  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.328719  6383 raft_consensus.cc:2368] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.331261  6424 leader_election.cc:400] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.331650  6424 leader_election.cc:403] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.334079  7086 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.336236  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.336638  6381 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:24:43.341784  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:43.341482  7097 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.324371  7060 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.347627  7096 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.351032  6383 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.351485  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.351696  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.352401  6377 raft_consensus.cc:2368] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.352671  6381 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:43.343327  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.325920  7020 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:24:43.334441  6893 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:43.354194  7060 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.365726  6378 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.377835  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.378767  6450 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.379938  7102 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.365922  7121 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:43.380919  7121 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:43.381306  7121 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:43.384342  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 342526us
I20260709 12:24:43.368659  6893 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.386687  6458 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.365121  7115 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 12:24:43.394210  7104 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.213s	user 0.002s	sys 0.000s
I20260709 12:24:43.395428  7097 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.353720  7094 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.403244  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.408406  7121 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:43.410632  7112 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.210s	user 0.002s	sys 0.000s
I20260709 12:24:43.412324  6204 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:43.420498  6459 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.416478  7025 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.366813  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:43.425662  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.426466  6456 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:43.428864  6383 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.429538  6348 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:24:43.428862  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62142.3us
I20260709 12:24:43.410684  7121 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.432905  6996 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:43.433307  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59388.5us
I20260709 12:24:43.433987  6272 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.434839  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.435883  6455 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:43.436275  6455 pending_rounds.cc:85] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 1
I20260709 12:24:43.436570  6455 pending_rounds.cc:107] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:43.437472  7091 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:43.437667  7122 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.436911  6455 raft_consensus.cc:2889] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.437964  7091 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.451179  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.452940  6307 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:43.456190  6351 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.456595  6351 leader_election.cc:403] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.457779  6351 leader_election.cc:400] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.458096  6351 leader_election.cc:403] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.468490  7126 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.469048  7126 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.471311  7126 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.471220  7127 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.474701  7097 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:43.437486  7008 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:43.480664  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:43.479924  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.484611  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.484252  6206 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:43.484303  6309 raft_consensus.cc:2368] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.488301  6383 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:43.448911  7091 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.443796  7120 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:43.496922  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.499097  6459 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.484040  7008 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.528789  7008 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.531011  6893 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:43.494088  7098 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:43.492154  7101 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:24:43.508692  6454 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.505623  7120 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:43.535295  7120 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.480589  7113 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:24:43.570938  6348 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.537151  6308 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:24:43.581094  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 257334us
I20260709 12:24:43.582816  7101 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:43.583479  7101 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:43.584452  7101 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.564690  6302 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.598711  6274 leader_election.cc:400] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.580343  6893 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.580837  7025 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.613991  6274 leader_election.cc:400] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.615262  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:43.627688  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.630648  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.634825  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.636274  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:43.649217  6381 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.624887  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:43.650238  6304 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:43.660338  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:43.661494  6457 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:24:43.601083  7096 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:24:43.622951  7011 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:43.613037  6378 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:43.662544  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:43.670415  7100 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.672626  7060 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.673413  7011 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.622139  7099 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.052s
I20260709 12:24:43.616276  7025 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.616456  6893 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.721041  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:43.655138  6450 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:43.726922  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:43.676934  7096 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:43.728802  6459 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:43.705013  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 78928.3us
I20260709 12:24:43.730151  6382 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:43.677107  7102 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:43.706530  7011 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:43.735036  7121 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.735796  7121 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.736449  7008 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.739508  6305 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:43.745344  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:43.727798  6457 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
W20260709 12:24:43.748314  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 183590us
I20260709 12:24:43.718972  7126 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:24:43.755155  7125 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:43.760154  6379 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:43.770237  7018 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:24:43.805339  7100 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.805778  7092 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:43.809422  7091 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.787860  7025 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:43.810698  7125 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:43.811143  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:43.754473  6300 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:43.822304  7092 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:43.825590  6304 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:43.825730  7092 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:43.849642  6310 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:44.001309  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 432560us
I20260709 12:24:44.021965  6305 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
W20260709 12:24:44.116449  6986 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:24:44.010957  6983 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.128674  7135 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.152s	user 0.000s	sys 0.000s
I20260709 12:24:44.132263  7091 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.119503  7125 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.115345  7011 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.145390  7098 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.158305  7098 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.159686  7011 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.084540  6893 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:44.095641  7060 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.171003  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:44.171886  6455 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:44.171157  7122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.015s	sys 0.029s
I20260709 12:24:44.157809  6983 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:44.227725  6307 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.027s	sys 0.038s
I20260709 12:24:44.213045  7095 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.229830  7122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.024s	sys 0.040s
I20260709 12:24:44.231695  6983 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:44.189884  7120 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.029s	sys 0.011s
W20260709 12:24:44.232698  7120 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.029s	sys 0.011s
W20260709 12:24:44.232805  6307 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.027s	sys 0.039s
W20260709 12:24:44.234808  6452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.020s	sys 0.012s
W20260709 12:24:44.235320  6452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.020s	sys 0.012s
I20260709 12:24:44.231612  7095 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:44.236028  6385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.017s	sys 0.016s
W20260709 12:24:44.245263  6385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.017s	sys 0.017s
I20260709 12:24:44.236299  7098 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.189538  6996 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.197935  6995 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.235101  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:24:44.252125  6302 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.001s	sys 0.020s
W20260709 12:24:44.236133  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.006s	sys 0.029s
W20260709 12:24:44.236308  6377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.010s	sys 0.037s
W20260709 12:24:44.256312  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.008s	sys 0.064s
I20260709 12:24:44.258737  6451 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:44.264505  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.006s	sys 0.029s
W20260709 12:24:44.239163  6383 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.019s	sys 0.001s
W20260709 12:24:44.243530  7007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.013s	sys 0.019s
W20260709 12:24:44.236354  6350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.016s	sys 0.018s
W20260709 12:24:44.267699  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.008s	sys 0.064s
W20260709 12:24:44.276165  6383 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.020s	sys 0.002s
W20260709 12:24:44.266181  6377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.011s	sys 0.037s
W20260709 12:24:44.265219  6302 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.001s	sys 0.021s
I20260709 12:24:44.255812  6995 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.255198  6996 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:44.284543  6350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.016s	sys 0.018s
I20260709 12:24:44.303835  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.305799  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.308902  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:44.282424  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.328878  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:44.331836  6384 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
W20260709 12:24:44.310312  7128 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.772s	user 0.003s	sys 0.016s
I20260709 12:24:44.330451  6381 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:44.336642  6455 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:44.338354  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:44.340574  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:44.341009  6378 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.342319  6996 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:44.342936  6349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.001s	sys 0.023s
W20260709 12:24:44.343302  6349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.002s	sys 0.023s
I20260709 12:24:44.313215  7099 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.379199  7133 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:24:44.343861  7011 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:24:44.386051  7139 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.386729  7139 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.388262  7102 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.389199  7139 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:44.380699  7099 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.402974  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:44.403782  6309 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.382479  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:24:44.419351  7141 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:24:44.424060  7099 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.396189  7102 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.439213  6307 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:24:44.448714  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 137241us
I20260709 12:24:44.460686  6309 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:44.464406  6274 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:44.464797  6274 leader_election.cc:403] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.467258  7099 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3.
I20260709 12:24:44.470242  6305 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:44.473559  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:44.469447  7095 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.480379  6451 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:44.481367  6995 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.472951  7144 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:44.463586  7098 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:24:44.490110  7157 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:44.492039  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 361) took 1166 ms. Trace:
I20260709 12:24:44.492213  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 303) took 1015 ms. Trace:
I20260709 12:24:44.486881  7125 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.493721  6310 rpcz_store.cc:276] 0709 12:24:43.477305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.477424 (+   119us) service_pool.cc:224] Handling call
0709 12:24:43.706178 (+228754us) spinlock_profiling.cc:243] Waited 5.14 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:24:43.718167 (+ 11989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.850157 (+131990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.491905 (+641748us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54000a9954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:24:44.492139 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313663744}
I20260709 12:24:44.492650  6379 rpcz_store.cc:276] 0709 12:24:43.325371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.325755 (+   384us) service_pool.cc:224] Handling call
0709 12:24:44.491752 (+1165997us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b540022f454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:24:44.491916 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905737600}
I20260709 12:24:44.495855  7083 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.503484  7083 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.505517  7083 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:44.503473  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.506309  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.486102  7098 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.506742  6308 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:44.484952  7102 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.508096  6351 leader_election.cc:400] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:44.508334  6351 leader_election.cc:403] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.508419  7098 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.508757  7060 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:24:44.513265  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.513922  6451 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:44.495318  6303 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:44.515029  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 363) took 1063 ms. Trace:
I20260709 12:24:44.516614  7125 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.520385  7098 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.526721  6310 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:44.527133  7137 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.523488  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.527694  7137 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.528939  6302 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:44.546595  7095 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.515404  6303 rpcz_store.cc:276] 0709 12:24:43.451687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.462885 (+ 11198us) service_pool.cc:224] Handling call
0709 12:24:43.496183 (+ 33298us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:24:44.377682 (+881499us) spinlock_profiling.cc:243] Waited 9.66 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182e9c 00007f6fa0182a5b 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:24:44.514939 (+137257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35219072}
I20260709 12:24:44.549553  7007 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:44.550211  7150 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:44.550798  7137 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:44.551033  7150 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.550348  7125 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.553470  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:44.555069  7008 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.552112  7150 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.558117  6204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 125) took 1170 ms. Trace:
I20260709 12:24:44.565455  7121 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.570075  7150 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.561553  6204 rpcz_store.cc:276] 0709 12:24:43.387905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.388081 (+   176us) service_pool.cc:224] Handling call
0709 12:24:44.558020 (+1169939us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:43.415598 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:43.416260 (+   662us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:43.416294 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:24:43.416315 (+    21us) tablet.cc:662] Decoding operations
0709 12:24:43.416634 (+   319us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:43.416690 (+    56us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:43.416722 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:43.427073 (+ 10351us) tablet.cc:701] Row locks acquired
0709 12:24:43.428139 (+  1066us) write_op.cc:260] PREPARE: finished
0709 12:24:43.445816 (+ 17677us) write_op.cc:270] Start()
0709 12:24:43.445948 (+   132us) write_op.cc:276] Timestamp: P: 1783599883443606 usec, L: 0
0709 12:24:43.445975 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:24:43.446636 (+   661us) log.cc:844] Serialized 531 byte log entry
0709 12:24:43.511153 (+ 64517us) op_driver.cc:464] REPLICATION: finished
0709 12:24:44.247286 (+736133us) write_op.cc:301] APPLY: starting
0709 12:24:44.252892 (+  5606us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:44.258411 (+  5519us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:44.287051 (+ 28640us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:44.307598 (+ 20547us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:44.334596 (+ 26998us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:44.346990 (+ 12394us) write_op.cc:312] APPLY: finished
0709 12:24:44.398766 (+ 51776us) log.cc:844] Serialized 22 byte log entry
0709 12:24:44.414144 (+ 15378us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:44.429023 (+ 14879us) write_op.cc:454] Released schema lock
0709 12:24:44.430942 (+  1919us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:44.473180 (+ 42238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:44.474123 (+   943us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":697905,"apply.run_cpu_time_us":3675,"apply.run_wall_time_us":210091,"num_ops":1,"prepare.queue_time_us":125,"prepare.run_cpu_time_us":2792,"prepare.run_wall_time_us":31330,"replication_time_us":58164,"spinlock_wait_cycles":1838336,"thread_start_us":280949,"threads_started":1}]]}
I20260709 12:24:44.577824  7092 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:44.582810  7165 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.106s	user 0.003s	sys 0.000s
I20260709 12:24:44.585220  6206 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 136) took 1116 ms. Trace:
I20260709 12:24:44.588976  7095 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:24:44.591501  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:44.595619  6302 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:44.601310  6457 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:24:44.601747  6457 pending_rounds.cc:85] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 1
I20260709 12:24:44.601958  6457 pending_rounds.cc:107] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:24:44.602187  6457 raft_consensus.cc:2889] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.603835  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.604455  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:44.607265  6455 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.608255  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:44.610327  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.612386  6424 leader_election.cc:400] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:44.612761  6424 leader_election.cc:403] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:24:44.613574  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.604588  6450 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:44.615541  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.618381  6456 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:24:44.614686  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.615425  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.615913  6380 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.616744  6995 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.620685  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:44.621096  6995 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.622084  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.623350  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:44.623751  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.623976  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.624403  6376 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:44.624641  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.625452  6310 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:44.625592  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.589748  7095 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.619328  6459 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:24:44.627295  6383 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.620024  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.629729  7095 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.629858  6378 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.626986  7162 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:44.623202  6995 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.593582  6205 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.628834  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.578445  7098 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:44.632921  7098 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.586028  6206 rpcz_store.cc:276] 0709 12:24:43.468618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.471863 (+  3245us) service_pool.cc:224] Handling call
0709 12:24:44.585114 (+1113251us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:43.507540 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:43.613639 (+106099us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:43.617181 (+  3542us) write_op.cc:435] Acquired schema lock
0709 12:24:43.617198 (+    17us) tablet.cc:662] Decoding operations
0709 12:24:43.686054 (+ 68856us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:43.706003 (+ 19949us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:43.719673 (+ 13670us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:43.741913 (+ 22240us) tablet.cc:701] Row locks acquired
0709 12:24:43.741942 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:24:43.760259 (+ 18317us) write_op.cc:270] Start()
0709 12:24:43.790969 (+ 30710us) write_op.cc:276] Timestamp: P: 1783599883757877 usec, L: 0
0709 12:24:43.791011 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 12:24:43.971434 (+180423us) log.cc:844] Serialized 531 byte log entry
0709 12:24:44.117876 (+146442us) op_driver.cc:464] REPLICATION: finished
0709 12:24:44.463888 (+346012us) write_op.cc:301] APPLY: starting
0709 12:24:44.469684 (+  5796us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:44.474462 (+  4778us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:44.494307 (+ 19845us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:44.500790 (+  6483us) spinlock_profiling.cc:243] Waited 86.3 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a2e28 00007f6fa6cdd581 00007f6fa6cdcd58 00007f6fa6ce19f1 00007f6fa6ce19a9 00007f6fa6ce1939 00007f6fa6ce1901 00007f6fa6ce0b2d 0000000000388634 0000000000388479
0709 12:24:44.509191 (+  8401us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:44.509369 (+   178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:44.509420 (+    51us) write_op.cc:312] APPLY: finished
0709 12:24:44.510902 (+  1482us) log.cc:844] Serialized 22 byte log entry
0709 12:24:44.511071 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:44.511150 (+    79us) write_op.cc:454] Released schema lock
0709 12:24:44.511182 (+    32us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:44.511519 (+   337us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:44.511544 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":46612,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":315556,"apply.run_cpu_time_us":2402,"apply.run_wall_time_us":72866,"num_ops":1,"prepare.queue_time_us":965,"prepare.run_cpu_time_us":20515,"prepare.run_wall_time_us":1000934,"replication_time_us":323671,"spinlock_wait_cycles":198536064,"thread_start_us":216724,"threads_started":1}]]}
I20260709 12:24:44.618395  7092 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:24:44.633783  7098 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.639178  7008 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:44.639882  7008 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:44.642684  7008 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:44.631181  7162 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.624727  6380 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:24:44.652030  7008 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:44.582351  7060 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.648336  7162 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.644496  6272 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:44.760059  7162 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:44.761260  7162 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.761847  7162 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:44.759807  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 115418us
I20260709 12:24:44.775805  6272 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:44.777745  6205 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 124) took 1278 ms. Trace:
I20260709 12:24:44.781351  7162 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.783164  7144 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:44.787256  7144 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:44.788934  6272 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:44.790429  7144 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:44.778116  6205 rpcz_store.cc:276] 0709 12:24:43.529037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.560944 (+ 31907us) service_pool.cc:224] Handling call
0709 12:24:44.546181 (+985237us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:24:44.777680 (+231499us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:44.652905 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:44.761053 (+108148us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:44.761101 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:24:44.761120 (+    19us) tablet.cc:662] Decoding operations
0709 12:24:44.761539 (+   419us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:44.761614 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:44.761642 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:44.766624 (+  4982us) tablet.cc:701] Row locks acquired
0709 12:24:44.766656 (+    32us) write_op.cc:260] PREPARE: finished
0709 12:24:44.766845 (+   189us) write_op.cc:270] Start()
0709 12:24:44.766971 (+   126us) write_op.cc:276] Timestamp: P: 1783599884766790 usec, L: 0
0709 12:24:44.766998 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:24:44.767715 (+   717us) log.cc:844] Serialized 531 byte log entry
0709 12:24:44.769379 (+  1664us) op_driver.cc:464] REPLICATION: finished
0709 12:24:44.769732 (+   353us) write_op.cc:301] APPLY: starting
0709 12:24:44.769810 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:44.769950 (+   140us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:44.770048 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:44.770230 (+   182us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:44.770379 (+   149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:44.770424 (+    45us) write_op.cc:312] APPLY: finished
0709 12:24:44.770644 (+   220us) log.cc:844] Serialized 22 byte log entry
0709 12:24:44.770807 (+   163us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:44.770896 (+    89us) write_op.cc:454] Released schema lock
0709 12:24:44.770943 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:44.771254 (+   311us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:44.771289 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":42237,"reported_tablets":2,"spinlock_wait_cycles":731484416,"child_traces":[["op",{"apply.queue_time_us":112,"apply.run_cpu_time_us":1479,"apply.run_wall_time_us":1502,"num_ops":1,"prepare.queue_time_us":18829,"prepare.run_cpu_time_us":11647,"prepare.run_wall_time_us":115117,"replication_time_us":2260}]]}
I20260709 12:24:44.794708  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:44.796934  7144 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.797310  7144 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.797641  7144 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.653692  7150 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.749336  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:44.807142  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56309.7us
I20260709 12:24:44.808923  7121 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.809505  7121 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.811728  7121 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.813237  6272 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:44.812911  6205 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:44.814433  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 160247us
I20260709 12:24:44.814805  6995 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:44.824802  7098 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:44.759683  7008 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:44.832403  7174 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:24:44.833253  7127 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:44.835485  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:44.840153  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:44.835628  6995 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:44.842125  6995 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:44.836570  6205 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:44.848384  6205 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:24:44.849727  7007 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:44.850358  6205 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:44.834465  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:44.850644  7097 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:24:44.837708  7127 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:44.835810  7098 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.849653  7101 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:44.850975  7102 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.851454  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:24:44.895535  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61189.5us
W20260709 12:24:44.858848  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45680.8us
I20260709 12:24:44.896473  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:44.862620  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.877336  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.881055  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.894836  6206 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:44.906394  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:44.911896  6893 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.905119  7094 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:44.912557  6893 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.857081  7102 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.866973  7160 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.854607  7149 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:44.920476  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.860613  6996 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:44.921263  6310 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.925494  7149 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.925678  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.859364  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:44.926342  7149 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.891803  6995 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.927157  7144 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.928719  7160 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.929119  7160 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.929128  6995 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.929927  7008 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.904979  7101 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:44.931586  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.932490  6302 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:44.932806  7144 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.857182  7007 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:24:44.935026  7007 consensus_queue.cc:1243] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:44.925028  6996 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.936060  6996 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.938320  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.922868  7102 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:44.938786  6303 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.941254  6303 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.943616  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.927552  7149 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.855173  7127 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.951609  7127 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.952790  7127 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.931901  7101 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:44.954131  7101 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.857290  7095 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.957535  7095 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.959841  7095 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.856184  7098 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.961935  7098 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:44.967229  6385 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:44.967732  6385 pending_rounds.cc:85] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 1
I20260709 12:24:44.967981  6385 pending_rounds.cc:107] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:44.968204  6385 raft_consensus.cc:2889] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:44.972905  7122 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:44.973505  7122 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.976172  7122 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.976415  7025 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.976863  7025 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.979844  7025 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.981974  6302 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:24:44.983198  6310 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:44.981961  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.984081  7180 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:44.986197  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.991041  6983 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:44.991223  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:44.991696  6983 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.992271  7092 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:44.992844  7092 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.994670  7008 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:44.995201  7008 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.996299  6983 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:44.997293  7008 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.984658  7180 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:24:44.999454  7092 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:44.996472  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94751.7us
I20260709 12:24:45.007133  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.001092  6996 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:44.950253  6893 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:45.009462  6996 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.009332  7146 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.004259  7160 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.009706  6348 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.010193  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.010588  6309 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:24:45.010838  7178 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.003s
I20260709 12:24:45.012290  7180 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.012997  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.013674  6310 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:45.013859  7160 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.015599  7102 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.017525  7102 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:44.854228  7162 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.018851  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.020371  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.023710  6305 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:45.025341  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:45.025872  7020 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:45.032950  7092 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.034319  7096 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:45.036325  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:45.037537  7099 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:45.038120  7099 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.039594  7092 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:45.043286  7099 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.043789  7092 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.044209  7092 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.047569  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.047963  7096 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.048281  6451 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:24:45.049263  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.050276  6424 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:45.051417  7096 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:45.051769  7096 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:45.052139  7096 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.052495  7092 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.056133  6983 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:45.057649  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.064204  7092 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:45.064677  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.064855  7092 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:45.065307  6458 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.068468  6455 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.068650  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.069366  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.069581  6274 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.070981  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.073602  7087 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.074545  7083 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.058363  6453 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.050385  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.086576  6458 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:45.088667  6457 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:45.092931  7096 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.095942  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.096346  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.097155  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.097805  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.098544  7098 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.099031  7098 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.100652  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.101023  7098 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.106601  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.128253  6459 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:45.129207  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:45.106841  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:45.110862  6455 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:45.098373  6454 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:45.116904  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:45.123047  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.123865  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.124490  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.120330  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.137802  6382 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:45.120015  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.115599  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.137547  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.127547  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.131515  6385 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:24:45.132344  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62849.5us
W20260709 12:24:45.132922  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65231.7us
I20260709 12:24:45.141417  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.141618  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.133150  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:45.134974  6310 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:45.141826  6274 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.143872  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:24:45.144544  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74159.9us
I20260709 12:24:45.146212  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.151459  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.121732  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:45.156623  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:45.156129  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.161425  6378 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:24:45.137250  7025 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:45.163250  7096 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:45.164108  7144 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.164911  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:45.165333  7180 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.165779  6308 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:45.166211  7101 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:45.166703  7101 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:45.167090  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.167716  7101 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.161741  6453 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:24:45.167116  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.170362  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.174604  6351 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.175453  6424 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.189622  7098 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:45.191238  7146 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.191728  6351 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.193735  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.178750  6983 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.193401  6206 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 3 to 4, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:24:45.196718  7120 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.182914  7122 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:45.172537  7087 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.205049  7025 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:45.207548  6454 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:45.173957  7125 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:45.174070  7020 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:45.167763  6303 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:45.174134  7101 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.210927  6893 raft_consensus.cc:2764] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:24:45.213729  6272 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.216305  6348 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:45.217769  6272 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:45.222723  7162 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:45.215440  6995 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:45.223469  7162 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:45.224706  7162 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.211493  7125 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:45.229645  7125 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.217971  7122 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.214758  6996 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:45.232163  6996 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:45.232955  6996 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.238658  6996 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.237996  7146 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:45.245054  7146 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:45.246078  7146 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.249272  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:45.241677  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.252722  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.214687  7160 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:24:45.251632  7122 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:45.255039  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:45.259388  7122 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:45.258646  7092 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:45.260610  7092 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:45.260435  7122 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.261155  7092 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.285356  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:45.286263  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.288779  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.272253  6995 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.290525  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.281177  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:45.290377  7096 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.301637  6302 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:45.301740  7095 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.314059  6351 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.323215  6983 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:45.323930  6983 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.324225  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.337532  6893 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:45.320075  7137 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:45.341673  7137 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.343930  7125 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.328897  7149 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.346055  7125 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:45.338294  6893 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.317847  7083 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.367920  6983 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:45.377295  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:45.485756  7184 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:24:45.528434  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 212862us
I20260709 12:24:45.529980  6424 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:45.530851  7096 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:45.531368  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:45.535356  6450 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.545303  6307 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:24:45.553207  7194 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:24:45.565348  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:45.557884  7150 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:45.596361  6304 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.604504  6302 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:45.605371  6300 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:45.609447  7092 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.619897  6454 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:45.620762  7092 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:45.592206  6995 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:24:45.660681  7120 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:45.632686  7098 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:24:45.666579  7137 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:45.667197  6893 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:24:45.714031  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 142522us
W20260709 12:24:45.728588  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 161659us
I20260709 12:24:45.730911  6305 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:45.732733  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:45.722704  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.746855  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.771377  7160 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:45.801801  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:24:45.810949  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.812254  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.792243  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:24:45.832291  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85718.7us
I20260709 12:24:45.832679  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:45.789331  7060 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
W20260709 12:24:45.876101  7204 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:24:45.876624  6379 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:45.882599  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:45.844468  6458 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:45.845263  7137 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:45.889683  7137 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:45.902136  7208 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:45.903707  7208 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:45.904160  7208 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:45.905282  7208 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:45.906993  7208 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:45.932250  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 99228.5us
I20260709 12:24:45.933338  6274 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.892283  7137 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:45.984053  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55032.5us
W20260709 12:24:45.984977  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 151170us
W20260709 12:24:45.987531  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 182618us
W20260709 12:24:45.992235  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 184831us
W20260709 12:24:45.998355  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65119us
I20260709 12:24:45.999440  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:46.004487  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:45.983958  7092 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:46.008296  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:46.009925  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:24:46.067384  7096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.019s	sys 0.034s
W20260709 12:24:46.076416  7096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.019s	sys 0.034s
I20260709 12:24:46.069669  7060 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:24:46.081799  6303 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:46.057607  7092 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.085637  7092 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:46.066460  7162 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:46.021106  7011 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:46.018627  7094 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.069372  7195 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:46.109536  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 112834us
W20260709 12:24:46.121745  6254 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:24:46.064136  7150 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:24:46.075498  7198 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.286s	user 0.000s	sys 0.003s
I20260709 12:24:46.073185  7225 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:46.072414  7206 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.126511  6205 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 3, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.131911  7150 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:46.074718  7213 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:24:46.165176  7144 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:46.165735  7144 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:46.173206  7225 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:24:46.174718  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.187546  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 177636us
W20260709 12:24:46.187877  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 124436us
I20260709 12:24:46.193367  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:46.198637  6425 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:46.202191  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:24:46.206419  6458 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.203857  7092 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.204542  7230 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.211743  6995 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:46.204773  6378 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:46.185583  7144 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.217679  7137 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:46.295244  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:46.296868  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.297632  6309 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:46.302619  7232 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.303174  7232 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.297906  7144 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:46.304361  7011 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:46.305347  7232 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:24:46.298354  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.237217  6995 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.315654  6451 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
W20260709 12:24:46.342689  7226 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:24:46.343595  7225 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:46.347703  6303 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.338685  7095 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:46.355376  7095 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:46.358436  7095 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.359557  7150 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.360898  7095 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.361827  7150 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.375945  6995 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.377370  6379 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:46.377787  6379 pending_rounds.cc:85] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 1
I20260709 12:24:46.378043  6379 pending_rounds.cc:107] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:46.378309  6379 raft_consensus.cc:2889] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.386050  6995 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.386656  6995 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.367184  7025 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 12:24:46.411859  7101 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:46.412534  7101 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.414341  7233 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:46.415892  6995 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.416353  7233 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.420234  7101 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.420867  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.422925  7233 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.400955  7150 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:46.427145  7150 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.424772  6450 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:24:46.429001  7150 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.493358  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:46.514997  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.517174  6301 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:46.546867  7160 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:46.552340  7011 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:46.558846  7223 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:24:46.556666  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:46.560936  6305 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:46.558101  6310 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:46.561702  6206 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.573262  6206 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.574980  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:46.553566  7011 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.595417  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:46.598507  7208 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:46.599231  7208 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:46.594208  7011 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.657011  7217 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.600929  7208 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.663136  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:46.658056  7217 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.679360  6206 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 3, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.694063  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.696774  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.697608  6300 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:46.699873  6378 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:46.708511  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.688829  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:46.831542  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.831934  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.831736  7121 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:24:46.832439  6453 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:24:46.831887  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.835625  7180 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:46.836326  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.836532  7180 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.838402  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.838894  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:46.838742  6893 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:46.839646  6378 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:24:46.820549  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 112671us
I20260709 12:24:46.842648  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:46.848291  7180 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.854825  6454 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:46.863348  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:46.864231  6378 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:46.865026  6379 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:24:46.865509  6379 pending_rounds.cc:85] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 1
W20260709 12:24:46.865576  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 182256us
I20260709 12:24:46.865854  6379 pending_rounds.cc:107] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:24:46.866278  6379 raft_consensus.cc:2889] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.866725  7011 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:46.867233  7011 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:46.867653  7011 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:46.874517  7217 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.876048  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.898761  7231 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:46.899813  7231 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.915261  7060 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:46.915890  7231 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:46.937111  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:46.944084  6379 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:24:46.939859  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:46.947712  7060 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.949680  7011 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.958177  7231 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.960050  7231 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:46.965582  6305 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:46.965612  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:46.974198  7231 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.003978  7060 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.003782  6205 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 2, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.005846  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.008160  6206 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 125) took 1650 ms. Trace:
I20260709 12:24:47.026957  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.026590  6205 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 0 to 2, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:24:47.027936  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.029011  7102 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.030511  7102 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:47.030988  7102 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.017215  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.032137  7231 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:24:47.025799  6206 rpcz_store.cc:276] 0709 12:24:45.357806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.358688 (+   882us) service_pool.cc:224] Handling call
0709 12:24:47.008064 (+1649376us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:46.698581 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:46.699408 (+   827us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:46.699479 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:24:46.699531 (+    52us) tablet.cc:662] Decoding operations
0709 12:24:46.700260 (+   729us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:46.700372 (+   112us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:46.700406 (+    34us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:24:46.700907 (+   501us) tablet.cc:701] Row locks acquired
0709 12:24:46.700966 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:24:46.701148 (+   182us) write_op.cc:270] Start()
0709 12:24:46.701340 (+   192us) write_op.cc:276] Timestamp: P: 1783599886701089 usec, L: 0
0709 12:24:46.701379 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:24:46.702280 (+   901us) log.cc:844] Serialized 1289 byte log entry
0709 12:24:46.856572 (+154292us) op_driver.cc:464] REPLICATION: finished
0709 12:24:46.857433 (+   861us) write_op.cc:301] APPLY: starting
0709 12:24:46.857556 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:46.857737 (+   181us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:46.857871 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:46.859619 (+  1748us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:46.859969 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:24:46.860029 (+    60us) write_op.cc:312] APPLY: finished
0709 12:24:46.863181 (+  3152us) log.cc:844] Serialized 34 byte log entry
0709 12:24:46.863398 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:46.863519 (+   121us) write_op.cc:454] Released schema lock
0709 12:24:46.863575 (+    56us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:46.871954 (+  8379us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:46.871989 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":349066,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":493,"apply.run_cpu_time_us":3777,"apply.run_wall_time_us":14495,"num_ops":3,"prepare.queue_time_us":276,"prepare.run_cpu_time_us":4186,"prepare.run_wall_time_us":9681,"replication_time_us":154980}]]}
I20260709 12:24:47.019276  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.034809  6310 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:47.038995  7246 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:24:47.039860  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.044389  6424 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:47.045321  7215 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:47.007967  7011 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.046685  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.051046  7189 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:47.051491  7189 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:47.052425  7188 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:47.053169  7188 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:47.052491  7189 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.053544  7188 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:47.054157  7189 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.031474  7102 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.067226  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:47.068581  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.055310  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.070924  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.071182  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.073143  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.081418  6204 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.092779  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.083606  7060 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.094571  6204 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.096123  7206 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.123070  6351 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.126529  7150 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:47.128254  6351 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.130957  7150 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.131697  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.132639  7217 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.135068  7188 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.166960  6454 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:47.168169  7188 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:47.169565  6348 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.170629  7231 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:47.171010  7231 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.171377  7231 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.171118  7150 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:47.172549  7180 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:47.172915  7180 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:47.173837  7180 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.175561  7180 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.187662  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.188462  7150 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:47.197602  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 129726us
I20260709 12:24:47.199339  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.200067  7060 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:47.202040  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 121007us
I20260709 12:24:47.210757  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.211398  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.211814  6304 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:47.201318  7231 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.214335  7231 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.216084  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:47.216634  6454 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.226311  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.227510  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:47.203014  7102 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.230372  7150 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.238683  7208 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:47.230108  7060 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.239837  7060 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.241707  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:47.242627  7149 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.243938  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.243125  7149 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.237466  6893 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:47.245453  6893 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:47.245782  6893 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:47.246434  7149 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.248088  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:47.246228  7102 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.242367  7060 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.256258  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:47.256938  6456 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:24:47.242437  6385 raft_consensus.cc:2368] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:24:47.263480  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:47.268360  6424 leader_election.cc:400] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:24:47.268656  6424 leader_election.cc:403] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:47.269084  6454 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:24:47.278321  6348 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.279258  6348 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.280333  7060 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:47.280596  7102 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:47.280695  7060 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.281095  7060 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.281153  7102 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:47.282078  7102 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.287833  6377 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:24:47.294441  7262 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:24:47.298763  6893 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.303231  6450 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.311549  7262 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:47.312122  7262 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.314252  7262 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:47.315897  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.316895  6300 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:47.319489  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.320075  6300 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.322018  6310 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:47.323315  6893 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:47.323525  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:47.329648  6382 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:24:47.325434  7208 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:47.331579  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.332453  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.333845  6310 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:47.335216  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.336050  6303 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.338341  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:47.340766  7208 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:47.341158  7208 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:24:47.341516  7208 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.345013  6303 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:24:47.327533  6450 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:24:47.327864  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:47.352757  6376 raft_consensus.cc:2368] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:24:47.354008  6424 leader_election.cc:400] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:24:47.355473  6424 leader_election.cc:403] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:47.356859  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.358161  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.359071  6303 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:24:47.359637  7231 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:47.360203  7231 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:47.361191  7231 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.356166  7262 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:24:47.371282  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.372141  7099 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:47.372606  7099 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:24:47.366881  7208 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.373564  7099 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.396185  6351 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.396642  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.397334  6303 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:24:47.401012  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:47.407140  7217 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:47.413226  7208 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:47.399637  7149 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:47.421558  6300 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:24:47.432108  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:47.442819  7060 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.445101  7060 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:47.447580  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:24:47.448276  6457 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:47.450878  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46951.8us
I20260709 12:24:47.455425  6376 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:47.456472  6385 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:24:47.458779  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:47.459141  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.459789  6305 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.461825  7262 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:47.460340  7208 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:24:47.478813  7262 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:47.479862  7262 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.485731  6305 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:24:47.489612  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:24:47.492926  7264 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:47.493533  7264 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:24:47.494405  7264 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.504627  6457 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:24:47.515743  7188 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.516291  7188 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.522439  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:47.523272  6303 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:24:47.524328  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.525170  7262 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:47.525553  7262 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.525892  7262 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:47.526201  6206 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.518405  7188 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:47.533085  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:47.553157  7262 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.555663  6205 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 3 to 4, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.574914  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:47.577000  6300 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:47.585083  6300 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:24:47.586612  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:24:47.587514  7188 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:47.588376  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:47.596406  7188 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:47.597505  7188 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:47.600692  7262 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:47.610057  6204 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 3, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:47.610998  6205 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.614230  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 12:24:47.613991  6205 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 5, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:47.615522  6205 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:47.627689  7276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:24:47.632977  7277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:47.644629  5137 server_base.cc:1061] running on GCE node
W20260709 12:24:47.644513  7280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:24:47.646353  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:24:47.646713  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:24:47.647003  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783599887646982 us; error 0 us; skew 500 ppm
I20260709 12:24:47.647861  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:24:47.667073  5137 webserver.cc:533] Webserver started at http://127.5.4.68:36765/ using document root <none> and password file <none>
I20260709 12:24:47.667956  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:24:47.668238  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:24:47.668669  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:24:47.670603  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-3-root/instance:
uuid: "561a4e21d8b344a79af44568ca80a3c5"
format_stamp: "Formatted at 2026-07-09 12:24:47 on dist-test-slave-lg8f"
I20260709 12:24:47.691460  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.020s	user 0.006s	sys 0.014s
I20260709 12:24:47.701602  7284 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:47.702656  5137 fs_manager.cc:730] Time spent opening block manager: real 0.008s	user 0.002s	sys 0.000s
I20260709 12:24:47.703107  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-3-root
uuid: "561a4e21d8b344a79af44568ca80a3c5"
format_stamp: "Formatted at 2026-07-09 12:24:47 on dist-test-slave-lg8f"
I20260709 12:24:47.703510  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:24:47.746268  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:24:47.747980  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:24:47.749676  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:24:47.752463  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:24:47.752779  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:47.753041  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:24:47.753201  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:24:47.779686  6451 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:47.782083  7231 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:47.815984  6300 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:47.825516  7150 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:47.889952  6305 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:47.905654  7217 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:47.910658  6376 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:47.912048  6303 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:47.918533  6450 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:47.926331  7231 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:47.919699  6451 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:47.922534  7188 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:47.946645  7217 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:47.941885  7262 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:47.958365  6450 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:47.969988  6385 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:48.008466  7234 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:48.050942  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45625.4us
I20260709 12:24:48.121003  6450 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:48.136529  7336 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:48.187825  6305 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:48.190817  7337 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:48.210726  7339 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:24:48.211377  7339 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:48.213568  7339 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:48.255380  6305 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:48.260982  6379 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:48.290596  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:48.295115  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:48.311108  6379 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:48.321413  7188 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:48.352082  7339 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:48.356294  6351 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:24:48.363224  7262 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.363862  7262 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:48.365870  7334 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:48.369226  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:24:48.372303  7262 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:48.387804  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:48.388592  6381 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:24:48.404728  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:48.408469  7209 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:48.409056  7209 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.409530  7209 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:48.451462  7209 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:48.454357  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:24:48.455926  7209 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:48.458521  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:24:48.459406  6381 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:48.471071  6381 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:24:48.472678  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:24:48.473768  7209 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:48.474289  7209 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:24:48.475185  7209 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:48.493139  6204 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:48.496618  7336 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:24:48.497221  7336 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:24:48.500296  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:24:48.500943  6381 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:24:48.501358  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:24:48.503505  6274 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:24:48.503837  6274 leader_election.cc:403] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:48.504412  7333 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:48.519663  7336 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:48.528705  7333 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:24:48.537258  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.68:43625
I20260709 12:24:48.537343  7371 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.68:43625 every 8 connection(s)
I20260709 12:24:48.549708  5137 auto_leader_rebalancer-test.cc:439] add a tserver: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:48.570365  7373 heartbeater.cc:344] Connected to a master server at 127.5.4.126:35813
I20260709 12:24:48.570998  7373 heartbeater.cc:461] Registering TS with master...
I20260709 12:24:48.572028  7373 heartbeater.cc:507] Master 127.5.4.126:35813 requested a full tablet report, sending...
I20260709 12:24:48.574208  6204 ts_manager.cc:194] Registered new tserver with Master: 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:24:48.575937  6204 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50336
I20260709 12:24:48.705931  6305 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:48.715636  6381 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:48.716229  6457 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:48.717756  7325 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:48.719906  7333 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:48.757603  7325 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:24:48.760116  6275 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f24bf2d0bc904aa495cdb5995b470fb0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:48.760921  6451 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:48.762341  6381 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:48.818451  6204 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:48.856978  6381 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:48.868196  6451 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:48.869761  7334 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:48.872323  6304 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:48.875100  7334 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:24:48.883610  6351 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2ddf475f266c47adb1a50aa986a0c44c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:48.890264  6305 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:48.893580  7262 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:48.917079  7334 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:48.928483  6451 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:48.931607  6379 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:48.939277  6303 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:48.941928  7391 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:48.962071  6204 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.029424  6450 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.043020  6378 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:49.044234  7391 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:49.051846  6305 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:49.055667  7188 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:24:49.064661  6425 consensus_peers.cc:597] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a2f5503246494eed8b91ac10cdf742e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:49.075361  7262 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.077898  6378 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.085649  6305 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.100526  6204 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:49.101938  6451 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:24:49.139200  6425 consensus_peers.cc:597] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7db3e095e8b44b23b1a439899d305bd7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:49.140262  6305 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:49.141683  6379 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:49.141842  7188 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:49.143944  7391 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:49.177815  7391 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.180562  6305 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.187297  6378 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.199397  6204 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:49.300379  7405 ts_tablet_manager.cc:933] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:24:49.302398  7405 tablet_copy_client.cc:323] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:24:49.331950  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet f24bf2d0bc904aa495cdb5995b470fb0 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:24:49.332688  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet f24bf2d0bc904aa495cdb5995b470fb0 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-f24bf2d0bc904aa495cdb5995b470fb0
I20260709 12:24:49.342473  6320 tablet_copy_source_session.cc:215] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:49.347780  7405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f24bf2d0bc904aa495cdb5995b470fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:49.367702  7405 tablet_copy_client.cc:806] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:49.368539  7405 tablet_copy_client.cc:670] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:49.373925  7405 tablet_copy_client.cc:538] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:49.386339  7405 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:49.423502  7405 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:49.424739  7405 tablet_bootstrap.cc:492] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:49.425557  7405 ts_tablet_manager.cc:1403] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.007s
I20260709 12:24:49.429499  7405 raft_consensus.cc:359] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.430452  7405 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:49.431185  7405 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.444571  7405 ts_tablet_manager.cc:1434] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:24:49.445600  7373 heartbeater.cc:499] Master 127.5.4.126:35813 was elected leader, sending a full tablet report...
I20260709 12:24:49.451452  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f24bf2d0bc904aa495cdb5995b470fb0 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:24:49.451995  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f24bf2d0bc904aa495cdb5995b470fb0 on tablet f24bf2d0bc904aa495cdb5995b470fb0 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:49.466725  7405 ts_tablet_manager.cc:933] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:49.517838  7405 tablet_copy_client.cc:323] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:24:49.577013  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 2ddf475f266c47adb1a50aa986a0c44c from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:24:49.577638  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 2ddf475f266c47adb1a50aa986a0c44c from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-2ddf475f266c47adb1a50aa986a0c44c
I20260709 12:24:49.584419  6395 tablet_copy_source_session.cc:215] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:49.589318  7405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ddf475f266c47adb1a50aa986a0c44c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:49.610544  7405 tablet_copy_client.cc:806] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:49.611073  7405 tablet_copy_client.cc:670] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:49.619817  7405 tablet_copy_client.cc:538] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:49.632180  7405 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:49.654989  7414 ts_tablet_manager.cc:933] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:49.659711  7405 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:49.662377  7405 tablet_bootstrap.cc:492] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:49.663035  7414 tablet_copy_client.cc:323] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:24:49.663302  7405 ts_tablet_manager.cc:1403] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.016s
I20260709 12:24:49.672971  7405 raft_consensus.cc:359] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.673900  7405 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:49.674733  7405 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.685686  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet a2f5503246494eed8b91ac10cdf742e2 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:24:49.686420  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet a2f5503246494eed8b91ac10cdf742e2 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-a2f5503246494eed8b91ac10cdf742e2
I20260709 12:24:49.687083  7405 ts_tablet_manager.cc:1434] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.003s
I20260709 12:24:49.692519  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-2ddf475f266c47adb1a50aa986a0c44c received from {username='slave'} at 127.0.0.1:46610
I20260709 12:24:49.693009  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-2ddf475f266c47adb1a50aa986a0c44c on tablet 2ddf475f266c47adb1a50aa986a0c44c with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:49.698853  6469 tablet_copy_source_session.cc:215] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:49.702453  7414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f5503246494eed8b91ac10cdf742e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:49.720867  7417 ts_tablet_manager.cc:933] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:49.732044  7414 tablet_copy_client.cc:806] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:49.734212  7414 tablet_copy_client.cc:670] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:49.733604  7417 tablet_copy_client.cc:323] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:24:49.735890  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 7db3e095e8b44b23b1a439899d305bd7 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:24:49.736714  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 7db3e095e8b44b23b1a439899d305bd7 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-7db3e095e8b44b23b1a439899d305bd7
I20260709 12:24:49.743963  7414 tablet_copy_client.cc:538] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:49.745824  6469 tablet_copy_source_session.cc:215] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:49.749071  7417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7db3e095e8b44b23b1a439899d305bd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:49.764267  7414 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:49.777900  7417 tablet_copy_client.cc:806] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:49.778738  7417 tablet_copy_client.cc:670] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:49.782107  7417 tablet_copy_client.cc:538] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:49.806932  7414 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:49.807960  7414 tablet_bootstrap.cc:492] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:49.841020  7417 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:49.854729  7414 ts_tablet_manager.cc:1403] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.094s	user 0.016s	sys 0.013s
I20260709 12:24:49.873760  7335 raft_consensus.cc:1217] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:24:49.863961  7414 raft_consensus.cc:359] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.886116  7414 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:49.886971  7414 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.903445  7414 ts_tablet_manager.cc:1434] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.045s	user 0.006s	sys 0.001s
I20260709 12:24:49.905155  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-a2f5503246494eed8b91ac10cdf742e2 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:24:49.905638  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-a2f5503246494eed8b91ac10cdf742e2 on tablet a2f5503246494eed8b91ac10cdf742e2 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:49.909762  7417 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:49.914899  7417 tablet_bootstrap.cc:492] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:49.915530  7417 ts_tablet_manager.cc:1403] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.075s	user 0.024s	sys 0.010s
I20260709 12:24:49.918017  7417 raft_consensus.cc:359] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.918668  7417 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:49.919536  7417 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:49.922206  7417 ts_tablet_manager.cc:1434] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:24:49.924229  7335 raft_consensus.cc:1217] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:24:49.924587  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-7db3e095e8b44b23b1a439899d305bd7 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:24:49.925001  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-7db3e095e8b44b23b1a439899d305bd7 on tablet 7db3e095e8b44b23b1a439899d305bd7 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:50.111936  7335 raft_consensus.cc:1217] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:24:50.203742  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85988.8us
I20260709 12:24:50.209282  7335 raft_consensus.cc:1217] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:24:50.323606  7334 raft_consensus.cc:1064] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:50.326416  7334 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:50.347095  7335 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:50.347131  6451 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:50.349133  7395 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.351476  7395 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.362131  6305 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:50.365715  7395 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.385634  7334 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.392040  6304 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.403867  7333 raft_consensus.cc:1064] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:50.407439  7333 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:50.415335  6451 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.435726  6206 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.477212  7332 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:50.479878  7325 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:50.499562  6451 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:50.501562  7325 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:50.639528  7391 raft_consensus.cc:1064] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:50.642416  7391 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:50.656080  7422 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.677170  6381 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:50.682760  7434 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.683650  7335 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:50.685998  7434 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:50.686929  7262 raft_consensus.cc:1064] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:50.689561  7262 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:50.708853  6381 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:50.728010  6305 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:50.720268  7332 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.740820  7209 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:50.758970  6385 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:50.759011  7331 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:50.760701  7262 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:50.772356  7329 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:50.776034  6305 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:50.788551  7391 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:50.794668  7434 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
W20260709 12:24:50.844753  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 66660.1us
I20260709 12:24:50.783612  7431 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:24:50.911805  6450 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:24:50.959125  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 111607us
I20260709 12:24:51.076131  7209 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.081714  7331 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.117362  6300 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.143168  7462 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.162848  6381 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.219235  7331 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:24:51.222711  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66320.6us
I20260709 12:24:51.229791  6304 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:24:51.249312  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97620.9us
I20260709 12:24:51.271018  6206 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.272822  6206 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.278678  6381 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.279742  6377 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.295776  6385 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:51.302896  7331 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:51.303897  6300 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:51.328526  7467 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:51.335583  7467 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:51.355743  7446 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.371937  7331 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.384137  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2ddf475f266c47adb1a50aa986a0c44c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:51.386284  6304 raft_consensus.cc:2955] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.391103  6204 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.403091  6450 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:51.412562  6377 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:51.415707  7331 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:51.417361  7391 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:51.418509  7209 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:51.421269  6204 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.457814  6450 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:51.462097  7462 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.464463  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 2ddf475f266c47adb1a50aa986a0c44c with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59628
I20260709 12:24:51.470052  6381 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.471596  7335 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:51.473619  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a2f5503246494eed8b91ac10cdf742e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:51.477295  7474 tablet_replica.cc:333] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:24:51.479324  7474 raft_consensus.cc:2243] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:51.479384  6379 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:51.480165  7474 raft_consensus.cc:2272] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:24:51.480294  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53063.9us
I20260709 12:24:51.480520  7434 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:51.481801  7462 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:51.483268  7474 ts_tablet_manager.cc:1916] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:51.492039  7335 raft_consensus.cc:2955] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.492617  7465 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.496353  6304 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:51.500651  7474 ts_tablet_manager.cc:1929] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:24:51.501252  7474 log.cc:1199] T 2ddf475f266c47adb1a50aa986a0c44c P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/2ddf475f266c47adb1a50aa986a0c44c
I20260709 12:24:51.503304  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 2ddf475f266c47adb1a50aa986a0c44c (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:51.507225  7331 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:51.510110  6456 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:51.511067  6379 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.515256  7470 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:51.519198  7420 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:24:51.525450  6202 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:51.537007  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7db3e095e8b44b23b1a439899d305bd7 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:51.546092  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet a2f5503246494eed8b91ac10cdf742e2 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57654
I20260709 12:24:51.547700  7460 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.550076  7331 raft_consensus.cc:2955] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.557512  7332 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.574254  7475 tablet_replica.cc:333] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:24:51.578876  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f24bf2d0bc904aa495cdb5995b470fb0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:51.583410  6450 raft_consensus.cc:2955] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.600112  6206 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:51.613062  7475 raft_consensus.cc:2243] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:51.609735  6201 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.615221  7475 raft_consensus.cc:2272] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:51.625900  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet 7db3e095e8b44b23b1a439899d305bd7 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57654
I20260709 12:24:51.663785  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet f24bf2d0bc904aa495cdb5995b470fb0 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51838
I20260709 12:24:51.658313  7475 ts_tablet_manager.cc:1916] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:51.669129  7476 tablet_replica.cc:333] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:24:51.670347  7476 raft_consensus.cc:2243] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:51.671350  7476 raft_consensus.cc:2272] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:51.677076  7476 ts_tablet_manager.cc:1916] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:51.688305  7475 ts_tablet_manager.cc:1929] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:51.688865  7475 log.cc:1199] T a2f5503246494eed8b91ac10cdf742e2 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/a2f5503246494eed8b91ac10cdf742e2
I20260709 12:24:51.690603  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet a2f5503246494eed8b91ac10cdf742e2 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:51.691687  7475 tablet_replica.cc:333] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:24:51.692818  7475 raft_consensus.cc:2243] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:51.693601  7475 raft_consensus.cc:2272] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:51.696403  7476 ts_tablet_manager.cc:1929] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:24:51.696955  7476 log.cc:1199] T f24bf2d0bc904aa495cdb5995b470fb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/f24bf2d0bc904aa495cdb5995b470fb0
I20260709 12:24:51.697955  7475 ts_tablet_manager.cc:1916] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:51.698835  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet f24bf2d0bc904aa495cdb5995b470fb0 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:51.710747  7475 ts_tablet_manager.cc:1929] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:24:51.711280  7475 log.cc:1199] T 7db3e095e8b44b23b1a439899d305bd7 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/7db3e095e8b44b23b1a439899d305bd7
I20260709 12:24:51.714008  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet 7db3e095e8b44b23b1a439899d305bd7 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:52.759465  6310 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:52.775665  6450 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:52.776437  6275 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f73da77e7bb0404e9f79f06e3dd200f4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:52.777662  6381 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:52.777823  7422 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:52.780046  7481 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:52.797323  7481 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.803515  6450 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.803395  6377 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.817373  6310 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:52.821363  6204 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:24:52.832677  6275 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3b2f25afefb44300a1d6297e2cc140f5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:52.832909  6450 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:52.834707  7481 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:52.835572  6377 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:52.839381  7481 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:52.849500  7422 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.856045  6450 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.866263  6385 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.890390  6385 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:52.910979  6202 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.921921  6450 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:52.923250  6351 consensus_peers.cc:597] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4d797adfa10541918dfc0ebf49b90784. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:52.923974  6310 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:52.927762  7479 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:52.929203  7480 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:52.947228  7478 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.970572  6450 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.975695  6305 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:52.984647  6377 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:52.990475  6202 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:53.007455  6451 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:24:53.008728  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dda9448d18794a44a75d0b570e998c3a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:53.009754  7479 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:53.018316  6305 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:53.049126  7478 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:53.051632  7478 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:53.065177  6450 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:53.090885  6204 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:53.085393  6305 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Committing config change with OpId 4.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:53.346263  7499 ts_tablet_manager.cc:933] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:24:53.352564  7500 ts_tablet_manager.cc:933] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:24:53.355952  7499 tablet_copy_client.cc:323] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:24:53.361430  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 3b2f25afefb44300a1d6297e2cc140f5 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:24:53.362318  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 3b2f25afefb44300a1d6297e2cc140f5 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-3b2f25afefb44300a1d6297e2cc140f5
I20260709 12:24:53.371428  7500 tablet_copy_client.cc:323] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:24:53.373796  6319 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet f73da77e7bb0404e9f79f06e3dd200f4 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:24:53.374689  6319 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet f73da77e7bb0404e9f79f06e3dd200f4 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-f73da77e7bb0404e9f79f06e3dd200f4
I20260709 12:24:53.377470  7503 ts_tablet_manager.cc:933] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:53.386864  6320 tablet_copy_source_session.cc:215] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:53.390734  7499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b2f25afefb44300a1d6297e2cc140f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:53.392602  6319 tablet_copy_source_session.cc:215] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:53.396021  7503 tablet_copy_client.cc:323] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:24:53.397863  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 4d797adfa10541918dfc0ebf49b90784 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:24:53.398473  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 4d797adfa10541918dfc0ebf49b90784 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-4d797adfa10541918dfc0ebf49b90784
I20260709 12:24:53.399039  7500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f73da77e7bb0404e9f79f06e3dd200f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:53.433684  7499 tablet_copy_client.cc:806] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:53.434427  7499 tablet_copy_client.cc:670] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:53.436259  7500 tablet_copy_client.cc:806] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:53.436892  7500 tablet_copy_client.cc:670] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:53.453831  6395 tablet_copy_source_session.cc:215] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:53.466285  7503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d797adfa10541918dfc0ebf49b90784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:53.471503  7500 tablet_copy_client.cc:538] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:53.474572  7499 tablet_copy_client.cc:538] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:53.486106  7503 tablet_copy_client.cc:806] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:53.498922  7503 tablet_copy_client.cc:670] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:53.498159  7499 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:53.490406  7500 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:53.516685  7503 tablet_copy_client.cc:538] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:53.525775  7503 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:53.576625  7503 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.587280  7503 tablet_bootstrap.cc:492] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:53.588122  7503 ts_tablet_manager.cc:1403] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.063s	user 0.022s	sys 0.007s
I20260709 12:24:53.591200  7503 raft_consensus.cc:359] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.592089  7503 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:53.592795  7503 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.612037  7503 ts_tablet_manager.cc:1434] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.000s
I20260709 12:24:53.615361  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4d797adfa10541918dfc0ebf49b90784 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:24:53.615931  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4d797adfa10541918dfc0ebf49b90784 on tablet 4d797adfa10541918dfc0ebf49b90784 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:53.618438  7503 ts_tablet_manager.cc:933] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:24:53.622598  7500 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.623871  7499 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.628036  7499 tablet_bootstrap.cc:492] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:53.629736  7499 ts_tablet_manager.cc:1403] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.141s	user 0.014s	sys 0.018s
I20260709 12:24:53.628796  7500 tablet_bootstrap.cc:492] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:53.636328  7500 ts_tablet_manager.cc:1403] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.146s	user 0.025s	sys 0.009s
I20260709 12:24:53.639346  7503 tablet_copy_client.cc:323] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:24:53.641942  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet dda9448d18794a44a75d0b570e998c3a from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:24:53.642609  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet dda9448d18794a44a75d0b570e998c3a from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-dda9448d18794a44a75d0b570e998c3a
I20260709 12:24:53.648167  7500 raft_consensus.cc:359] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.651561  7500 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:53.650092  7499 raft_consensus.cc:359] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.652268  7500 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.652854  7499 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:53.653663  7499 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.664036  6395 tablet_copy_source_session.cc:215] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:53.668146  7503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dda9448d18794a44a75d0b570e998c3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:53.675565  7499 ts_tablet_manager.cc:1434] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:24:53.688769  7500 ts_tablet_manager.cc:1434] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.003s
I20260709 12:24:53.693485  6319 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-3b2f25afefb44300a1d6297e2cc140f5 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:24:53.693619  7503 tablet_copy_client.cc:806] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:53.693996  6319 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-3b2f25afefb44300a1d6297e2cc140f5 on tablet 3b2f25afefb44300a1d6297e2cc140f5 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:53.694339  7503 tablet_copy_client.cc:670] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:53.698295  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f73da77e7bb0404e9f79f06e3dd200f4 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:24:53.698791  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f73da77e7bb0404e9f79f06e3dd200f4 on tablet f73da77e7bb0404e9f79f06e3dd200f4 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:53.711671  7503 tablet_copy_client.cc:538] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:53.715709  7331 raft_consensus.cc:1217] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:53.733726  7503 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:53.782680  7503 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:53.783725  7503 tablet_bootstrap.cc:492] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:53.785504  7503 ts_tablet_manager.cc:1403] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.052s	user 0.023s	sys 0.007s
I20260709 12:24:53.792527  7503 raft_consensus.cc:359] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.793484  7503 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:53.794198  7503 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:53.807435  7331 raft_consensus.cc:1217] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:24:53.820104  7503 ts_tablet_manager.cc:1434] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.034s	user 0.001s	sys 0.005s
I20260709 12:24:53.822520  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-dda9448d18794a44a75d0b570e998c3a received from {username='slave'} at 127.0.0.1:46610
I20260709 12:24:53.823024  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-dda9448d18794a44a75d0b570e998c3a on tablet dda9448d18794a44a75d0b570e998c3a with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:53.899884  7331 raft_consensus.cc:1217] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
W20260709 12:24:53.988605  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 86293.2us
I20260709 12:24:54.102329  7513 raft_consensus.cc:1064] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:54.105530  7513 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.119518  7331 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:54.121006  6458 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:54.124555  7514 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:54.126468  7514 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:54.134698  6385 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:54.136410  7481 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:54.192762  7332 raft_consensus.cc:1217] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:24:54.205091  7514 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.224934  6377 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.239081  7331 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.266990  6201 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:54.284436  6456 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.399766  6304 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.406710  7331 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:54.410064  7515 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:54.414069  7480 raft_consensus.cc:1064] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:54.416924  7480 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.427616  7460 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.432875  7332 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.434312  6451 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:54.452553  7513 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:54.458999  6451 raft_consensus.cc:2955] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.501227  7514 raft_consensus.cc:1064] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:54.494199  6206 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.509063  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3b2f25afefb44300a1d6297e2cc140f5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:54.504776  7514 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.515775  7331 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:54.518095  6451 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:54.525939  6304 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:54.527999  7511 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:54.540215  7521 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:54.556483  7332 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:54.558372  7514 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:54.558786  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet 3b2f25afefb44300a1d6297e2cc140f5 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51838
I20260709 12:24:54.564095  6456 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:54.570592  7514 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:54.576333  7521 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:54.584643  7530 tablet_replica.cc:333] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:24:54.591419  7480 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.591512  7530 raft_consensus.cc:2243] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:54.594758  7530 raft_consensus.cc:2272] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:54.598569  7530 ts_tablet_manager.cc:1916] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:54.603698  6377 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:54.618877  7460 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:54.636716  7331 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.639730  7530 ts_tablet_manager.cc:1929] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:54.640322  7530 log.cc:1199] T 3b2f25afefb44300a1d6297e2cc140f5 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/3b2f25afefb44300a1d6297e2cc140f5
I20260709 12:24:54.642652  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet 3b2f25afefb44300a1d6297e2cc140f5 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:54.637142  6456 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.668146  6206 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.686136  6303 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.705221  6385 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.714510  7516 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.727288  7521 raft_consensus.cc:1064] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:54.727964  6451 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:54.729722  7478 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:54.730165  7521 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 2, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.737771  7331 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:54.743320  7480 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.750551  6456 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:54.755280  6304 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:54.751641  7331 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:54.759321  6458 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.766016  6451 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.789816  6202 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.798957  7479 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:54.809062  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet 4d797adfa10541918dfc0ebf49b90784 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57654
W20260709 12:24:54.813792  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56197.1us
I20260709 12:24:54.821668  7478 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:54.823554  7478 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:54.832270  7538 tablet_replica.cc:333] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:24:54.834161  7538 raft_consensus.cc:2243] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:54.845605  6202 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:24:54.850140  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51089us
I20260709 12:24:54.851321  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4d797adfa10541918dfc0ebf49b90784 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:54.867468  7538 raft_consensus.cc:2272] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:54.873883  7538 ts_tablet_manager.cc:1916] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:54.888000  6381 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:54.896034  7331 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:24:54.958065  7539 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:24:54.959066  7538 ts_tablet_manager.cc:1929] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:24:54.967911  7538 log.cc:1199] T 4d797adfa10541918dfc0ebf49b90784 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/4d797adfa10541918dfc0ebf49b90784
I20260709 12:24:54.969871  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet 4d797adfa10541918dfc0ebf49b90784 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:54.973762  6303 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:54.989355  7331 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:54.992512  7513 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:55.002234  7460 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.005187  7331 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:24:55.020246  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64091.5us
I20260709 12:24:55.021871  7480 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:55.024749  7332 raft_consensus.cc:2955] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.102239  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f73da77e7bb0404e9f79f06e3dd200f4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:55.113106  6202 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.119246  6381 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:55.131788  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet f73da77e7bb0404e9f79f06e3dd200f4 with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59628
I20260709 12:24:55.138413  7541 tablet_replica.cc:333] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:24:55.149639  7541 raft_consensus.cc:2243] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:55.150502  7541 raft_consensus.cc:2272] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:55.159162  7543 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:55.154956  7541 ts_tablet_manager.cc:1916] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:55.192272  7511 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.207823  7541 ts_tablet_manager.cc:1929] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:24:55.208630  7541 log.cc:1199] T f73da77e7bb0404e9f79f06e3dd200f4 P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/f73da77e7bb0404e9f79f06e3dd200f4
I20260709 12:24:55.210548  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet f73da77e7bb0404e9f79f06e3dd200f4 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:55.215179  6458 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.229517  6304 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.230238  6381 raft_consensus.cc:2955] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.242254  7331 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.285372  6204 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.333863  6381 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:55.343240  7331 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:55.348052  6304 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:55.354247  7480 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:55.356356  7480 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:55.374620  7537 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.380514  7331 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.381843  6304 raft_consensus.cc:2955] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.403843  6204 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:55.425590  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dda9448d18794a44a75d0b570e998c3a with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:55.452551  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet dda9448d18794a44a75d0b570e998c3a with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:59628
I20260709 12:24:55.453634  7541 tablet_replica.cc:333] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:24:55.454741  7541 raft_consensus.cc:2243] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:55.455617  7541 raft_consensus.cc:2272] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:55.459376  7541 ts_tablet_manager.cc:1916] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:55.493641  7541 ts_tablet_manager.cc:1929] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.5
I20260709 12:24:55.494031  7541 log.cc:1199] T dda9448d18794a44a75d0b570e998c3a P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/dda9448d18794a44a75d0b570e998c3a
I20260709 12:24:55.495441  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet dda9448d18794a44a75d0b570e998c3a (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:56.554200  6458 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:24:56.565202  6425 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cd67024481794a7db3a98df0cd400c9d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:56.566025  6379 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:56.618623  6304 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:56.621130  7545 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:24:56.630326  7465 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:56.703290  6303 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.699587  7549 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.719247  6379 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.742332  6202 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.789338  6450 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:56.823976  6381 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:56.826634  7549 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:56.848259  6303 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:24:56.850039  6425 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 55b778a085744d27a1adc35598438230. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:56.852440  7556 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:56.861339  7545 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.886621  6377 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.890687  6458 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:56.913440  6303 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.922338  6377 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:24:56.925624  6425 consensus_peers.cc:597] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4a0ddc1149fd4b03af2abd36909591d0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:56.928545  6304 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:56.929081  7555 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:56.931730  7462 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:56.958230  7549 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.970762  6377 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.975524  6204 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:56.979756  6303 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:57.007313  6305 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.014356  6206 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:57.024923  6458 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:57.025175  6275 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f745d20b9f8c4aca9adf20ed14f509ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:57.026641  7548 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:57.029798  6377 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:57.032625  7515 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:57.043049  6458 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:57.045282  6377 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:57.041127  7515 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:57.061267  6202 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:57.121660  7568 ts_tablet_manager.cc:933] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:57.130481  7568 tablet_copy_client.cc:323] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:24:57.132937  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet cd67024481794a7db3a98df0cd400c9d from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:24:57.133626  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet cd67024481794a7db3a98df0cd400c9d from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-cd67024481794a7db3a98df0cd400c9d
I20260709 12:24:57.139424  6469 tablet_copy_source_session.cc:215] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:57.142897  7568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd67024481794a7db3a98df0cd400c9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:57.160430  7568 tablet_copy_client.cc:806] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:57.161116  7568 tablet_copy_client.cc:670] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:57.165375  7568 tablet_copy_client.cc:538] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:57.172886  7568 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:57.205655  7568 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:57.206873  7568 tablet_bootstrap.cc:492] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:57.207710  7568 ts_tablet_manager.cc:1403] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.006s
I20260709 12:24:57.211273  7568 raft_consensus.cc:359] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.212267  7568 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:57.212986  7568 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.216431  7568 ts_tablet_manager.cc:1434] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.007s
I20260709 12:24:57.218937  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-cd67024481794a7db3a98df0cd400c9d received from {username='slave'} at 127.0.0.1:38272
I20260709 12:24:57.219437  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-cd67024481794a7db3a98df0cd400c9d on tablet cd67024481794a7db3a98df0cd400c9d with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:57.240959  7568 ts_tablet_manager.cc:933] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:57.243120  7568 tablet_copy_client.cc:323] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:24:57.245472  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 55b778a085744d27a1adc35598438230 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:24:57.246052  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 55b778a085744d27a1adc35598438230 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-55b778a085744d27a1adc35598438230
I20260709 12:24:57.251971  6469 tablet_copy_source_session.cc:215] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:57.256173  7568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55b778a085744d27a1adc35598438230. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:57.271528  7568 tablet_copy_client.cc:806] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:57.272156  7568 tablet_copy_client.cc:670] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:57.275980  7568 tablet_copy_client.cc:538] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:57.286562  7568 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:57.306381  7568 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:57.307336  7568 tablet_bootstrap.cc:492] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:57.308008  7568 ts_tablet_manager.cc:1403] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.005s
I20260709 12:24:57.311796  7568 raft_consensus.cc:359] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.312667  7568 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:57.313232  7568 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.315368  7568 ts_tablet_manager.cc:1434] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.006s
I20260709 12:24:57.317282  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-55b778a085744d27a1adc35598438230 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:24:57.317703  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-55b778a085744d27a1adc35598438230 on tablet 55b778a085744d27a1adc35598438230 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:57.425261  7568 ts_tablet_manager.cc:933] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:24:57.427268  7568 tablet_copy_client.cc:323] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:24:57.428922  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 4a0ddc1149fd4b03af2abd36909591d0 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:24:57.429471  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 4a0ddc1149fd4b03af2abd36909591d0 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-4a0ddc1149fd4b03af2abd36909591d0
I20260709 12:24:57.437741  6469 tablet_copy_source_session.cc:215] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:57.443372  7568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a0ddc1149fd4b03af2abd36909591d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:57.467020  7568 tablet_copy_client.cc:806] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:57.467546  7568 tablet_copy_client.cc:670] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:57.471343  7568 tablet_copy_client.cc:538] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:57.478315  7568 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:57.503420  7568 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:57.504355  7568 tablet_bootstrap.cc:492] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:57.504937  7568 ts_tablet_manager.cc:1403] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.007s
I20260709 12:24:57.508268  7568 raft_consensus.cc:359] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.509042  7568 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:57.509644  7568 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.511823  7568 ts_tablet_manager.cc:1434] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:24:57.513593  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4a0ddc1149fd4b03af2abd36909591d0 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:24:57.513965  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4a0ddc1149fd4b03af2abd36909591d0 on tablet 4a0ddc1149fd4b03af2abd36909591d0 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:57.647218  7568 ts_tablet_manager.cc:933] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:24:57.648906  7568 tablet_copy_client.cc:323] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:24:57.650868  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet f745d20b9f8c4aca9adf20ed14f509ac from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:24:57.651605  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet f745d20b9f8c4aca9adf20ed14f509ac from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-f745d20b9f8c4aca9adf20ed14f509ac
I20260709 12:24:57.661103  6320 tablet_copy_source_session.cc:215] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:24:57.664582  7568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f745d20b9f8c4aca9adf20ed14f509ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:57.680156  7568 tablet_copy_client.cc:806] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:57.680685  7568 tablet_copy_client.cc:670] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:57.687873  7568 tablet_copy_client.cc:538] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:57.694309  7568 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:24:57.720399  7568 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:57.721387  7568 tablet_bootstrap.cc:492] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:24:57.722074  7568 ts_tablet_manager.cc:1403] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.009s
I20260709 12:24:57.724967  7568 raft_consensus.cc:359] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.725780  7568 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:24:57.726380  7568 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:57.729249  7568 ts_tablet_manager.cc:1434] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:24:57.734205  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f745d20b9f8c4aca9adf20ed14f509ac received from {username='slave'} at 127.0.0.1:53970
I20260709 12:24:57.734680  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f745d20b9f8c4aca9adf20ed14f509ac on tablet f745d20b9f8c4aca9adf20ed14f509ac with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:24:57.738176  7335 raft_consensus.cc:1217] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:24:57.761473  7335 raft_consensus.cc:1217] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:24:57.817718  7335 raft_consensus.cc:1217] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:24:58.137718  7556 raft_consensus.cc:1064] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:58.139966  7556 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.146921  7335 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:58.147814  6305 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:58.146978  6379 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:58.149159  7556 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.150131  7555 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.154587  7335 raft_consensus.cc:1217] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:24:58.156752  7549 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:58.177275  7549 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.182421  6305 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.192612  7331 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.206010  6204 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:58.229743  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53457us
I20260709 12:24:58.232406  6377 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.241264  7584 raft_consensus.cc:1064] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:58.241607  7582 raft_consensus.cc:1064] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:58.245687  7584 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.256997  7582 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.282220  6377 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:58.282316  7331 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:58.283878  7584 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:58.284090  6303 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:24:58.289183  7586 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:58.297923  7586 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:58.310436  6379 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:58.313306  7332 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:58.315881  7549 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.317739  7582 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:58.325778  6303 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:58.335754  7549 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:24:58.360070  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47470.6us
I20260709 12:24:58.392367  7584 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.396988  7332 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.406107  7331 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.399392  6305 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.412277  7586 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.415102  6456 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.416651  6204 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.441462  6377 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.467150  7331 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:58.466271  6201 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.470474  6305 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.472147  6303 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:58.460819  6379 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.495139  7586 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:58.498041  7584 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:58.499675  6456 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.509688  7586 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.513513  6379 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:58.524487  7331 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:24:58.525678  7549 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.528369  7585 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.535593  7549 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.540848  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 55b778a085744d27a1adc35598438230 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:58.544008  6379 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.549058  7332 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.549885  6305 raft_consensus.cc:2955] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.555119  7331 raft_consensus.cc:2955] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.560791  6456 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.563453  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4a0ddc1149fd4b03af2abd36909591d0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:58.572335  6201 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.588739  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet 55b778a085744d27a1adc35598438230 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51838
I20260709 12:24:58.595182  6305 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:58.596206  6201 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.598774  7331 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:58.606477  7586 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:58.607800  7585 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:58.602123  7600 tablet_replica.cc:333] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:24:58.634238  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet 4a0ddc1149fd4b03af2abd36909591d0 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57654
I20260709 12:24:58.640611  7600 raft_consensus.cc:2243] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.644028  7549 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.646639  6303 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.647578  7600 raft_consensus.cc:2272] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.652537  7600 ts_tablet_manager.cc:1916] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:58.655858  7601 tablet_replica.cc:333] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:24:58.662683  7332 raft_consensus.cc:2955] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.675208  6204 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.677989  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cd67024481794a7db3a98df0cd400c9d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:58.678592  7601 raft_consensus.cc:2243] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.679555  7601 raft_consensus.cc:2272] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.699959  7515 raft_consensus.cc:1064] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:24:58.700378  7601 ts_tablet_manager.cc:1916] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:58.703634  7515 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.710134  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet cd67024481794a7db3a98df0cd400c9d with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51838
I20260709 12:24:58.721454  7332 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:58.724335  7543 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.727866  6456 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:58.729321  7543 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.729575  6379 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:58.731524  7600 ts_tablet_manager.cc:1929] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:24:58.732079  7600 log.cc:1199] T 55b778a085744d27a1adc35598438230 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/55b778a085744d27a1adc35598438230
I20260709 12:24:58.732546  7515 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.734025  7600 tablet_replica.cc:333] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:24:58.733901  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet 55b778a085744d27a1adc35598438230 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:58.735363  7600 raft_consensus.cc:2243] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.736339  7600 raft_consensus.cc:2272] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.740689  7600 ts_tablet_manager.cc:1916] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:58.752583  7601 ts_tablet_manager.cc:1929] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:24:58.759039  7593 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.765717  7601 log.cc:1199] T 4a0ddc1149fd4b03af2abd36909591d0 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/4a0ddc1149fd4b03af2abd36909591d0
I20260709 12:24:58.764799  6450 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.760686  6379 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.768011  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet 4a0ddc1149fd4b03af2abd36909591d0 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:58.765141  7600 ts_tablet_manager.cc:1929] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:24:58.778594  7332 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.779474  7600 log.cc:1199] T cd67024481794a7db3a98df0cd400c9d P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/cd67024481794a7db3a98df0cd400c9d
I20260709 12:24:58.782591  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet cd67024481794a7db3a98df0cd400c9d (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:58.785554  6201 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.812606  6303 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:24:58.817411  7332 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:58.818943  7593 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:58.822542  6379 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:58.824856  7543 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:58.828146  7543 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.829816  7332 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.838297  6381 raft_consensus.cc:2955] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.845319  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f745d20b9f8c4aca9adf20ed14f509ac with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:24:58.846134  6201 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:24:58.865753  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet f745d20b9f8c4aca9adf20ed14f509ac with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59628
I20260709 12:24:58.883306  7606 tablet_replica.cc:333] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:24:58.884132  7606 raft_consensus.cc:2243] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:24:58.884824  7606 raft_consensus.cc:2272] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:24:58.887573  7606 ts_tablet_manager.cc:1916] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:24:58.902810  7606 ts_tablet_manager.cc:1929] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:24:58.903228  7606 log.cc:1199] T f745d20b9f8c4aca9adf20ed14f509ac P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/f745d20b9f8c4aca9adf20ed14f509ac
I20260709 12:24:58.905138  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet f745d20b9f8c4aca9adf20ed14f509ac (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:24:59.947506  6305 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:24:59.953251  6275 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 150319986ee143d5a746f901c5b4adcc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:59.955049  6456 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:59.957197  7460 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:59.957583  6381 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:24:59.963330  7460 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:59.974107  7610 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:59.979786  6456 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:59.982796  6381 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:24:59.993119  6450 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:24:59.999312  6201 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:00.009977  6381 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:00.010871  6303 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:25:00.010967  6425 consensus_peers.cc:597] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34bae875d7104f298673c223da6caf2e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:00.012116  7585 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.013124  7549 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.026273  7585 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.027383  6381 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.030572  6303 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.048225  6381 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.057210  6202 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:00.062799  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 034bd5c36e71466da14c6a4850a920cb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:00.064244  6456 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:00.064843  6303 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:00.065743  7550 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.066936  7511 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.084131  7511 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.085964  6456 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.115029  6303 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.143611  6201 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.146018  6303 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.155501  6456 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:00.155859  6275 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1b749ec31e454255bbbba4a4bb493f28. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:00.158229  6381 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:00.158641  7460 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.160578  7543 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:00.176750  7543 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.178969  6381 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.182958  6456 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.203063  6202 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:00.430859  7630 ts_tablet_manager.cc:933] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:00.432868  7630 tablet_copy_client.cc:323] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:25:00.434688  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 34bae875d7104f298673c223da6caf2e from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:25:00.435307  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 34bae875d7104f298673c223da6caf2e from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-34bae875d7104f298673c223da6caf2e
I20260709 12:25:00.448604  7632 ts_tablet_manager.cc:933] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:00.453172  7632 tablet_copy_client.cc:323] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:25:00.454277  6469 tablet_copy_source_session.cc:215] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:00.455355  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 034bd5c36e71466da14c6a4850a920cb from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:25:00.455963  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 034bd5c36e71466da14c6a4850a920cb from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-034bd5c36e71466da14c6a4850a920cb
I20260709 12:25:00.458070  7630 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 34bae875d7104f298673c223da6caf2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:00.476279  7630 tablet_copy_client.cc:806] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:00.476954  7630 tablet_copy_client.cc:670] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:00.482594  6395 tablet_copy_source_session.cc:215] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:00.484463  7630 tablet_copy_client.cc:538] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:00.486969  7632 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 034bd5c36e71466da14c6a4850a920cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:00.514040  7630 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:00.515797  7634 ts_tablet_manager.cc:933] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:00.523428  7632 tablet_copy_client.cc:806] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:00.524137  7632 tablet_copy_client.cc:670] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:00.526512  7634 tablet_copy_client.cc:323] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:00.528445  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 150319986ee143d5a746f901c5b4adcc from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:00.528605  7632 tablet_copy_client.cc:538] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:00.529055  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 150319986ee143d5a746f901c5b4adcc from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-150319986ee143d5a746f901c5b4adcc
I20260709 12:25:00.538678  7632 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:00.569348  6320 tablet_copy_source_session.cc:215] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:00.580407  7632 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:00.581424  7632 tablet_bootstrap.cc:492] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:00.582104  7632 ts_tablet_manager.cc:1403] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.044s	user 0.020s	sys 0.011s
I20260709 12:25:00.583576  7634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 150319986ee143d5a746f901c5b4adcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:00.585100  7632 raft_consensus.cc:359] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.585997  7632 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:00.586714  7632 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.585991  7630 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:00.592388  7630 tablet_bootstrap.cc:492] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:00.593253  7630 ts_tablet_manager.cc:1403] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.079s	user 0.028s	sys 0.009s
I20260709 12:25:00.597167  7632 ts_tablet_manager.cc:1434] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.007s
I20260709 12:25:00.596735  7630 raft_consensus.cc:359] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.597822  7630 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:00.599310  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-034bd5c36e71466da14c6a4850a920cb received from {username='slave'} at 127.0.0.1:46610
I20260709 12:25:00.602897  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-034bd5c36e71466da14c6a4850a920cb on tablet 034bd5c36e71466da14c6a4850a920cb with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:00.603093  7630 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.607228  7630 ts_tablet_manager.cc:1434] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:25:00.609377  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-34bae875d7104f298673c223da6caf2e received from {username='slave'} at 127.0.0.1:38272
I20260709 12:25:00.609833  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-34bae875d7104f298673c223da6caf2e on tablet 34bae875d7104f298673c223da6caf2e with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:00.615188  7634 tablet_copy_client.cc:806] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:00.617262  7634 tablet_copy_client.cc:670] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:00.622526  7634 tablet_copy_client.cc:538] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:00.633666  7634 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:00.655562  7634 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:00.656510  7634 tablet_bootstrap.cc:492] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:00.657213  7634 ts_tablet_manager.cc:1403] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.014s
I20260709 12:25:00.660362  7634 raft_consensus.cc:359] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.661188  7634 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:00.661829  7634 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.665027  7634 ts_tablet_manager.cc:1434] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:25:00.672408  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-150319986ee143d5a746f901c5b4adcc received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:00.672883  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-150319986ee143d5a746f901c5b4adcc on tablet 150319986ee143d5a746f901c5b4adcc with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:00.775314  7634 ts_tablet_manager.cc:933] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:00.802702  7634 tablet_copy_client.cc:323] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:00.804435  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 1b749ec31e454255bbbba4a4bb493f28 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:00.804945  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 1b749ec31e454255bbbba4a4bb493f28 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-1b749ec31e454255bbbba4a4bb493f28
I20260709 12:25:00.811558  6320 tablet_copy_source_session.cc:215] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:00.819222  7634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b749ec31e454255bbbba4a4bb493f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:00.840732  7634 tablet_copy_client.cc:806] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:00.841380  7634 tablet_copy_client.cc:670] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:00.845474  7634 tablet_copy_client.cc:538] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:00.853087  7634 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:00.873687  7332 raft_consensus.cc:1217] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:25:00.877426  7634 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:00.878541  7634 tablet_bootstrap.cc:492] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:00.881512  7634 ts_tablet_manager.cc:1403] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.009s
I20260709 12:25:00.884887  7634 raft_consensus.cc:359] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.885792  7634 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:00.886507  7634 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:00.889461  7634 ts_tablet_manager.cc:1434] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:25:00.893608  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-1b749ec31e454255bbbba4a4bb493f28 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:00.894088  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-1b749ec31e454255bbbba4a4bb493f28 on tablet 1b749ec31e454255bbbba4a4bb493f28 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:00.940232  7332 raft_consensus.cc:1217] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:25:00.993626  7331 raft_consensus.cc:1217] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:01.300280  7642 raft_consensus.cc:1064] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:01.302954  7642 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.316704  7331 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:01.318691  6305 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:01.318692  6379 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:01.320075  7643 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.321000  7642 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.321831  7549 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.354398  7549 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.356950  6305 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.365435  7331 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.375089  6379 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.377630  7332 raft_consensus.cc:1217] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:25:01.377350  6204 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:01.444888  7610 raft_consensus.cc:1064] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:01.448734  7610 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.464732  7626 raft_consensus.cc:1064] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:01.469588  7626 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.494212  7331 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.496003  7543 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:01.503082  7335 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:01.504915  6305 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:01.506996  7511 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.508436  7656 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.517720  6450 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.523597  7652 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.553268  6451 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:01.557934  6379 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:01.566859  7543 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:01.592024  6379 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.589830  7610 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.606190  6456 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.608552  7651 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.641839  6450 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.664212  6204 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.677500  7331 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.681736  6377 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:01.683874  7549 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.694933  7335 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:01.697228  7643 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.698438  6377 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.711357  7626 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.718439  6303 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.722685  6451 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.736682  7666 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:01.744939  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 34bae875d7104f298673c223da6caf2e with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:01.743031  6305 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.763195  7335 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.771803  6202 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.776423  7332 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:01.779213  7664 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.781922  6454 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:01.784662  6204 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.786664  7331 raft_consensus.cc:2955] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.802047  7664 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.804419  7543 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.805481  7332 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.809844  7610 raft_consensus.cc:1064] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:01.816668  7610 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.820725  6451 raft_consensus.cc:2955] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.823601  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet 34bae875d7104f298673c223da6caf2e with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57654
I20260709 12:25:01.830430  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 150319986ee143d5a746f901c5b4adcc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:01.838166  6206 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:01.874430  7332 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:01.875965  7460 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.904507  7668 tablet_replica.cc:333] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:25:01.905234  6377 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:01.906004  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet 150319986ee143d5a746f901c5b4adcc with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51838
I20260709 12:25:01.908996  7668 raft_consensus.cc:2243] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:01.914202  7668 raft_consensus.cc:2272] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:01.922785  7335 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:01.923118  6305 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:01.924638  7671 tablet_replica.cc:333] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:25:01.924834  7668 ts_tablet_manager.cc:1916] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:01.926966  6377 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:01.927386  7656 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.931424  7511 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:01.934752  7664 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:01.938737  6451 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:01.937085  7671 raft_consensus.cc:2243] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:01.949488  7671 raft_consensus.cc:2272] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:01.953764  7671 ts_tablet_manager.cc:1916] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:01.954806  7610 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:01.956471  7668 ts_tablet_manager.cc:1929] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:25:01.956923  7668 log.cc:1199] T 34bae875d7104f298673c223da6caf2e P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/34bae875d7104f298673c223da6caf2e
I20260709 12:25:01.958647  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet 34bae875d7104f298673c223da6caf2e (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:01.963711  7626 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.968979  6305 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.976785  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 034bd5c36e71466da14c6a4850a920cb with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:01.978349  7335 raft_consensus.cc:2955] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:01.996153  6206 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:01.995013  7664 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.005177  7671 ts_tablet_manager.cc:1929] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:02.005961  7671 log.cc:1199] T 150319986ee143d5a746f901c5b4adcc P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/150319986ee143d5a746f901c5b4adcc
I20260709 12:25:02.007843  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet 150319986ee143d5a746f901c5b4adcc (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:02.016570  7335 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.007977  6451 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.024046  6201 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.038004  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 034bd5c36e71466da14c6a4850a920cb with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59628
I20260709 12:25:02.045662  6377 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.064462  7674 tablet_replica.cc:333] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:25:02.072666  7674 raft_consensus.cc:2243] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:02.073647  7674 raft_consensus.cc:2272] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:02.078454  7674 ts_tablet_manager.cc:1916] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:02.078619  6305 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:02.090436  7335 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:02.091990  7664 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:02.092253  6377 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:02.094285  7664 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:02.100042  7610 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.105484  7335 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.114050  6381 raft_consensus.cc:2955] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.115000  7674 ts_tablet_manager.cc:1929] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:25:02.115533  7674 log.cc:1199] T 034bd5c36e71466da14c6a4850a920cb P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/034bd5c36e71466da14c6a4850a920cb
I20260709 12:25:02.117421  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 034bd5c36e71466da14c6a4850a920cb (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:02.123409  6201 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:02.134967  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1b749ec31e454255bbbba4a4bb493f28 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:02.149277  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 1b749ec31e454255bbbba4a4bb493f28 with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:59628
I20260709 12:25:02.150262  7674 tablet_replica.cc:333] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:25:02.151326  7674 raft_consensus.cc:2243] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:02.151981  7674 raft_consensus.cc:2272] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:02.158205  7674 ts_tablet_manager.cc:1916] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:02.175362  7674 ts_tablet_manager.cc:1929] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:25:02.175799  7674 log.cc:1199] T 1b749ec31e454255bbbba4a4bb493f28 P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/1b749ec31e454255bbbba4a4bb493f28
I20260709 12:25:02.177244  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 1b749ec31e454255bbbba4a4bb493f28 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:03.334214  7687 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:03.334436  7682 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:03.334856  7687 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } }
I20260709 12:25:03.334977  7682 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:03.337724  7682 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:03.340380  7687 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:03.354317  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:03.356263  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:03.370900  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:03.371166  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:03.381508  6274 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:03.382426  7652 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.382875  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:03.386265  7626 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:03.386900  7626 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:03.388593  7652 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:03.390121  7626 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:03.396114  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:03.413595  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:03.426252  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:03.427170  7626 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:03.433383  6304 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:03.441311  6451 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:03.441643  6381 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:03.442204  6275 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f0250a69b3914cb38265a7457b73a158. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:03.442668  7652 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:03.443642  7682 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:03.453567  7687 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.455426  6451 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.455514  6381 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.470342  6451 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:03.473562  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.480674  6381 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:25:03.481022  6425 consensus_peers.cc:597] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c2d3ed2ffee2408ebcb407323a99660b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:03.482204  7691 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:03.482460  6304 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:03.484809  7691 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:03.494618  7692 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.496752  6381 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.497088  6304 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.511303  6377 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:03.514206  6206 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:25:03.523952  6351 consensus_peers.cc:597] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8729b74dbaf94d7092d811f0a55205f2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:03.528301  6451 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:03.530019  6304 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:03.529973  7626 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:03.532902  7675 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:03.545061  7686 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.546702  6304 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.556522  6451 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.558921  6377 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:03.561864  6201 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:03.577348  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7878a824e1e94db6a0b83ab47738f4b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:03.578779  6451 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:03.579548  6304 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:03.580440  7686 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:03.581487  7626 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:03.594780  7626 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.596678  6451 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.598239  6304 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Committing config change with OpId 5.4: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.620036  6206 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:03.985858  7709 ts_tablet_manager.cc:933] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:03.987686  7709 tablet_copy_client.cc:323] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:03.989287  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet f0250a69b3914cb38265a7457b73a158 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:03.989804  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet f0250a69b3914cb38265a7457b73a158 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-f0250a69b3914cb38265a7457b73a158
I20260709 12:25:03.996420  6320 tablet_copy_source_session.cc:215] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:03.999429  7709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0250a69b3914cb38265a7457b73a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:04.011231  7709 tablet_copy_client.cc:806] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:04.011763  7709 tablet_copy_client.cc:670] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:04.015887  7709 tablet_copy_client.cc:538] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:04.024227  7709 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:04.042315  7709 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:04.043124  7709 tablet_bootstrap.cc:492] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:04.043805  7709 ts_tablet_manager.cc:1403] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.010s
I20260709 12:25:04.046809  7709 raft_consensus.cc:359] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.047699  7709 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:04.048409  7709 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.051671  7709 ts_tablet_manager.cc:1434] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:25:04.053339  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f0250a69b3914cb38265a7457b73a158 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:04.053820  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f0250a69b3914cb38265a7457b73a158 on tablet f0250a69b3914cb38265a7457b73a158 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:04.061013  7713 ts_tablet_manager.cc:933] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:04.063092  7713 tablet_copy_client.cc:323] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:25:04.065460  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 8729b74dbaf94d7092d811f0a55205f2 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:25:04.066059  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 8729b74dbaf94d7092d811f0a55205f2 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-8729b74dbaf94d7092d811f0a55205f2
I20260709 12:25:04.075006  6395 tablet_copy_source_session.cc:215] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:04.079675  7713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8729b74dbaf94d7092d811f0a55205f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:04.095702  7713 tablet_copy_client.cc:806] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:04.096293  7713 tablet_copy_client.cc:670] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:04.097734  7709 ts_tablet_manager.cc:933] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:04.100155  7709 tablet_copy_client.cc:323] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:25:04.101712  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet c2d3ed2ffee2408ebcb407323a99660b from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:25:04.102250  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet c2d3ed2ffee2408ebcb407323a99660b from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-c2d3ed2ffee2408ebcb407323a99660b
I20260709 12:25:04.102766  7713 tablet_copy_client.cc:538] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:04.114629  7716 ts_tablet_manager.cc:933] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:04.117372  6469 tablet_copy_source_session.cc:215] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:04.130892  7709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c2d3ed2ffee2408ebcb407323a99660b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:04.140137  7713 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:04.168479  7716 tablet_copy_client.cc:323] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:25:04.169210  7709 tablet_copy_client.cc:806] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:04.169894  7709 tablet_copy_client.cc:670] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:04.171356  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 7878a824e1e94db6a0b83ab47738f4b0 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:25:04.171967  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 7878a824e1e94db6a0b83ab47738f4b0 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-7878a824e1e94db6a0b83ab47738f4b0
I20260709 12:25:04.180701  6395 tablet_copy_source_session.cc:215] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:04.185734  7716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7878a824e1e94db6a0b83ab47738f4b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:25:04.204559  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7a80 after lost signal to thread 5140
W20260709 12:25:04.205737  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08000c0aa0 after lost signal to thread 6240
W20260709 12:25:04.206970  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080029ef00 after lost signal to thread 6334
W20260709 12:25:04.208032  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080020b200 after lost signal to thread 6409
W20260709 12:25:04.209088  6480 debug-util.cc:398] Leaking SignalData structure 0x7b0800325020 after lost signal to thread 6483
W20260709 12:25:04.210156  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08002be6e0 after lost signal to thread 7371
I20260709 12:25:04.212462  7716 tablet_copy_client.cc:806] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:04.213246  7716 tablet_copy_client.cc:670] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:04.217180  7716 tablet_copy_client.cc:538] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:04.219455  7709 tablet_copy_client.cc:538] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:04.232988  7709 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:04.233325  7716 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:04.254063  7713 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:04.260957  7713 tablet_bootstrap.cc:492] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:04.267083  7713 ts_tablet_manager.cc:1403] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.016s
I20260709 12:25:04.286455  7713 raft_consensus.cc:359] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.288295  7716 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:04.289208  7716 tablet_bootstrap.cc:492] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:04.289862  7716 ts_tablet_manager.cc:1403] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.057s	user 0.029s	sys 0.009s
I20260709 12:25:04.293884  7716 raft_consensus.cc:359] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.294678  7716 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:04.295372  7716 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.298899  7716 ts_tablet_manager.cc:1434] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:25:04.302179  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-7878a824e1e94db6a0b83ab47738f4b0 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:25:04.302762  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-7878a824e1e94db6a0b83ab47738f4b0 on tablet 7878a824e1e94db6a0b83ab47738f4b0 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:04.287343  7713 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:04.310840  7713 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.314838  7713 ts_tablet_manager.cc:1434] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.047s	user 0.002s	sys 0.005s
I20260709 12:25:04.320993  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-8729b74dbaf94d7092d811f0a55205f2 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:25:04.321484  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-8729b74dbaf94d7092d811f0a55205f2 on tablet 8729b74dbaf94d7092d811f0a55205f2 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:04.328181  7709 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:04.329154  7709 tablet_bootstrap.cc:492] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:04.329833  7709 ts_tablet_manager.cc:1403] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.097s	user 0.030s	sys 0.012s
I20260709 12:25:04.332719  7709 raft_consensus.cc:359] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.333521  7709 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:04.334129  7709 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:04.336651  7709 ts_tablet_manager.cc:1434] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:25:04.338653  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-c2d3ed2ffee2408ebcb407323a99660b received from {username='slave'} at 127.0.0.1:38272
I20260709 12:25:04.339087  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-c2d3ed2ffee2408ebcb407323a99660b on tablet c2d3ed2ffee2408ebcb407323a99660b with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:04.480058  7331 raft_consensus.cc:1217] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:25:04.496345  7331 raft_consensus.cc:1217] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEARNER]: Deduplicated request from leader. Original: 5.3->[5.4-5.4]   Dedup: 5.4->[]
I20260709 12:25:04.515599  7331 raft_consensus.cc:1217] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:25:04.601317  7331 raft_consensus.cc:1217] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:04.936609  7722 raft_consensus.cc:1064] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:04.944794  7722 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:04.952497  7331 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:04.954062  7721 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:04.973973  6385 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:04.980686  6305 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:04.986773  7722 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:05.014467  7721 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:25:05.019941  7610 raft_consensus.cc:1064] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:05.035447  7610 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:05.039605  7724 raft_consensus.cc:1064] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:05.096283  7724 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 2, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:05.107191  7722 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.122709  7335 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.142473  6201 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:05.151949  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68958.8us
I20260709 12:25:05.160086  6305 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.163126  7736 raft_consensus.cc:1064] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:05.166326  7332 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:05.172992  7610 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:05.186134  7335 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:05.195410  6377 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:05.197676  6305 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:05.199308  7724 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:05.201385  7736 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:05.218354  6454 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:05.223366  7687 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:05.228993  7724 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:05.232837  6456 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:05.244076  7332 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:05.245996  6305 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:25:05.256587  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82429.2us
I20260709 12:25:05.270439  7687 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:05.277678  6385 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.284996  6456 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:05.261896  6454 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:05.307372  7738 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:05.323393  7329 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:05.321969  7730 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:25:05.325342  7721 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:05.335469  7610 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.342789  7692 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.349012  7329 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.366807  7689 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:05.377837  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58953us
I20260709 12:25:05.392627  7736 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.393947  6451 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.382210  6206 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 5 to 6, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.403355  7329 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.416172  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.459424  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet c2d3ed2ffee2408ebcb407323a99660b with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:57654
I20260709 12:25:05.449358  6376 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:25:05.391021  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71372.3us
I20260709 12:25:05.484414  6204 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.494320  6305 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.492259  7689 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.499305  6310 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.502358  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c2d3ed2ffee2408ebcb407323a99660b with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.498165  7720 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:25:05.533638  7329 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.554600  7753 tablet_replica.cc:333] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:25:05.561774  7753 raft_consensus.cc:2243] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.561020  7332 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.559785  6201 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.563107  6377 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:05.564751  7753 raft_consensus.cc:2272] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.583135  7753 ts_tablet_manager.cc:1916] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.596675  7751 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:05.607146  6454 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.622107  6310 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:05.628690  6377 raft_consensus.cc:2955] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.637293  6451 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:05.643283  6450 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 4 to 5, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.647488  7682 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:05.658118  7329 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:05.660293  7682 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:05.669245  6379 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:05.670184  7610 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.677814  7332 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:05.678241  6310 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:05.680374  7754 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:05.686021  7753 ts_tablet_manager.cc:1929] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:25:05.692196  7753 log.cc:1199] T c2d3ed2ffee2408ebcb407323a99660b P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/c2d3ed2ffee2408ebcb407323a99660b
I20260709 12:25:05.691140  6450 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.693323  7754 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:05.693962  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet c2d3ed2ffee2408ebcb407323a99660b (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:05.708146  7689 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.709894  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f0250a69b3914cb38265a7457b73a158 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.713975  7332 raft_consensus.cc:2955] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.723573  6304 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.717438  7329 raft_consensus.cc:2955] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.737236  6206 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 3 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:05.754711  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7878a824e1e94db6a0b83ab47738f4b0 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:05.781924  6204 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 5 to 6, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:05.803448  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet f0250a69b3914cb38265a7457b73a158 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51838
I20260709 12:25:05.813424  7757 tablet_replica.cc:333] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:25:05.820240  7757 raft_consensus.cc:2243] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.820995  7757 raft_consensus.cc:2272] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.828437  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 7878a824e1e94db6a0b83ab47738f4b0 with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:59628
I20260709 12:25:05.830377  7758 tablet_replica.cc:333] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:25:05.834067  7758 raft_consensus.cc:2243] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:05.835556  7758 raft_consensus.cc:2272] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:05.840967  7758 ts_tablet_manager.cc:1916] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.878021  7757 ts_tablet_manager.cc:1916] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:05.911237  7758 ts_tablet_manager.cc:1929] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 5.5
I20260709 12:25:05.911919  7758 log.cc:1199] T 7878a824e1e94db6a0b83ab47738f4b0 P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/7878a824e1e94db6a0b83ab47738f4b0
I20260709 12:25:05.913846  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 7878a824e1e94db6a0b83ab47738f4b0 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:05.917591  7757 ts_tablet_manager.cc:1929] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:25:05.918392  7757 log.cc:1199] T f0250a69b3914cb38265a7457b73a158 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/f0250a69b3914cb38265a7457b73a158
I20260709 12:25:05.922649  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet f0250a69b3914cb38265a7457b73a158 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:06.068543  6377 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:06.080200  7332 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:06.082567  7738 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:06.114182  6451 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:06.112483  7738 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:06.115700  7754 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:06.118623  7329 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:06.119740  6451 raft_consensus.cc:2955] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:06.157934  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8729b74dbaf94d7092d811f0a55205f2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:06.157618  6204 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:06.173877  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet 8729b74dbaf94d7092d811f0a55205f2 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:57654
I20260709 12:25:06.175064  7753 tablet_replica.cc:333] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:25:06.191402  7753 raft_consensus.cc:2243] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:06.192365  7753 raft_consensus.cc:2272] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:06.196610  7753 ts_tablet_manager.cc:1916] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:06.215093  7753 ts_tablet_manager.cc:1929] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:06.215572  7753 log.cc:1199] T 8729b74dbaf94d7092d811f0a55205f2 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/8729b74dbaf94d7092d811f0a55205f2
I20260709 12:25:06.217561  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet 8729b74dbaf94d7092d811f0a55205f2 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:07.294708  6303 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:07.304900  6454 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:07.305614  6275 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 62bd177b33be425d81796cc414f86f76. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:07.307322  7682 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:07.307880  6377 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.358912  7737 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:25:07.372102  7741 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.400193  6456 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.443883  6204 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.444444  6304 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:07.465013  6377 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.470868  6385 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:07.471354  6275 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: eede017c955d4746bbbd40d9384932e7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:07.474678  6454 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.475116  7764 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:07.477291  7682 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:07.507339  7725 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.510426  6379 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:25:07.518844  7769 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:25:07.539698  6379 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:07.541604  6454 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.575567  6454 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.593698  7736 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:07.598402  6351 consensus_peers.cc:597] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a5012c92e5bb4fe188dca8f60efceeb0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:07.589510  6202 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.604182  6303 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.627872  7738 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.634516  7689 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:07.643491  6456 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.654296  6379 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:07.680601  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fe2196990ce4d60adc195d394d4c01c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:07.681066  6454 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.683427  6304 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:07.685266  7778 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:07.685698  6206 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.687539  7778 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:07.693626  6303 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.718364  7736 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.721923  6456 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.725415  6304 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:07.742251  6202 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:07.856344  7784 ts_tablet_manager.cc:933] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:07.879410  7784 tablet_copy_client.cc:323] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:07.882318  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 62bd177b33be425d81796cc414f86f76 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:07.883528  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 62bd177b33be425d81796cc414f86f76 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-62bd177b33be425d81796cc414f86f76
I20260709 12:25:07.897568  6320 tablet_copy_source_session.cc:215] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:07.901753  7784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62bd177b33be425d81796cc414f86f76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:07.925838  7784 tablet_copy_client.cc:806] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:07.926717  7784 tablet_copy_client.cc:670] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:07.963927  7784 tablet_copy_client.cc:538] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:07.977605  7784 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:08.018826  7784 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.019881  7784 tablet_bootstrap.cc:492] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:08.020627  7784 ts_tablet_manager.cc:1403] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.043s	user 0.019s	sys 0.012s
I20260709 12:25:08.041426  7784 raft_consensus.cc:359] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.042421  7784 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:08.043249  7784 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.046981  7784 ts_tablet_manager.cc:1434] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 12:25:08.049175  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-62bd177b33be425d81796cc414f86f76 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:08.049780  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-62bd177b33be425d81796cc414f86f76 on tablet 62bd177b33be425d81796cc414f86f76 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:08.080049  7784 ts_tablet_manager.cc:933] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:08.099349  7789 ts_tablet_manager.cc:933] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:08.105856  7784 tablet_copy_client.cc:323] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:25:08.106745  7789 tablet_copy_client.cc:323] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:08.112466  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet eede017c955d4746bbbd40d9384932e7 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:08.112531  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet a5012c92e5bb4fe188dca8f60efceeb0 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:25:08.112990  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet eede017c955d4746bbbd40d9384932e7 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-eede017c955d4746bbbd40d9384932e7
I20260709 12:25:08.113302  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet a5012c92e5bb4fe188dca8f60efceeb0 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-a5012c92e5bb4fe188dca8f60efceeb0
I20260709 12:25:08.124061  6395 tablet_copy_source_session.cc:215] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.128509  6320 tablet_copy_source_session.cc:215] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.132162  7789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eede017c955d4746bbbd40d9384932e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.133363  7784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5012c92e5bb4fe188dca8f60efceeb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.165233  7789 tablet_copy_client.cc:806] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:08.165999  7789 tablet_copy_client.cc:670] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:08.167690  7784 tablet_copy_client.cc:806] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:08.168383  7784 tablet_copy_client.cc:670] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:08.175154  7784 tablet_copy_client.cc:538] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:08.176126  7789 tablet_copy_client.cc:538] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:08.193804  7789 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:08.218022  7784 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:08.252588  7789 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.253463  7789 tablet_bootstrap.cc:492] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:08.256552  7789 ts_tablet_manager.cc:1403] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.063s	user 0.025s	sys 0.024s
I20260709 12:25:08.260020  7789 raft_consensus.cc:359] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.261005  7789 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:08.261694  7789 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.264672  7789 ts_tablet_manager.cc:1434] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:25:08.270438  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-eede017c955d4746bbbd40d9384932e7 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:08.270970  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-eede017c955d4746bbbd40d9384932e7 on tablet eede017c955d4746bbbd40d9384932e7 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:08.277545  7784 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.279381  7784 tablet_bootstrap.cc:492] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:08.280181  7784 ts_tablet_manager.cc:1403] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.062s	user 0.020s	sys 0.011s
I20260709 12:25:08.285302  7784 raft_consensus.cc:359] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.286250  7784 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:08.286948  7784 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.289899  7784 ts_tablet_manager.cc:1434] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:25:08.291698  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-a5012c92e5bb4fe188dca8f60efceeb0 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:25:08.292233  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-a5012c92e5bb4fe188dca8f60efceeb0 on tablet a5012c92e5bb4fe188dca8f60efceeb0 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:08.295580  7329 raft_consensus.cc:1217] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:08.303043  7789 ts_tablet_manager.cc:933] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:08.315528  7789 tablet_copy_client.cc:323] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:25:08.318110  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 0fe2196990ce4d60adc195d394d4c01c from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:25:08.318790  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 0fe2196990ce4d60adc195d394d4c01c from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-0fe2196990ce4d60adc195d394d4c01c
I20260709 12:25:08.328612  6395 tablet_copy_source_session.cc:215] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:08.332218  7789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe2196990ce4d60adc195d394d4c01c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:08.356215  7789 tablet_copy_client.cc:806] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:08.356949  7789 tablet_copy_client.cc:670] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:08.362910  7789 tablet_copy_client.cc:538] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:08.374588  7789 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:08.419450  7789 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:08.420578  7789 tablet_bootstrap.cc:492] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:08.421973  7789 ts_tablet_manager.cc:1403] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.048s	user 0.032s	sys 0.008s
I20260709 12:25:08.425485  7789 raft_consensus.cc:359] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.426434  7789 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:08.427218  7789 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:08.429915  7789 ts_tablet_manager.cc:1434] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:25:08.433121  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-0fe2196990ce4d60adc195d394d4c01c received from {username='slave'} at 127.0.0.1:46610
I20260709 12:25:08.433657  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-0fe2196990ce4d60adc195d394d4c01c on tablet 0fe2196990ce4d60adc195d394d4c01c with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:08.455379  7329 raft_consensus.cc:1217] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:08.524221  7329 raft_consensus.cc:1217] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:08.827864  7329 raft_consensus.cc:1217] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:08.880908  7767 raft_consensus.cc:1064] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:08.898765  7767 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:08.932309  7329 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.934185  7795 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.941036  6451 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.943778  7795 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:08.955622  6379 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:08.957387  7767 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:08.963478  7725 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:08.965386  6456 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:08.967132  7738 raft_consensus.cc:1064] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:08.969358  7738 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:08.986436  6202 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.002022  7329 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.030790  6451 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.032361  6304 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.032523  7332 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.034189  7736 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.035382  7796 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.036283  7736 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:25:09.134752  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67953.5us
I20260709 12:25:09.171793  7725 raft_consensus.cc:1064] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:09.173889  7725 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:09.265518  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48200us
I20260709 12:25:09.283995  7329 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.286605  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:09.290516  7810 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:09.347699  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43525.6us
W20260709 12:25:09.366580  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 84848.5us
I20260709 12:25:09.400271  7738 raft_consensus.cc:1064] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:09.408403  7738 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:09.475790  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45194.9us
W20260709 12:25:09.467705  7804 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
W20260709 12:25:09.479034  7813 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:25:09.506104  7329 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:25:09.513449  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 117440us
I20260709 12:25:09.522715  7796 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.532724  6451 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.537040  6458 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.545468  7781 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.596036  6309 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.613423  7832 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:09.627020  6450 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.632278  7832 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:09.635912  6379 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.639161  6377 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:09.653095  6300 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:09.660648  6204 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.666929  7332 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:09.668407  7829 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:09.676421  7829 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.683789  6309 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.690965  7335 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.692095  7725 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:09.699762  7332 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.739995  7832 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.750973  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 62bd177b33be425d81796cc414f86f76 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:09.767446  6304 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.775233  7809 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:09.793656  6450 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:09.802541  6377 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:09.807313  6204 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:09.827729  6456 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.846798  7795 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:25:09.868368  7320 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:09.869948  7329 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.901029  6450 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:09.927392  6455 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.910341  7811 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.959043  6376 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.922395  7832 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:25:09.957752  6450 raft_consensus.cc:2955] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:09.969386  7796 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:09.975414  6206 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:09.990870  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet 62bd177b33be425d81796cc414f86f76 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51838
I20260709 12:25:09.995791  7832 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.000242  7844 tablet_replica.cc:333] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:25:10.001667  7844 raft_consensus.cc:2243] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:10.004498  7332 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.004604  6450 raft_consensus.cc:2955] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.008852  7844 raft_consensus.cc:2272] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:10.025537  7844 ts_tablet_manager.cc:1916] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:10.028936  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a5012c92e5bb4fe188dca8f60efceeb0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:10.034663  6379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:10.035557  6379 raft_consensus.cc:606] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:10.048705  7844 ts_tablet_manager.cc:1929] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:10.048171  6206 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.049456  7844 log.cc:1199] T 62bd177b33be425d81796cc414f86f76 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/62bd177b33be425d81796cc414f86f76
I20260709 12:25:10.050120  6206 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 3 to 4, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.018152  7335 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.064877  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet 62bd177b33be425d81796cc414f86f76 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
W20260709 12:25:10.073709  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57194.8us
I20260709 12:25:10.110699  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet a5012c92e5bb4fe188dca8f60efceeb0 with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 3 (current committed config index is 4)) from {username='slave'} at 127.0.0.1:57654
I20260709 12:25:10.117858  6379 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.116986  6304 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.133366  6309 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:10.134052  6309 raft_consensus.cc:606] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:10.136677  7335 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:10.138296  6305 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:10.141592  6309 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:10.143517  6309 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:10.145625  7738 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:10.146688  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:10.147336  6377 raft_consensus.cc:606] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:10.148134  7738 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:10.161263  7736 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.169063  7335 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.166162  6309 raft_consensus.cc:2955] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.190937  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:25:10.191690  6450 raft_consensus.cc:606] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:10.196971  6379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:10.197737  6379 raft_consensus.cc:606] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:10.199249  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:10.201200  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:10.202370  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:10.204393  7332 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:10.204393  6450 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:10.206472  7823 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:10.208684  7823 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:10.212234  6204 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 1 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.226897  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0fe2196990ce4d60adc195d394d4c01c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:10.227999  7822 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.240474  7335 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.254573  7846 tablet_replica.cc:333] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:25:10.254763  7808 raft_consensus.cc:993] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:10.258618  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fe2196990ce4d60adc195d394d4c01c with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59628
I20260709 12:25:10.258876  7820 raft_consensus.cc:1081] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:10.263311  6456 raft_consensus.cc:2955] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.272400  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet eede017c955d4746bbbd40d9384932e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:10.276819  6378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:25:10.277478  6378 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.277797  6378 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.283783  7846 raft_consensus.cc:2243] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:10.285272  7846 raft_consensus.cc:2272] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:10.288611  6202 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:10.293041  6378 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.294652  7848 tablet_replica.cc:333] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:25:10.308007  6378 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:10.309322  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:10.310110  6456 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.316134  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:10.316767  6304 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:10.317055  6304 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:10.317829  6304 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.318869  6304 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.307714  7848 raft_consensus.cc:2243] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:10.323477  7848 raft_consensus.cc:2272] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:10.330237  7846 ts_tablet_manager.cc:1916] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:10.331782  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet eede017c955d4746bbbd40d9384932e7 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51838
I20260709 12:25:10.344404  7844 tablet_replica.cc:333] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:25:10.345595  7844 raft_consensus.cc:2243] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:10.346442  7844 raft_consensus.cc:2272] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:10.342790  7848 ts_tablet_manager.cc:1916] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:10.352716  7844 ts_tablet_manager.cc:1916] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:10.322263  6456 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:10.355453  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:10.356587  7826 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:10.357167  7826 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:10.357999  7826 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.366602  7736 raft_consensus.cc:993] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:10.367167  7736 raft_consensus.cc:1081] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:10.368961  7329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:10.369570  7329 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.369904  7329 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.382974  7329 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.387495  6304 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:10.390699  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:10.391342  6309 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.391520  7844 ts_tablet_manager.cc:1929] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:10.391942  7844 log.cc:1199] T eede017c955d4746bbbd40d9384932e7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/eede017c955d4746bbbd40d9384932e7
I20260709 12:25:10.393388  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet eede017c955d4746bbbd40d9384932e7 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:10.390100  7848 ts_tablet_manager.cc:1929] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:10.395697  7848 log.cc:1199] T 0fe2196990ce4d60adc195d394d4c01c P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/0fe2196990ce4d60adc195d394d4c01c
I20260709 12:25:10.388024  7329 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:10.388657  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:10.396601  6378 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:10.396930  6378 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:10.397341  7736 raft_consensus.cc:993] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:10.397821  7736 raft_consensus.cc:1081] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:10.397850  6378 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.399260  6378 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:10.399537  7335 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:10.400184  7335 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.400532  7335 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:10.403002  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 0fe2196990ce4d60adc195d394d4c01c (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:10.409279  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:10.408977  7335 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.411708  7335 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:10.412336  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:10.413437  7840 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:10.413908  7840 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:10.413978  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:10.414620  6379 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:10.414935  6379 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:10.415709  6379 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.416996  6379 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:10.419308  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:10.419951  6309 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:10.423107  7840 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.424050  6379 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:10.425770  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:10.427320  7838 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:10.416991  6378 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:10.431824  6309 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:10.437703  7846 ts_tablet_manager.cc:1929] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:10.438226  7846 log.cc:1199] T a5012c92e5bb4fe188dca8f60efceeb0 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/a5012c92e5bb4fe188dca8f60efceeb0
I20260709 12:25:10.443454  6204 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.442142  7838 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:10.449847  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet a5012c92e5bb4fe188dca8f60efceeb0 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:10.459406  6206 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 3 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.448642  7838 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.484730  6206 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 5 to 6, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 6 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.503872  7812 raft_consensus.cc:993] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:10.504410  7819 raft_consensus.cc:1081] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:10.505820  6378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:25:10.506276  6378 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.506708  6378 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.512883  6378 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.515429  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:10.515977  7829 raft_consensus.cc:993] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:10.516629  7810 raft_consensus.cc:1081] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:10.516649  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:10.517396  6456 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:10.516011  6309 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.517765  6456 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:10.519052  6456 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.520036  6456 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:10.514796  6378 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:10.525043  6378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:25:10.525578  6378 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.525987  6378 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.536130  6378 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.538507  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:10.539037  6304 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:10.539319  6304 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:10.539979  6304 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.540956  6304 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.543249  6309 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:10.548454  6304 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:10.550258  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:10.550648  6456 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:10.551241  7738 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:10.555006  7738 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:10.555876  7738 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.551385  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:10.559068  7796 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:10.559592  7796 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:10.560483  7796 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:10.561678  6378 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:10.563709  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:10.564272  6456 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:10.577879  6456 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:10.579339  6206 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 4, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.598722  6206 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:10.606169  7736 raft_consensus.cc:993] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:10.606719  7738 raft_consensus.cc:1081] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:10.608260  7329 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:10.608776  7329 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:10.609125  7329 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:10.619030  7329 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.621521  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:10.621922  7329 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:10.622316  6304 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:10.630000  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:10.630577  6379 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:10.630879  6379 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:10.631670  6379 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.632912  6379 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:10.635098  6304 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:10.636067  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:10.636739  7838 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:10.637168  7838 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:10.638013  7838 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:10.642740  6379 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:10.652714  6206 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 4 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 5 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:10.782933  6309 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:10.784439  7796 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:10.818895  6456 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:10.821367  7826 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:10.836479  6379 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:10.839845  7838 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:10.855614  6309 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:10.862610  7850 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:10.923514  6377 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:10.924714  7850 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:10.935973  6309 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:10.937505  7838 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:11.008687  6309 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:11.009897  7826 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:11.028291  6456 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:11.032706  7835 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:11.033988  6456 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:11.037421  7738 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:11.062705  6304 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:11.064505  7835 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:11.112082  6309 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:11.114806  7838 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:11.308218  6377 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:11.459331  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:11.809036  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.010s	sys 0.009s
W20260709 12:25:11.809404  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.011s	sys 0.009s
W20260709 12:25:11.810393  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 729426us
W20260709 12:25:11.939703  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51864.3us
I20260709 12:25:11.986837  7888 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
W20260709 12:25:12.016418  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62671.1us
I20260709 12:25:12.027786  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:12.028609  6383 raft_consensus.cc:606] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:12.032385  6458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:25:12.032974  6458 raft_consensus.cc:606] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:12.036825  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:12.037570  6383 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:12.042706  6304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:12.043313  6304 raft_consensus.cc:606] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:12.046476  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:12.047449  6383 raft_consensus.cc:606] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:12.046418  6310 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:12.048461  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54358.6us
I20260709 12:25:12.051100  6304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:12.051666  6304 raft_consensus.cc:606] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:12.053099  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:12.054842  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:12.055603  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:12.065586  6458 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:25:12.065922  6275 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0ddf1d667df04473a598939605e43a1e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:12.067202  7873 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:12.077783  6385 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:12.079366  7876 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:12.079136  7879 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.084717  6458 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.110407  6204 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.116746  6310 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:12.125167  6385 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:25:12.128589  6275 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c413b06caa444496aecb13ac2142c246. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:12.134200  6458 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:12.138324  6381 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:25:12.143844  7875 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.146003  7875 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:12.169198  7875 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.170614  7885 raft_consensus.cc:993] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:12.172297  7885 raft_consensus.cc:1081] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:12.181175  6458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:25:12.181790  6458 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.182188  6458 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.189724  6458 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.190575  6385 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:12.191707  6458 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:12.195526  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:12.196204  6304 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.196091  6458 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.200032  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:12.200668  6383 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:12.200941  6383 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:12.201908  6383 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.203097  6383 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.211432  6304 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:12.212806  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:12.213802  7819 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:12.214298  7819 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:12.215288  7819 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.215912  6201 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.212888  6383 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:25:12.244568  6351 consensus_peers.cc:597] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e600ce2bd05948cf8ce5a5f292f57f28. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:12.245309  7876 raft_consensus.cc:993] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:12.227084  6381 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.245802  7876 raft_consensus.cc:1081] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:12.249114  6383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:25:12.249642  6383 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.249981  6383 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:12.252954  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08000e13a0 after lost signal to thread 5140
W20260709 12:25:12.254149  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08002fb380 after lost signal to thread 6240
I20260709 12:25:12.255388  6304 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:12.255479  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf060 after lost signal to thread 6334
W20260709 12:25:12.256781  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08000b19e0 after lost signal to thread 6409
W20260709 12:25:12.258018  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08000fed40 after lost signal to thread 6483
I20260709 12:25:12.257624  7883 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:12.259318  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08002426c0 after lost signal to thread 7371
I20260709 12:25:12.258843  6202 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.278882  6383 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.280823  6383 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:12.281452  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:12.282877  6309 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.283183  6309 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:12.283946  6309 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.284375  6458 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:12.285115  6309 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.287623  7875 raft_consensus.cc:993] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:12.288111  7873 raft_consensus.cc:1081] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:12.289835  6383 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:25:12.291503  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:12.291729  6383 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.292119  6454 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.292276  6383 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.297645  7887 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.310770  6383 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.312692  6383 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:12.319648  6454 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:12.320698  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:12.324612  6451 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.326206  6309 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:12.328315  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:12.328975  6300 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.329285  6300 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:12.330093  6300 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.337450  7738 raft_consensus.cc:993] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:12.338466  7738 raft_consensus.cc:1081] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:12.338617  7884 raft_consensus.cc:993] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:12.325860  6304 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.341902  6300 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.347150  7332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:12.347821  7332 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.348196  7332 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.348546  7884 raft_consensus.cc:1081] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:12.351749  6455 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:25:12.352267  6455 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.352576  6455 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.359249  6455 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.360718  6300 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:12.361869  6455 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:12.365795  7332 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:12.369112  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:12.369778  6309 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.369889  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:12.369341  7332 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:12.370441  6454 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.371213  7886 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.372952  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:12.379953  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:12.380597  7883 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:12.387902  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:12.407696  7883 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:12.408680  7883 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.388422  6456 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:12.417194  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:12.418859  7872 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:12.419442  7872 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:12.420519  7872 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.425143  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:12.425936  6385 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.430927  6385 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:12.431831  6385 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:12.432798  6385 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.425987  7914 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:12.404604  6454 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:12.405830  6455 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.443539  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:12.444882  7892 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:12.445638  7892 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:12.446616  7892 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:12.403304  6451 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:12.441720  6385 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:12.464358  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:12.465257  6385 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:12.467278  6385 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:12.468339  6425 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4db5d1ae5f524ad799ccb78373ce3ff7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:12.468107  6385 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.469792  6385 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:12.469964  7914 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:12.471297  7914 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:12.425665  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45789.3us
I20260709 12:25:12.477483  6309 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:12.479324  7819 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:12.490190  7920 ts_tablet_manager.cc:933] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:12.493233  6385 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:12.495057  6383 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:25:12.496378  6456 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:12.497540  7812 raft_consensus.cc:993] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:12.498061  7812 raft_consensus.cc:1081] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:12.500867  6201 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 3, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.511556  7819 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:12.551348  6202 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.555365  6204 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:12.606036  7812 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.617653  6304 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.626703  7920 tablet_copy_client.cc:323] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
W20260709 12:25:12.650591  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 144337us
I20260709 12:25:12.652009  6376 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:25:12.652525  6376 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:12.652863  6376 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.659360  6376 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.664109  6376 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:12.665690  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 0ddf1d667df04473a598939605e43a1e from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:12.666584  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 0ddf1d667df04473a598939605e43a1e from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-0ddf1d667df04473a598939605e43a1e
I20260709 12:25:12.673561  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:12.672636  6385 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.674202  6451 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:12.674567  6451 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:12.677533  6451 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.678802  6451 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.692005  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:12.692713  6300 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:12.693142  6206 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.694996  6206 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.700234  6320 tablet_copy_source_session.cc:215] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:12.707767  7929 ts_tablet_manager.cc:933] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:12.710210  6300 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:12.715960  6204 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:12.717556  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:12.719173  7796 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:12.719769  7796 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:12.720706  7796 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:12.730958  7929 tablet_copy_client.cc:323] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:12.734068  6451 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:12.749411  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet c413b06caa444496aecb13ac2142c246 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:12.749712  7920 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ddf1d667df04473a598939605e43a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:12.753798  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet c413b06caa444496aecb13ac2142c246 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-c413b06caa444496aecb13ac2142c246
I20260709 12:25:12.791239  6204 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:12.804814  7920 tablet_copy_client.cc:806] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:12.805488  7920 tablet_copy_client.cc:670] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:12.811890  7920 tablet_copy_client.cc:538] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:12.813059  6320 tablet_copy_source_session.cc:215] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:12.817099  7929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c413b06caa444496aecb13ac2142c246. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:12.819199  6376 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:12.820449  7920 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:12.821153  7872 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:12.841373  6300 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:12.843261  7928 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:12.857369  6385 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:12.857928  6381 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:12.861280  6309 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:12.863942  6451 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:12.869403  7926 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:12.931087  7934 ts_tablet_manager.cc:933] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:12.903198  7872 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 12:25:12.955261  7920 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:25:12.988991  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 99404.8us
I20260709 12:25:12.920701  7926 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:25:12.992918  7934 tablet_copy_client.cc:323] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:25:13.023021  6304 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:13.036197  7920 tablet_bootstrap.cc:492] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:13.039223  7886 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:25:13.043572  7812 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.025s
I20260709 12:25:13.089628  6455 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:13.094036  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet e600ce2bd05948cf8ce5a5f292f57f28 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:25:13.094708  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet e600ce2bd05948cf8ce5a5f292f57f28 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-e600ce2bd05948cf8ce5a5f292f57f28
I20260709 12:25:13.116101  6450 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:13.122380  7920 ts_tablet_manager.cc:1403] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.302s	user 0.026s	sys 0.006s
I20260709 12:25:13.131126  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:13.136757  6310 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:13.114090  6395 tablet_copy_source_session.cc:215] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:13.145802  7914 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:25:13.153118  7887 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
W20260709 12:25:13.160764  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 118573us
I20260709 12:25:13.166755  6309 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:13.165381  7920 raft_consensus.cc:359] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:13.167282  7886 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:13.168097  7920 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:13.168869  7886 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:13.170969  7934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e600ce2bd05948cf8ce5a5f292f57f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:13.178577  7920 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:13.159627  7929 tablet_copy_client.cc:806] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:13.164302  7946 ts_tablet_manager.cc:933] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:13.190286  7929 tablet_copy_client.cc:670] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
W20260709 12:25:13.305217  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63720.1us
W20260709 12:25:13.330725  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92137.6us
W20260709 12:25:13.466966  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 281480us
W20260709 12:25:13.559183  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 243201us
W20260709 12:25:13.560601  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 304191us
I20260709 12:25:13.583518  6458 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:13.560791  7946 tablet_copy_client.cc:323] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
W20260709 12:25:13.581424  7933 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:25:13.586419  7920 ts_tablet_manager.cc:1434] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.464s	user 0.015s	sys 0.013s
I20260709 12:25:13.625797  7887 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:13.641047  7950 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:25:13.649195  7955 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:25:13.651686  7962 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:13.739992  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:13.747373  6383 raft_consensus.cc:606] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:13.767243  7960 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.192s	user 0.001s	sys 0.000s
I20260709 12:25:13.776580  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:13.777475  6376 raft_consensus.cc:606] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:13.781101  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:13.781725  6376 raft_consensus.cc:606] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:13.784767  6309 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "62bd177b33be425d81796cc414f86f76"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:13.785146  7934 tablet_copy_client.cc:806] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:13.785454  6309 raft_consensus.cc:606] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:13.785827  7934 tablet_copy_client.cc:670] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:13.798171  7319 raft_consensus.cc:1217] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:13.850363  6319 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-0ddf1d667df04473a598939605e43a1e received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:13.855387  6319 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-0ddf1d667df04473a598939605e43a1e on tablet 0ddf1d667df04473a598939605e43a1e with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:13.866184  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 285432us
W20260709 12:25:13.889107  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59911.9us
W20260709 12:25:13.907050  7957 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:25:13.944547  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64847.2us
W20260709 12:25:13.985230  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86938.9us
I20260709 12:25:13.985116  7929 tablet_copy_client.cc:538] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.132308  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 4db5d1ae5f524ad799ccb78373ce3ff7 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:25:14.135732  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 4db5d1ae5f524ad799ccb78373ce3ff7 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-4db5d1ae5f524ad799ccb78373ce3ff7
W20260709 12:25:14.142946  7967 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:25:14.156311  6469 tablet_copy_source_session.cc:215] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:25:14.164057  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49482us
W20260709 12:25:14.181437  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 65945.2us
I20260709 12:25:14.184350  7946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4db5d1ae5f524ad799ccb78373ce3ff7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:25:14.254359  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 273854us
W20260709 12:25:14.260505  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 276631us
W20260709 12:25:14.276087  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 292730us
I20260709 12:25:14.329319  7938 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.330124  7938 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.333169  7938 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.334615  7973 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.335240  7973 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.337646  7973 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.341382  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:25:14.348980  6451 raft_consensus.cc:606] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:14.358821  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:14.380340  7941 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.381403  7941 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.389832  7985 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.390899  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:14.390470  7985 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.439913  7999 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.440722  7999 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.428506  7992 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.447029  7992 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.454581  7985 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.470719  7992 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:14.473441  7941 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.482846  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:14.488533  7999 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.491282  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:14.500370  7971 raft_consensus.cc:993] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:14.503794  7993 raft_consensus.cc:993] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
W20260709 12:25:14.545751  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 279495us
W20260709 12:25:14.549419  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56746.6us
W20260709 12:25:14.559543  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85954.8us
W20260709 12:25:14.558609  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 129486us
I20260709 12:25:14.573776  7883 raft_consensus.cc:1081] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:14.581176  7983 raft_consensus.cc:1081] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:14.584951  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:14.582716  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:14.586231  7319 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:14.588253  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:14.588912  7319 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.589409  7319 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.593592  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:14.595934  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47311.5us
I20260709 12:25:14.593675  7997 raft_consensus.cc:993] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:14.597873  7997 raft_consensus.cc:1081] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:14.590587  7883 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.602665  7318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:14.603266  7318 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.603662  7318 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.603718  7946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43766 (request call id 262) took 1581 ms. Trace:
I20260709 12:25:14.599814  7983 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:14.613986  7929 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:14.629915  6451 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:25:14.639226  6451 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.639797  6451 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:14.647724  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:14.656736  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 383330us
W20260709 12:25:14.659579  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62611.1us
I20260709 12:25:14.651160  7971 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:14.650105  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 315513us
I20260709 12:25:14.675928  7913 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:14.717885  6451 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.730723  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:14.733832  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:14.744015  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:14.752473  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:14.604262  7946 rpcz_store.cc:276] 0709 12:25:13.022113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.022287 (+   174us) service_pool.cc:224] Handling call
0709 12:25:13.189687 (+167400us) ts_tablet_manager.cc:934] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
0709 12:25:14.182384 (+992697us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:14.476446 (+294062us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:14.483399 (+  6953us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:14.602883 (+119484us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:14.603205 (+   322us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:14.603559 (+   354us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:14.603626 (+    67us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":110292,"tablet-copy.queue_time_us":134782,"thread_start_us":269631,"threads_started":2}
I20260709 12:25:14.754947  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:14.718187  7915 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.739557  7923 raft_consensus.cc:993] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
W20260709 12:25:14.809175  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 218107us
I20260709 12:25:14.794350  7974 raft_consensus.cc:1081] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:14.793076  7883 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.722915  7913 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.833413  6302 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:14.806854  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:14.782277  7915 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.834110  6302 raft_consensus.cc:606] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:14.843012  7319 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.847682  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:14.849407  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:14.850195  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:14.854370  7934 tablet_copy_client.cc:538] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.864131  7319 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:14.868330  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:14.868949  6378 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:14.869249  6378 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:14.870035  6378 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.871340  6378 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.873234  7335 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "62bd177b33be425d81796cc414f86f76"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43752
I20260709 12:25:14.873850  7335 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:14.874200  7335 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.874099  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:14.884949  7913 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.887976  7335 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.890986  7335 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.832597  7883 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.891844  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:14.892490  6457 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.898286  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:14.900381  8000 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.903574  8001 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:14.893613  7883 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.870340  7318 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.912277  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:14.913473  7318 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:14.916621  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:14.917868  8000 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:14.924189  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:14.924855  6309 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:14.925283  7946 tablet_copy_client.cc:806] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:14.925794  7946 tablet_copy_client.cc:670] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:14.921201  6457 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:14.929198  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:14.931207  7934 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:14.933593  7958 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:14.933480  7876 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:14.934340  7958 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:14.935267  6451 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:14.934373  7876 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.938395  7946 tablet_copy_client.cc:538] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:14.940114  6309 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:14.938879  7958 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.951113  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:14.954330  7915 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.955049  7959 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:14.960278  7959 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:14.967005  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:14.967665  6309 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:14.970031  6309 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:14.970242  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:14.970927  6379 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:14.971321  6379 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:14.970820  6309 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.972059  6379 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:14.968698  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:14.961151  7959 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.973686  6305 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.973263  6309 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.977048  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:14.977905  6382 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:14.978178  6382 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:14.986550  6305 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:14.978856  6382 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:14.988042  6382 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:14.988440  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:14.989789  8003 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:14.993067  6309 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:14.997330  7876 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:14.997284  7946 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:15.010675  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:15.017653  6382 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:15.020668  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:15.028587  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:15.030570  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:15.032583  6309 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:15.039705  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:15.040604  6309 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.055042  6309 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:15.056663  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:15.063084  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:15.065546  7974 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:15.087674  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:15.091125  8001 raft_consensus.cc:993] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:15.092375  8001 raft_consensus.cc:1081] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
W20260709 12:25:15.095039  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 59729us
I20260709 12:25:15.099961  7970 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:15.102821  7970 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:15.048813  7937 raft_consensus.cc:993] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:15.109632  6385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:25:15.110296  6385 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:15.111172  6385 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:15.103811  7970 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:15.127029  6379 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:15.127635  6308 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:15.137951  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:15.107923  8003 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:15.143611  6379 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:15.117048  7928 raft_consensus.cc:1081] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:15.154654  7332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43766
I20260709 12:25:15.155301  7332 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:15.155674  7332 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:15.159562  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:15.151044  7938 raft_consensus.cc:1064] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:15.178123  6385 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:15.181420  6385 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:15.183636  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:15.191330  6309 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:15.191663  6309 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:15.193444  7332 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:15.153472  8003 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:15.212579  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:15.213181  6455 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:15.213500  6455 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:15.213779  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:15.214315  6379 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:15.214332  7332 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:15.226259  6455 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:15.235783  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:15.236409  6458 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:15.255785  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:15.239362  7929 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:15.261883  6309 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:15.263736  6458 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:15.266029  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:15.267256  6379 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:15.277393  6309 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:15.288546  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:15.283176  7968 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:15.295931  7938 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:15.327607  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:15.336426  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47986.8us
I20260709 12:25:15.339083  7992 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:15.349086  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:15.353633  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:15.357412  6309 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:15.350654  7929 tablet_bootstrap.cc:492] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:15.359673  7886 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:15.365662  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:15.359988  7996 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:15.373042  7929 ts_tablet_manager.cc:1403] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.775s	user 0.031s	sys 0.012s
I20260709 12:25:15.373278  7996 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:15.372608  8003 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:15.388067  8003 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:15.415512  7996 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:15.436168  6455 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:15.436408  8003 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:15.457404  6450 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:15.462589  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:15.480008  6206 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.483042  6302 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:15.485749  7959 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:15.492902  6379 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:15.496253  6385 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:15.497769  8003 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:15.515815  7319 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.517096  6450 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.517402  8000 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:15.519919  8000 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:15.523247  6204 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.535933  7929 raft_consensus.cc:359] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:15.540160  7929 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:15.540840  7929 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:15.541431  6202 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:15.547123  6378 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:15.549468  6308 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:25:15.550052  7946 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.050s	user 0.002s	sys 0.000s
I20260709 12:25:15.555094  7934 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:15.554903  8012 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:15.555693  8020 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:15.556154  7934 tablet_bootstrap.cc:492] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:15.557036  7934 ts_tablet_manager.cc:1403] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.626s	user 0.045s	sys 0.043s
I20260709 12:25:15.559299  6455 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:15.558799  6204 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.567610  7929 ts_tablet_manager.cc:1434] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.187s	user 0.007s	sys 0.001s
I20260709 12:25:15.569316  6204 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.568400  7946 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:25:15.576515  6204 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:15.564908  8020 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:25:15.598568  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-c413b06caa444496aecb13ac2142c246 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:15.599176  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-c413b06caa444496aecb13ac2142c246 on tablet c413b06caa444496aecb13ac2142c246 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:15.605561  6378 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:15.607800  6378 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:15.609192  7940 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:15.694687  8033 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:25:15.609836  8000 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:15.679900  7946 tablet_bootstrap.cc:492] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:15.730041  7946 ts_tablet_manager.cc:1403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.733s	user 0.022s	sys 0.031s
W20260709 12:25:15.829044  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83794.1us
I20260709 12:25:15.838562  7934 raft_consensus.cc:359] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:15.839645  7934 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:15.840454  7934 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:15.842898  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 108950us
W20260709 12:25:15.850730  8034 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 12:25:15.894560  7934 ts_tablet_manager.cc:1434] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.337s	user 0.010s	sys 0.006s
W20260709 12:25:15.899158  8039 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:25:15.898342  7946 raft_consensus.cc:359] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:15.901209  7946 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:15.901901  7946 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:15.921797  8036 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.200s	user 0.003s	sys 0.002s
I20260709 12:25:15.926904  6455 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:15.936426  6379 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:15.974334  7959 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:15.981027  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-e600ce2bd05948cf8ce5a5f292f57f28 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:25:15.981585  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-e600ce2bd05948cf8ce5a5f292f57f28 on tablet e600ce2bd05948cf8ce5a5f292f57f28 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:15.987859  7318 raft_consensus.cc:1217] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:25:15.994028  7317 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:15.995906  7946 ts_tablet_manager.cc:1434] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.265s	user 0.007s	sys 0.004s
I20260709 12:25:15.998737  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4db5d1ae5f524ad799ccb78373ce3ff7 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:25:15.999495  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4db5d1ae5f524ad799ccb78373ce3ff7 on tablet 4db5d1ae5f524ad799ccb78373ce3ff7 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:16.011795  6310 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:25:16.021070  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50472.8us
I20260709 12:25:16.027990  6302 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:16.034353  6456 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:16.042084  6457 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:16.060283  8038 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.329s	user 0.000s	sys 0.006s
W20260709 12:25:16.119690  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50473us
I20260709 12:25:16.134601  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:16.114136  8033 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:16.057199  7915 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.077s
W20260709 12:25:16.141266  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 83336.9us
I20260709 12:25:16.120319  8049 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:25:16.205874  8051 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:25:16.242290  6309 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:16.255339  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 176186us
I20260709 12:25:16.267009  7317 raft_consensus.cc:1217] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
W20260709 12:25:16.286461  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 266636us
I20260709 12:25:16.312199  8058 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:16.334781  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44381.1us
I20260709 12:25:16.345502  7812 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:16.363595  7317 raft_consensus.cc:1217] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
I20260709 12:25:16.366209  6380 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:16.415568  8052 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:25:16.481787  7937 raft_consensus.cc:1064] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
W20260709 12:25:16.789565  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56951.2us
I20260709 12:25:16.658005  7937 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:16.816918  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:16.817914  6300 raft_consensus.cc:606] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:16.833048  8073 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:25:16.854907  8053 raft_consensus.cc:1064] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:16.857829  8053 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:16.874397  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66863.1us
W20260709 12:25:16.898700  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49647.2us
W20260709 12:25:16.923751  8076 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.274s	user 0.003s	sys 0.000s
W20260709 12:25:16.991017  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 103997us
W20260709 12:25:16.994742  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94086.4us
I20260709 12:25:17.034296  6204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 125) took 1520 ms. Trace:
I20260709 12:25:17.039734  6204 rpcz_store.cc:276] 0709 12:25:15.513629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.513759 (+   130us) service_pool.cc:224] Handling call
0709 12:25:17.034008 (+1520249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:15.717121 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:15.717638 (+   517us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:15.717663 (+    25us) write_op.cc:435] Acquired schema lock
0709 12:25:15.717680 (+    17us) tablet.cc:662] Decoding operations
0709 12:25:15.728204 (+ 10524us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:15.728264 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:15.728298 (+    34us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:15.728723 (+   425us) tablet.cc:701] Row locks acquired
0709 12:25:15.728743 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:25:15.728888 (+   145us) write_op.cc:270] Start()
0709 12:25:15.729013 (+   125us) write_op.cc:276] Timestamp: P: 1783599915728845 usec, L: 0
0709 12:25:15.729038 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:25:15.729632 (+   594us) log.cc:844] Serialized 1650 byte log entry
0709 12:25:16.425486 (+695854us) op_driver.cc:464] REPLICATION: finished
0709 12:25:16.475104 (+ 49618us) write_op.cc:301] APPLY: starting
0709 12:25:16.477508 (+  2404us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:16.499702 (+ 22194us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:16.513693 (+ 13991us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:16.596085 (+ 82392us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:16.644954 (+ 48869us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:16.653391 (+  8437us) write_op.cc:312] APPLY: finished
0709 12:25:16.673021 (+ 19630us) log.cc:844] Serialized 41 byte log entry
0709 12:25:16.701219 (+ 28198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.726972 (+ 25753us) write_op.cc:454] Released schema lock
0709 12:25:16.727560 (+   588us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:16.805253 (+ 77693us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:16.809454 (+  4201us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"spinlock_wait_cycles":2944,"child_traces":[["op",{"apply.queue_time_us":16603,"apply.run_cpu_time_us":7205,"apply.run_wall_time_us":283543,"num_ops":4,"prepare.queue_time_us":36180,"prepare.run_cpu_time_us":3094,"prepare.run_wall_time_us":19233,"replication_time_us":696141}]]}
I20260709 12:25:17.109158  6201 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 279) took 1502 ms. Trace:
I20260709 12:25:17.109753  6201 rpcz_store.cc:276] 0709 12:25:15.606753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.607019 (+   266us) service_pool.cc:224] Handling call
0709 12:25:17.108865 (+1501846us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":1}
I20260709 12:25:17.111775  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:17.113098  6385 raft_consensus.cc:606] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:17.118448  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:25:17.121201  6458 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:17.133505  7318 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:17.148594  7317 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:17.148847  6202 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:25:17.159909  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100297us
I20260709 12:25:17.162406  8062 raft_consensus.cc:1064] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:25:17.151624  6450 raft_consensus.cc:606] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:17.167930  6380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:17.168915  6380 raft_consensus.cc:606] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:17.167156  8062 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:17.172690  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:17.185179  8053 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:17.186134  7943 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:17.244037  6305 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:17.248035  6378 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:25:17.325949  6385 raft_consensus.cc:606] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:17.332239  7989 raft_consensus.cc:993] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:17.337287  6376 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:25:17.334168  7989 raft_consensus.cc:1081] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:17.379884  8083 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:17.419795  6304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:17.420536  6304 raft_consensus.cc:606] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:17.423095  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:17.436509  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:17.438031  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:17.479044  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 77579.1us
I20260709 12:25:17.484902  8065 raft_consensus.cc:993] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:17.485946  7332 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:17.488389  8041 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:17.488866  7970 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:25:17.491816  8000 raft_consensus.cc:1081] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
W20260709 12:25:17.495422  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 73114.5us
I20260709 12:25:17.505121  7996 raft_consensus.cc:993] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:17.505820  7996 raft_consensus.cc:1081] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:17.473527  6302 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:17.551699  7994 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:17.583029  8062 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
W20260709 12:25:17.628104  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 147282us
I20260709 12:25:17.631036  7331 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:17.631613  7331 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:17.632017  7331 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.632447  7317 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43762
I20260709 12:25:17.633122  7317 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:17.633524  7317 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:17.638259  8096 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:17.638404  8085 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:25:17.639103  8096 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:17.646737  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 182355us
I20260709 12:25:17.660686  7331 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:17.664996  7331 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:17.678160  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:17.679244  6450 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.683137  6456 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:25:17.689762  8063 raft_consensus.cc:993] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:17.690397  7940 raft_consensus.cc:1081] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:17.718801  8062 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:17.731638  8096 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:17.748943  7314 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43752
I20260709 12:25:17.749788  7314 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:17.750272  7314 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.768837  6450 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:17.778967  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:17.783255  6456 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:17.787046  8084 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:17.788096  8084 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:17.788435  8084 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:17.789383  8084 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:17.794728  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 79636.2us
W20260709 12:25:17.743472  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 98072.4us
I20260709 12:25:17.838135  6385 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:25:17.855238  6385 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:17.855738  6385 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:17.844005  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:17.872418  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49121.8us
W20260709 12:25:17.873626  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50321.4us
I20260709 12:25:17.786104  7317 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:17.913431  6202 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 247) took 1549 ms. Trace:
I20260709 12:25:17.913730  8084 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:17.935022  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90524.7us
W20260709 12:25:17.943925  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67861.7us
W20260709 12:25:17.933365  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 96010.5us
I20260709 12:25:17.987047  7320 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:17.989372  8105 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:25:17.991725  7317 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:18.069149  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 109947us
I20260709 12:25:17.923391  6202 rpcz_store.cc:276] 0709 12:25:16.363908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.364066 (+   158us) service_pool.cc:224] Handling call
0709 12:25:16.389599 (+ 25533us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:17.913368 (+1523769us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:17.405120 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:17.494398 (+ 89278us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:17.494456 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:25:17.494477 (+    21us) tablet.cc:662] Decoding operations
0709 12:25:17.494922 (+   445us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:17.495032 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:17.495063 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:17.495432 (+   369us) tablet.cc:701] Row locks acquired
0709 12:25:17.495487 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:25:17.495667 (+   180us) write_op.cc:270] Start()
0709 12:25:17.495818 (+   151us) write_op.cc:276] Timestamp: P: 1783599917495603 usec, L: 0
0709 12:25:17.495858 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:25:17.496665 (+   807us) log.cc:844] Serialized 587 byte log entry
0709 12:25:17.636642 (+139977us) op_driver.cc:464] REPLICATION: finished
0709 12:25:17.724718 (+ 88076us) write_op.cc:301] APPLY: starting
0709 12:25:17.724833 (+   115us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:17.724965 (+   132us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:17.725120 (+   155us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:17.725466 (+   346us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:17.725813 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:17.725892 (+    79us) write_op.cc:312] APPLY: finished
0709 12:25:17.726232 (+   340us) log.cc:844] Serialized 23 byte log entry
0709 12:25:17.726440 (+   208us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:17.738222 (+ 11782us) write_op.cc:454] Released schema lock
0709 12:25:17.744593 (+  6371us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:17.785791 (+ 41198us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:17.785833 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30071,"reported_tablets":6,"spinlock_wait_cycles":53451648,"child_traces":[["op",{"apply.queue_time_us":87742,"apply.run_cpu_time_us":2507,"apply.run_wall_time_us":61043,"num_ops":1,"prepare.queue_time_us":185933,"prepare.run_cpu_time_us":4285,"prepare.run_wall_time_us":114562,"replication_time_us":140612,"thread_start_us":87538,"threads_started":2}]]}
W20260709 12:25:18.129159  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 178613us
I20260709 12:25:18.128013  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:18.130638  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:18.154657  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 203138us
I20260709 12:25:18.159155  7332 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:17.937110  8000 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.244741  8065 raft_consensus.cc:993] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:18.245522  8065 raft_consensus.cc:1081] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Signalling peer 561a4e21d8b344a79af44568ca80a3c5 to start an election
I20260709 12:25:18.250032  8093 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:18.250645  8093 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:18.251726  8093 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.254535  7318 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:43752
I20260709 12:25:18.255141  7318 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:18.255541  7318 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.296533  6385 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.327634  6384 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:25:18.371950  7314 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.015s
I20260709 12:25:18.372689  7314 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.374135  7331 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:18.375102  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54971.3us
I20260709 12:25:18.377518  6274 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:18.378237  8096 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.385653  6385 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:18.387800  8115 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.440364  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:18.440999  7329 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.457338  6201 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 126) took 1112 ms. Trace:
I20260709 12:25:18.457850  6201 rpcz_store.cc:276] 0709 12:25:17.343119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.343339 (+   220us) service_pool.cc:224] Handling call
0709 12:25:18.455639 (+1112300us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":8}
I20260709 12:25:18.463619  7318 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.467674  7316 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:18.468250  8062 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:18.469404  8062 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:18.469753  8062 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:18.470697  7318 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:18.470657  8062 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.464712  7329 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:18.473007  8062 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.474911  6451 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.478098  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:18.478034  7886 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.481006  8006 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:18.481551  8006 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:18.482478  8006 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:18.471805  8084 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.522s	user 0.019s	sys 0.000s
I20260709 12:25:18.499943  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:18.500831  6454 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:18.538038  6204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 280) took 1037 ms. Trace:
I20260709 12:25:18.539535  6204 rpcz_store.cc:276] 0709 12:25:17.500880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.501024 (+   144us) service_pool.cc:224] Handling call
0709 12:25:17.513405 (+ 12381us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa01830b2 00007f6fa90fc759 00007f6fa9094a76 00007f6fa9065d58 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc
0709 12:25:18.537954 (+1024549us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44011,"reported_tablets":7,"spinlock_wait_cycles":17736448}
I20260709 12:25:18.540742  7314 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:18.539721  6307 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.542914  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:18.553253  6456 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.574649  6376 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.575282  7319 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.579960  6300 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.592625  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:18.593408  6302 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:18.593747  6302 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:18.595703  6380 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:18.594995  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:18.600100  6304 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:18.600471  6304 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:18.604300  6456 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:18.605145  6378 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:18.606823  6454 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:18.612133  6202 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.605818  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:18.617857  6304 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.635673  8091 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:18.637100  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:18.637758  6383 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.632917  7940 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:18.642944  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:18.639416  7940 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:18.649237  8052 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:18.648630  7317 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.663517  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:18.594638  6302 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.629308  6308 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.661806  8052 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:18.648716  7940 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:18.691289  7332 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:18.699226  6304 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.700898  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:18.701665  6456 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:18.702008  6456 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:18.704540  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:18.704967  6302 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:18.706791  6383 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:18.703341  6456 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.700408  8097 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:18.709021  6459 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.712896  8097 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:18.713971  8097 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.683909  8052 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.732277  7943 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:18.733142  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:18.762773  6304 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:18.747431  8117 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:25:18.797324  8117 consensus_queue.cc:1243] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:25:18.797456  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50283.9us
I20260709 12:25:18.809132  6456 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:18.682539  8113 consensus_queue.cc:1175] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 4 status { last_received { term: 3 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:18.839267  6302 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
W20260709 12:25:18.851792  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 79285us
W20260709 12:25:18.856436  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 85868.6us
I20260709 12:25:18.877053  6456 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
W20260709 12:25:18.893638  8137 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.003s
W20260709 12:25:18.896579  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.918196  8135 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:18.905494  6201 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.906739  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:18.919399  6379 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:18.932291  7319 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:18.935178  8117 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:18.938455  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:18.940497  8003 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:18.940999  8003 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:18.943490  6456 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:18.943655  6204 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 4, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:18.941931  8003 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:18.945873  8139 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:18.963883  6202 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.965776  6202 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 2 to 3, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:25:18.974246  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 173403us
I20260709 12:25:18.975334  7319 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:25:18.978688  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 127244us
W20260709 12:25:18.979408  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 125141us
I20260709 12:25:18.967347  6202 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:18.988004  6378 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:18.982321  6202 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:18.989820  8006 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:19.003906  6307 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:19.007557  8041 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:19.015620  8118 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:19.034206  6383 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:19.046854  7317 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.050459  8041 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.069581  6459 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:19.071082  8136 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:19.077800  8091 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.086251  8065 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.089701  7314 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:19.097522  8117 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:19.103922  6459 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.109018  6381 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.110440  7317 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.127167  6301 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:25:19.115566  7314 raft_consensus.cc:2955] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.155769  8126 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.151623  6201 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.166803  7999 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.168601  7332 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.176692  6383 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:19.182368  7928 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:19.184394  6459 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:19.203035  7317 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:19.205586  7989 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:19.206372  6459 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:19.208191  8097 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:19.210924  6201 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.209182  6456 raft_consensus.cc:2955] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.216347  8127 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 12:25:19.220163  6459 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:19.225131  6301 raft_consensus.cc:2955] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.244230  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4db5d1ae5f524ad799ccb78373ce3ff7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:19.247648  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0ddf1d667df04473a598939605e43a1e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:19.249204  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c413b06caa444496aecb13ac2142c246 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:19.264967  6383 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.268621  8139 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:19.273602  6366 tablet_service.cc:1558] Processing DeleteTablet for tablet c413b06caa444496aecb13ac2142c246 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51838
I20260709 12:25:19.275321  8058 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.278069  8127 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:19.283156  7332 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.286095  6451 raft_consensus.cc:2955] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.276600  6204 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 3 to 4, VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) evicted. New cstate: current_term: 1 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:19.312764  6300 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:19.316004  6456 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:19.317997  8097 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:19.323711  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e600ce2bd05948cf8ce5a5f292f57f28 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:25:19.323902  8139 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:19.360749  6371 tablet_service.cc:1558] Processing DeleteTablet for tablet 4db5d1ae5f524ad799ccb78373ce3ff7 with delete_type TABLET_DATA_TOMBSTONED (TS 0bdd5f9c03b84f7289e750ebaab2fda6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51838
I20260709 12:25:19.424072  8159 tablet_replica.cc:333] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:25:19.425411  8159 raft_consensus.cc:2243] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:19.426329  8159 raft_consensus.cc:2272] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:19.464704  6310 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:19.476745  6201 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 3 to 4, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 1 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.478348  6201 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 4 to 5, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:25:19.486088  8093 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:19.491797  8159 ts_tablet_manager.cc:1916] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:25:19.591745  8161 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.113s	user 0.003s	sys 0.003s
W20260709 12:25:19.598604  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50766.3us
W20260709 12:25:19.621598  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84037us
I20260709 12:25:19.635128  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 0ddf1d667df04473a598939605e43a1e with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:59628
I20260709 12:25:19.646512  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet e600ce2bd05948cf8ce5a5f292f57f28 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:57654
I20260709 12:25:19.723387  8166 tablet_replica.cc:333] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:25:19.755594  8168 tablet_replica.cc:333] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:25:19.767275  8166 raft_consensus.cc:2243] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:19.840394  8166 raft_consensus.cc:2272] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:19.768843  8159 ts_tablet_manager.cc:1929] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
W20260709 12:25:19.833395  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76316.3us
I20260709 12:25:19.839234  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:19.864519  6381 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:19.866186  8166 ts_tablet_manager.cc:1916] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:19.871340  8159 log.cc:1199] T c413b06caa444496aecb13ac2142c246 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/c413b06caa444496aecb13ac2142c246
I20260709 12:25:19.873881  8159 tablet_replica.cc:333] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:25:19.891283  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet c413b06caa444496aecb13ac2142c246 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:19.892863  8168 raft_consensus.cc:2243] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:19.893455  8094 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:25:19.893715  8168 raft_consensus.cc:2272] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:19.915241  8168 ts_tablet_manager.cc:1916] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:25:19.943609  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49374.5us
W20260709 12:25:20.095892  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100363us
I20260709 12:25:20.610167  8159 raft_consensus.cc:2243] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:25:20.779172  8159 raft_consensus.cc:2272] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:25:20.806062  8166 ts_tablet_manager.cc:1929] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
W20260709 12:25:20.886682  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.705s	user 0.019s	sys 0.006s
W20260709 12:25:20.887172  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.020s	sys 0.006s
I20260709 12:25:20.860196  8166 log.cc:1199] T 0ddf1d667df04473a598939605e43a1e P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/0ddf1d667df04473a598939605e43a1e
I20260709 12:25:20.890584  8166 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:59628 (request call id 75) took 1256 ms. Trace:
I20260709 12:25:20.905820  8159 ts_tablet_manager.cc:1916] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:25:20.906543  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.003s	sys 0.016s
W20260709 12:25:20.906888  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.003s	sys 0.016s
I20260709 12:25:20.893944  8166 rpcz_store.cc:276] 0709 12:25:19.634581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.634799 (+   218us) service_pool.cc:224] Handling call
0709 12:25:19.722960 (+ 88161us) ts_tablet_manager.cc:1151] Deleting tablet 0ddf1d667df04473a598939605e43a1e
0709 12:25:19.723025 (+    65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:19.914740 (+191715us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:20.804157 (+889417us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:20.890475 (+ 86318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42225,"tablet-delete.queue_time_us":87178,"thread_start_us":88409,"threads_started":1}
I20260709 12:25:20.955958  8174 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:20.957010  8174 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:20.986796  8058 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:20.987692  8058 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.045624  8159 ts_tablet_manager.cc:1929] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:25:21.070801  8179 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:21.076305  8179 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.125133  8175 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.126065  8175 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.129113  8175 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.143186  8168 ts_tablet_manager.cc:1929] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:25:21.144201  8168 log.cc:1199] T e600ce2bd05948cf8ce5a5f292f57f28 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/e600ce2bd05948cf8ce5a5f292f57f28
I20260709 12:25:21.183866  8175 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:21.187378  8094 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.313347  8094 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.316349  8094 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.317026  8094 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.317498  8183 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:21.193826  8136 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.318195  8183 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.319058  8136 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.320592  8185 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.321095  8185 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.314265  8168 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57654 (request call id 78) took 1670 ms. Trace:
I20260709 12:25:21.323434  8175 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.323328  8168 rpcz_store.cc:276] 0709 12:25:19.644362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.646089 (+  1727us) service_pool.cc:224] Handling call
0709 12:25:19.755220 (+109131us) ts_tablet_manager.cc:1151] Deleting tablet e600ce2bd05948cf8ce5a5f292f57f28
0709 12:25:19.755285 (+    65us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:20.860312 (+1105027us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:21.142960 (+282648us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:21.314156 (+171196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38122,"tablet-delete.queue_time_us":107980,"thread_start_us":108821,"threads_started":1}
I20260709 12:25:20.990423  8058 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.322161  8136 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.327137  8136 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.327873  8136 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.330231  8058 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:21.331197  8058 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.317603  8094 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.251432  8174 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.342864  8174 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:21.046334  8159 log.cc:1199] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/wals/4db5d1ae5f524ad799ccb78373ce3ff7
I20260709 12:25:21.349745  8159 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51838 (request call id 77) took 1989 ms. Trace:
I20260709 12:25:21.350317  8159 rpcz_store.cc:276] 0709 12:25:19.360143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.360387 (+   244us) service_pool.cc:224] Handling call
0709 12:25:19.873612 (+513225us) ts_tablet_manager.cc:1151] Deleting tablet 4db5d1ae5f524ad799ccb78373ce3ff7
0709 12:25:19.873652 (+    40us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:20.955517 (+1081865us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:21.045397 (+ 89880us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:21.349629 (+304232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":449606,"tablet-delete.queue_time_us":512306}
I20260709 12:25:21.366062  8185 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.303422  8179 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.306133  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 0ddf1d667df04473a598939605e43a1e (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
W20260709 12:25:21.386917  6190 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 80801.2us
W20260709 12:25:21.306663  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 385920us
I20260709 12:25:21.308382  8181 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.309080  8177 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.311638  8178 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.384588  8187 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.343864  8174 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.412988  8185 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.402969  8179 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:21.424253  8185 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.425705  8179 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.433408  8185 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.433820  8179 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.433991  8185 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.435119  8185 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.407380  8187 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.402415  8181 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.438678  8094 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.439007  8187 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.408960  8182 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:21.410709  8186 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:21.444370  8183 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:21.439970  8058 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:21.424600  8184 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:21.448956  6378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.547s	user 0.015s	sys 0.017s
I20260709 12:25:21.446794  8174 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.403486  8177 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.404903  8178 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:21.454150  6378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.552s	user 0.017s	sys 0.018s
I20260709 12:25:21.429358  8136 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.456338  8183 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:21.465031  8195 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:21.457043  8193 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:21.471303  8195 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.463120  8179 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:21.472975  8179 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.474002  8117 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:21.446274  8182 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.471655  8193 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:21.455031  8186 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.474632  8117 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.480448  8178 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.456102  8058 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:21.480814  8117 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:21.481389  8197 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:21.478353  8177 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.462296  8175 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.462623  8189 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:21.459000  8184 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:21.478053  8193 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.541399  8181 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:21.969854  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.002s	sys 0.004s
W20260709 12:25:21.972992  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.003s	sys 0.004s
I20260709 12:25:21.975000  8184 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.942831  8185 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.067396  8200 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.068177  8200 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.071002  8200 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.073379  8197 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.117139  8197 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:21.463063  8187 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.118039  8187 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.120587  8187 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.121210  8194 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.121906  8194 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:22.122640  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.006s	sys 0.001s
I20260709 12:25:21.466053  8174 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:22.123030  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.006s	sys 0.001s
I20260709 12:25:22.123360  8174 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.081218  8183 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.086158  8208 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.091686  8210 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.091861  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 119) took 2193 ms. Trace:
I20260709 12:25:22.124879  8208 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.124895  8210 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.128715  8174 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.125034  6378 rpcz_store.cc:276] 0709 12:25:19.898921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.900212 (+  1291us) service_pool.cc:224] Handling call
0709 12:25:19.900626 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.901018 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:19.901055 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:19.901659 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.461066 (+1559407us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:21.544139 (+ 83073us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.091493 (+547354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.091578 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.091621 (+    43us) raft_consensus.cc:1685] finished
0709 12:25:22.091638 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.091780 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":510470,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":34555,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":323,"thread_start_us":1646802,"threads_started":3,"wal-append.queue_time_us":848944}
I20260709 12:25:21.475806  8179 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.134821  8191 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.136093  8198 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:21.476715  8182 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:21.459196  8190 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:21.481062  8186 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:22.113042  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.011s	sys 0.014s
I20260709 12:25:22.078958  8189 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.131829  8194 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:21.474390  8195 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:22.135396  8191 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.138412  8209 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.136909  8197 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:22.144706  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.011s	sys 0.015s
I20260709 12:25:22.236578  8205 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.266144  8197 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.347404  8211 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.143534  8190 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.266012  8209 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.348428  8211 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.366245  8200 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.362579  8205 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.247140  8198 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.383915  8213 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.384208  8215 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.394402  8219 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.395260  8219 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.386390  8211 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.378116  8200 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.393047  8194 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.403405  8197 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.403342  8194 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.404023  8197 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.404470  8197 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.405485  8194 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.405982  8194 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.383948  8216 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.406626  8212 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.398566  8219 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.400827  8191 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.401904  8211 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.407255  8214 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.415167  8205 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.415360  8200 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.407114  8189 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.408974  8197 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.406445  8194 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.415946  8210 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.406888  8216 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.384763  8213 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.423434  8211 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.452718  8218 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.385427  8215 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.469247  8201 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.470711  8210 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.423995  8219 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.454121  8205 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.434245  8217 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.433601  8186 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.471151  8197 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.407184  8212 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.475317  8189 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.433233  8214 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.494848  8220 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.473560  8209 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.491151  8186 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.504951  8209 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.490767  8190 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.487171  8218 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.489598  8205 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.505848  8223 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.512444  8213 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.483336  8210 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.516292  8209 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.488049  8219 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.525319  8210 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.496313  8197 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.500666  8189 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.486071  8201 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.501447  8220 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.512538  8186 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.512861  8212 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.544925  8218 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.522396  8224 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.519470  8190 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.501756  8222 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.515134  8221 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.523577  8200 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.502987  8217 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.551225  8198 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.563115  8214 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.552598  8186 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.531579  8223 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.566664  8229 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.567099  8219 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.563709  8214 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.554981  6308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:22.567224  8229 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.559113  8190 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.564195  8221 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.559947  8224 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.572890  8209 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.559161  8222 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.565456  8200 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.570308  8211 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.677758  8220 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.686468  8205 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.696044  8222 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.578883  6308 raft_consensus.cc:606] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:22.696777  8197 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.697595  8189 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.638331  8225 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.707684  8225 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.673530  8226 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.606173  8230 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.699982  8209 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.600705  8194 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.590164  8214 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.700728  8201 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.605767  8219 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.717552  8221 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.718168  8214 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.709676  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.698585  8228 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.719863  8219 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.720841  6201 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 261) took 2831 ms. Trace:
I20260709 12:25:22.636397  8217 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.569092  8186 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.702023  8211 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.714476  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.715655  8223 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.715750  8198 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.718233  8209 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.716408  8230 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.719674  8190 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.720474  8228 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.629456  8216 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.720438  6300 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:22.723496  8194 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.723635  8229 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.722303  6201 rpcz_store.cc:276] 0709 12:25:19.889392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.890124 (+   732us) service_pool.cc:224] Handling call
0709 12:25:22.720738 (+2830614us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20260709 12:25:22.724721  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.728179  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.729020  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.711462  8205 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.731513  6307 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:22.716377  8226 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.852403  6192 catalog_manager.cc:5028] TS 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): tablet 4db5d1ae5f524ad799ccb78373ce3ff7 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:22.854216  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.859054  8190 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.863001  8190 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.740084  8221 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.864813  8226 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.866645  8225 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:22.868057  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.442s	user 0.004s	sys 0.002s
I20260709 12:25:22.740923  8233 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:22.871042  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.448s	user 0.004s	sys 0.004s
I20260709 12:25:22.883006  7940 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.885504  8126 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.885634  7940 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.886246  8126 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.850533  8200 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.739449  8224 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.872372  8215 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.885576  8233 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.878885  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.740056  8234 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:22.789543  8217 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:22.852620  8197 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.864892  8221 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.859493  8235 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:22.789186  8222 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:22.854920  6307 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:22.871516  8194 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.896019  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.859733  8186 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.860255  8205 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.862087  8183 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.739734  8198 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.740000  8211 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.904206  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet e600ce2bd05948cf8ce5a5f292f57f28 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:23.016746  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.945031  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.075060  6300 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:25:23.075987  6381 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:22.897207  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.013515  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:23.073567  6193 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 169389us
I20260709 12:25:22.868072  8219 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.079967  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.086148  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.086524  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.087180  6381 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:23.084710  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:22.942808  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.494s	user 0.002s	sys 0.010s
I20260709 12:25:22.860371  8229 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.079346  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.091717  8229 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:22.956815  8228 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:22.993916  8209 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.081532  8237 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.022274  7937 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.077642  8171 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.097512  8237 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.097662  7937 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.098071  8171 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.099646  8237 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.100131  8171 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.100648  8171 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.100668  7937 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.090359  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:22.972151  6406 debug-util.cc:398] Leaking SignalData structure 0x7b08001694a0 after lost signal to thread 5140
I20260709 12:25:23.103242  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.081388  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:22.888039  7940 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.094733  6304 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:23.095252  6309 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:23.096962  6301 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:23.096733  8222 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.106904  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.080967  8197 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:22.999801  8190 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.096486  8208 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.109642  8197 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.078897  8230 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.079669  8217 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.103585  8171 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.103053  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.113368  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.103276  6305 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:23.096640  8229 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.095948  8235 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.128957  8235 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.129386  8230 raft_consensus.cc:993] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:23.079819  8234 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.130007  8230 raft_consensus.cc:1081] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:25:23.130599  6308 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:23.132596  8234 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.111063  8211 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:22.896417  8126 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.113973  6382 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:23.114284  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.134245  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.134828  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.135147  6382 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:25:23.114662  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.114811  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.116308  8198 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.136370  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.117319  8171 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.118021  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:21.481181  8058 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.120519  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.137970  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.138098  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.138821  6382 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:23.124245  8194 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:23.122370  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.697s	user 0.003s	sys 0.006s
W20260709 12:25:23.110189  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.671s	user 0.008s	sys 0.018s
W20260709 12:25:23.140396  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.715s	user 0.005s	sys 0.006s
I20260709 12:25:23.139181  7928 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.141124  7928 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.135581  6300 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:23.137130  6307 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:23.138394  6379 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:23.142817  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.146783  8198 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:23.148067  7928 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.148142  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.149003  6300 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:23.149309  8217 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.149873  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.151153  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.152098  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.152324  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.152851  6307 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:23.153059  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.153690  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.218465  8164 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.243901  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.245071  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.226864  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.163112  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.248265  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:23.165632  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.743s	user 0.000s	sys 0.005s
I20260709 12:25:23.243649  8164 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.154979  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.154458  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:23.253058  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 98671.6us
I20260709 12:25:23.195120  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.254195  6305 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:23.196233  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.255182  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.256532  8221 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.256628  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.257545  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.257625  8208 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:23.258289  8208 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.258760  8208 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:23.259389  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800167b80 after lost signal to thread 6240
I20260709 12:25:23.257769  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.262012  8222 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.262236  8197 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.262651  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.236989  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.262646  8197 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.265345  8197 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.265368  8207 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.265957  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.272249  8164 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.272636  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.162642  8235 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.273339  6302 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:23.273494  8235 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.274000  8235 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.276736  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.266027  8207 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.263832  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.264367  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.166934  8186 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.163079  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.175611  8211 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.286900  8211 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.287353  8211 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.287204  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.288335  8233 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.288580  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.289150  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:23.283032  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.861s	user 0.005s	sys 0.016s
I20260709 12:25:23.249099  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.154461  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.286226  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.292817  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.294231  8183 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.294158  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.285524  8190 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:23.294561  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.295780  8190 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.294824  8183 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.296367  8171 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.296021  7314 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:23.296986  8183 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.297116  8171 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.297808  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.298439  7314 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:23.299260  8209 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.299664  8209 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.300088  8209 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.299747  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.301929  8115 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.296326  8190 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:23.302569  8115 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.303426  8237 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.303931  8237 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.304486  8186 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.290141  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:23.301362  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.162774  8058 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.403795  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.404932  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.405690  6381 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:23.406597  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.407590  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.377342  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.408671  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.409200  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.409709  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.410326  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.296011  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.410779  8180 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.410292  6381 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 1.
I20260709 12:25:23.411569  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:23.411342  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115406us
I20260709 12:25:23.399541  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.404902  8171 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.411576  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.282549  8207 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:23.294854  7285 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.520s (SENT) [suppressed 2 similar messages]
I20260709 12:25:23.413529  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.415455  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.416657  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.301872  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.418869  7331 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:25:23.419929  6376 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:23.421167  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.421834  6379 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:23.422598  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.425371  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.425371  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.428107  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.418273  8058 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.411409  8180 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.419013  7928 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.430634  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.428654  7331 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
W20260709 12:25:23.415066  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92528.9us
I20260709 12:25:23.423336  8227 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.420248  7314 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:23.439158  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.447470  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.449982  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.450066  8115 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.432227  8058 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.436825  8216 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.436439  8239 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.460829  8239 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.445659  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.463951  8239 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.445895  8223 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.464789  8223 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.465278  8223 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.457737  8058 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.472174  8186 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.459717  8180 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.460208  8216 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.445833  8227 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:23.490408  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 195574us
I20260709 12:25:23.445461  7928 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:23.510107  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50235us
W20260709 12:25:23.509428  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64810.1us
I20260709 12:25:23.455106  8198 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.509743  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.490650  8216 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.492697  8227 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:23.516042  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 77144.6us
I20260709 12:25:23.517499  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.517134  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.519366  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:23.524380  6422 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.113s	user 0.008s	sys 0.024s
I20260709 12:25:23.540390  7329 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
W20260709 12:25:23.531008  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.121s	user 0.015s	sys 0.020s
I20260709 12:25:23.507200  8237 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:23.542265  6422 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.133s	user 0.010s	sys 0.025s
I20260709 12:25:23.513513  8198 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:23.543794  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.135s	user 0.016s	sys 0.020s
I20260709 12:25:23.542317  7317 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:23.520910  7928 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.529410  8208 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.545912  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:23.528173  8197 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.535470  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.555764  7316 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:25:23.556723  8208 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.549198  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:25:23.557667  6450 raft_consensus.cc:606] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.525851  8058 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.563371  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.564942  8197 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.565140  8125 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:23.568666  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.558223  8223 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.573284  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.574148  6450 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:23.574863  8223 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.575554  8223 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.575917  8223 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.576280  8223 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.577553  8209 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.579630  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.579921  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:25:23.580242  6456 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:23.566000  8125 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.582057  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:23.583056  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:23.582072  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.584167  6454 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:23.583690  6376 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:23.557878  8190 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.529829  8211 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.571822  8186 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.586834  8186 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.587253  8186 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.588025  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.520257  8183 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:23.589897  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800343960 after lost signal to thread 6334
I20260709 12:25:23.594805  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.591245  8209 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.587906  8125 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.582713  7928 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.598217  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.598522  7928 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.598603  8207 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.595082  7940 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.600023  7940 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.600729  7928 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.601928  7940 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.604672  8211 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.604773  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.605264  6379 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.606045  8180 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:23.594915  6406 debug-util.cc:398] Leaking SignalData structure 0x7b08003756a0 after lost signal to thread 6335
I20260709 12:25:23.606668  8180 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:23.607522  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800375ec0 after lost signal to thread 6409
I20260709 12:25:23.608036  8124 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.608582  8124 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.609445  8180 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.610467  8124 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.611009  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.615044  6382 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:23.616461  6381 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.617372  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.617976  6451 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:23.595367  8235 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.639986  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:23.642228  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73604.8us
I20260709 12:25:23.644429  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.646845  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.648506  8222 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:23.648391  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.649029  8222 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.649401  8222 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:23.648000  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.650760  6450 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:23.651863  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.652758  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.653313  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.654424  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.656291  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.656595  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.657174  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.658267  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.660115  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:23.660594  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.661303  6454 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:23.661577  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.662999  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.663343  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.664287  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.663894  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.666082  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.667785  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.665037  8058 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.672354  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.672562  8233 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:23.673233  8233 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.673588  8233 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:23.673466  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:23.672514  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.671681  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:23.675752  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.653546  8191 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.675752  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.677274  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.677520  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:23.679177  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.679265  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.680749  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.681013  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.682684  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.685478  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.674201  8183 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:23.687282  6406 debug-util.cc:398] Leaking SignalData structure 0x7b080026d800 after lost signal to thread 6483
I20260709 12:25:23.687500  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.688553  8245 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:23.688851  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.689070  8245 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.689138  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.682180  8208 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:23.690393  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.691671  8208 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.692157  8208 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:23.692734  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.693885  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.694017  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.689965  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:23.692770  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.687532  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:23.678099  8209 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.695840  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.695914  8209 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.690632  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.696480  8209 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.696439  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.696952  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.697160  8200 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.698019  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.698108  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.696986  6458 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:25:23.699427  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.700690  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.702322  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.703312  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.683096  8058 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.703979  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.704469  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.706120  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.689694  8197 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.668843  8221 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:23.706661  8223 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.707369  8221 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.707484  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.707906  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.708596  7331 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:23.707036  8197 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:23.711709  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.712357  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.675818  8211 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.715595  6379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:25:23.716354  6379 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:23.716703  6379 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.713323  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.710865  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.718080  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.719622  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.720875  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.722802  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.680544  8191 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:23.723021  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.690215  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.707921  8221 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.726410  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.726737  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.726904  8207 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.714162  8197 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.726155  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.737643  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.704201  8058 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.739900  8190 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.740314  8058 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.731724  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.744607  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.745784  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.745609  8198 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.734948  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.747081  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.747340  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.747664  6309 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:23.727568  8235 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.709568  8245 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:23.738040  6380 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:23.754168  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.754177  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.734102  6379 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.729259  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.719130  8211 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.739574  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.724640  8191 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:23.722566  6406 debug-util.cc:398] Leaking SignalData structure 0x7b08003691c0 after lost signal to thread 7371
I20260709 12:25:23.730008  8227 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.760685  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.752519  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:23.760010  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800369200 after lost signal to thread 7373
I20260709 12:25:23.761740  6450 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:23.764022  6379 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.799734  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.833407  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:23.834192  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.834954  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.815948  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.838811  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.794715  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.841850  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.800447  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.793699  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.789106  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.783102  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:23.849593  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 129496us
I20260709 12:25:23.851069  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.851969  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.852031  8216 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.852298  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.794387  8058 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.853982  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.854684  6377 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:23.856027  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.868232  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.854790  8058 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.869233  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.766598  6308 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:23.872162  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.874027  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.874974  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.875175  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.876199  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.876626  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.877319  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.878340  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.878799  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.878894  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.879827  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.881043  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.882805  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.881819  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:23.883826  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800369460 after lost signal to thread 8140
W20260709 12:25:23.884352  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800369560 after lost signal to thread 8168
I20260709 12:25:23.884272  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.884862  6309 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:23.885879  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.879038  6457 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:23.886792  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.887992  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.888546  6305 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:23.888880  6305 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:23.781237  8205 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:23.889259  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.889685  6305 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:23.854350  8216 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.855026  8186 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.874313  8221 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.891134  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.904673  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.905550  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.892565  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.907028  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.907251  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.893458  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.908196  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.909170  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:23.910696  8186 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:23.910912  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.911512  7314 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.912256  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.895808  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:23.908198  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.913417  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.895936  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.914786  8221 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.758951  8211 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.873481  8208 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.925278  8208 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.854315  8197 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.793203  8227 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.928864  6376 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.868824  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:23.901315  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.929788  6380 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:23.910251  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:23.931129  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.902906  6305 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:23.903213  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.895272  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:23.911163  6382 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:23.914203  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.895829  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.934569  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.914808  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.935931  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.910724  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:23.815265  8223 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:23.917243  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.917877  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.918646  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.902688  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:23.941788  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.938905  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.942884  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.796089  8198 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.946162  6384 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:23.943161  6379 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:23.943274  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.948325  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:23.945053  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.946751  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.941138  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.950551  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:23.941882  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.951311  6383 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:23.951849  6382 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:23.952540  6383 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:23.953716  6383 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:23.948251  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:23.957681  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.955572  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.959291  6206 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 287) took 4038 ms. Trace:
I20260709 12:25:23.959684  6206 rpcz_store.cc:276] 0709 12:25:19.897339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.677101 (+2779762us) service_pool.cc:224] Handling call
0709 12:25:23.935834 (+1258733us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33040,"reported_tablets":1}
I20260709 12:25:23.962287  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 1831) took 1092 ms. Trace:
I20260709 12:25:23.962682  6385 rpcz_store.cc:276] 0709 12:25:22.869876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.869993 (+   117us) service_pool.cc:224] Handling call
0709 12:25:22.870745 (+   752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.095839 (+225094us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.095867 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:23.095902 (+    35us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:23.095936 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.962025 (+866089us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400065254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:23.962073 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.962202 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":517275008}
I20260709 12:25:23.956161  8136 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.956317  7940 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.965458  8245 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.966434  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:23.933753  8185 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.976830  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.977815  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.977568  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:23.937315  8172 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:23.978756  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:23.900624  8233 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:23.981160  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.944754  8180 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.944168  8227 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:23.981078  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.985136  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:23.980365  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.941459  8197 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:23.983877  6381 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:23.976171  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.983590  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.986367  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.982016  8163 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:23.967567  7928 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:23.982180  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.989524  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.969087  8093 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:23.995393  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.980630  8185 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.983841  8183 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:25:23.997375  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 94905.6us
I20260709 12:25:23.982406  8172 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.001928  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:23.999583  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.999801  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.004590  6450 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.003937  8124 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.007956  8124 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:24.008339  8124 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:24.003878  8171 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.008985  8124 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:23.991334  8227 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:23.996433  8180 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.011801  8227 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.000903  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:23.987123  7937 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.012739  7937 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:24.013154  7937 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:23.987493  8136 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.002774  6302 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:23.998853  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.011245  8221 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.009861  8211 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.158737  8221 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:24.159250  8221 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.161273  6450 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:24.161996  8211 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:24.162385  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 147565us
W20260709 12:25:24.072789  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 70958us
I20260709 12:25:24.163867  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.032809  8222 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:24.032337  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52040.3us
I20260709 12:25:24.056828  8173 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.166184  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.033084  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.126909  8124 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:24.126022  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109738us
I20260709 12:25:24.125730  8246 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:24.166147  8222 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:24.174032  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 175247us
I20260709 12:25:24.175333  8253 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:24.175791  8253 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:24.176156  8253 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.010777  8172 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.010651  8197 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:24.129770  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.198684  8242 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.199105  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.130095  8245 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.031209  8191 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.199683  7329 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.201361  8191 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.198731  8197 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.202425  8197 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.203207  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 122) took 1348 ms. Trace:
I20260709 12:25:24.132721  7937 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.136448  6305 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:24.149252  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:23.998749  8185 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.130180  8227 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.125517  8209 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:24.182928  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 232445us
I20260709 12:25:24.183159  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.176657  8124 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.181082  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.188537  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.177382  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:24.222393  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45075us
I20260709 12:25:24.223376  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:24.223546  7937 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:24.224937  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.193457  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.226619  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.226795  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.227283  6381 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:24.228652  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.194978  8254 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.195994  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.229352  6377 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:24.196740  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.197556  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.235360  8209 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.198215  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.188911  8246 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:24.236783  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.236737  8246 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.133625  8233 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.238875  8233 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:24.238968  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57986.8us
I20260709 12:25:24.202042  8221 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.240092  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.241122  8221 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.241681  8172 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.241954  8185 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.242864  6378 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.242709  8191 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:24.243860  8227 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.245371  8249 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:24.245714  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.246441  8249 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.246853  8249 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:24.246917  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.247233  8217 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:24.247658  8239 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:24.248075  8239 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:24.248435  8239 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.242888  6381 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.214197  7329 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:24.218089  8136 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:24.251387  8136 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:24.253314  8136 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.253676  8172 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.242210  8221 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.230396  6307 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:24.230859  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.259233  6376 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:24.231419  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.206065  6302 rpcz_store.cc:276] 0709 12:25:22.854424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.792415 (+937991us) service_pool.cc:224] Handling call
0709 12:25:23.792713 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.157508 (+364795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.202916 (+ 45408us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54002076d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:24.203142 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463805184}
W20260709 12:25:24.211459  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62314.3us
I20260709 12:25:24.265584  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.235532  7317 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:24.237802  6377 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:24.132680  6202 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 134) took 3218 ms. Trace:
I20260709 12:25:24.274150  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.220203  7928 raft_consensus.cc:993] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:25:24.276681  7928 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:24.277026  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.277480  8254 raft_consensus.cc:1081] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Signalling peer 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:25:24.277733  7928 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:24.278811  8254 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.279278  8254 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.280049  8254 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.281384  8254 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:24.213147  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 243537us
I20260709 12:25:24.220306  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:24.283564  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63340.2us
I20260709 12:25:24.278825  7928 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.252671  8253 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.270797  8163 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:24.271528  8093 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.260680  8249 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.274071  6202 rpcz_store.cc:276] 0709 12:25:20.914759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.060734 (+1145975us) service_pool.cc:224] Handling call
0709 12:25:24.132608 (+2071874us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235187,"reported_tablets":5}
I20260709 12:25:24.250651  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.276065  8200 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:24.284644  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.285216  8233 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.252605  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:24.302289  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49757.8us
I20260709 12:25:24.302733  8249 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.303115  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.306273  8093 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:24.306762  8093 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.306963  7937 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.307379  7937 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:24.282819  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.306660  7332 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.308905  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.307569  8173 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.308002  7940 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.309837  8173 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:24.310727  8198 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.311154  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.310902  8173 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.284608  7928 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.312353  8173 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.282557  8136 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.313241  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.315475  8197 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.316277  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.287076  8242 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.286861  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:24.318821  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44771.7us
I20260709 12:25:24.320147  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.290926  8217 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.311206  8198 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.312597  7928 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.307796  7937 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.325860  8207 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.334069  8207 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.334518  8207 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.330771  8198 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.341615  7937 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.329813  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.319248  8200 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.324110  8222 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:24.349615  8200 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.328377  7940 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.333096  8237 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:25:24.350670  8136 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.324158  8227 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.351053  8237 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.328969  8217 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:24.351539  8237 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.351681  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.352041  8185 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.352468  8237 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.353717  8237 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.354166  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.333411  7928 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.332000  8252 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.343165  8253 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.328006  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.337854  8093 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.348217  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.346771  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:24.349710  8191 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.349756  8222 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.358560  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.367436  8191 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.368324  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.368463  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.368887  8222 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.346287  8197 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.369678  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.365873  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.370761  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.371302  8197 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.371443  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.362988  8207 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.365530  8211 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:24.372498  8211 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:24.373605  8207 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.373375  8211 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.374256  6302 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.374956  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.360723  8200 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.378402  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.371449  8233 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:24.377858  8191 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.376422  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.353303  8217 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.376432  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.371608  8252 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.386101  7332 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.366582  8242 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.387029  8197 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:24.387434  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.388899  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.390275  8197 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.382512  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:24.383560  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.384430  8239 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.393038  8239 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.393635  8239 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:24.394603  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.364218  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.395826  7937 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.396399  8242 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.394708  8239 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.397161  8239 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.397442  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.398259  8239 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:24.399118  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.398313  8222 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.399887  8239 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.396117  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.401288  8222 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.401815  8198 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.402658  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.403447  7937 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.403761  7928 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:24.404489  8227 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:24.403954  8198 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.405048  8227 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:24.406114  7940 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.406872  7940 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:24.407876  8253 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.408259  8253 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.408948  8253 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.406116  8227 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.408224  7928 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.410432  7928 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.410527  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.411517  7928 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.411895  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.412809  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.415527  8253 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.410588  7317 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.416052  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.416769  8249 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.417124  8228 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.418720  8242 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.417416  8249 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.419137  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.422165  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.423003  8217 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.423223  8242 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.423578  6302 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:24.424467  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.425558  8093 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.425714  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.426585  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.427248  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.427639  6348 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.429315  8252 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.425897  8207 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.431808  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.433815  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.435011  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:24.428894  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.408883  6202 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 5, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.436879  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.437539  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.434713  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.438175  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.441565  8198 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.441519  8242 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.451359  7940 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.450846  8222 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.454967  8207 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.461263  8200 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.465487  8200 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.431747  8197 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.473099  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.430974  8233 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:25:24.450975  8254 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.486406  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.445529  7928 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.436266  8190 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.436673  8221 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.493326  8233 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.494544  8243 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.489959  8246 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.488355  7332 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:24.498795  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63841.8us
I20260709 12:25:24.513747  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.506803  8222 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:24.488754  8058 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:25:24.519665  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.507441  8233 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:24.522145  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.511643  8246 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:24.524442  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.516861  8227 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.521567  8221 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.530957  6378 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.520030  8222 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:24.534098  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.534157  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.525727  8246 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.539058  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.535841  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.525285  8198 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.517182  8253 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:25:24.540829  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.540910  8233 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.536664  8222 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.552435  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.539961  8216 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.554373  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.557704  8211 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.558665  8227 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.560422  8200 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.560849  8200 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:24.562330  8228 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.564016  8233 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.582692  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.553246  8251 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.586647  8251 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.587932  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.547372  8235 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.584411  8222 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:24.590973  6275 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2483d7801887403e873adc62429fe046. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:24.592875  8235 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.594309  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.596288  8222 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.598905  8235 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.600184  8254 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.600210  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.601087  8254 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.601472  8235 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.602949  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.603828  8245 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.605114  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.605865  8245 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.607170  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.607853  8245 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.610021  8252 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.598018  8200 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.611841  8200 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.612704  8254 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.616264  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.650938  6451 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.651223  8058 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.626228  8246 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:24.630115  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 105776us
I20260709 12:25:24.652150  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.652617  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.652937  8058 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.653877  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.654024  8246 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:24.654947  8260 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.655167  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.656622  8260 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.657105  8261 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.656996  8173 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:24.645507  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.646333  6381 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:24.656244  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.658797  8173 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:24.659946  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:24.659993  8136 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:24.660699  8136 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:24.660780  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.661134  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.661823  8233 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.662142  8185 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.662565  8233 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:24.662966  8233 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:24.663242  8252 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:24.663323  8233 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.658679  8093 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:24.655067  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.664621  8093 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:24.665439  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.666172  7928 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.665858  8093 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.666721  8253 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.667198  8093 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:24.666944  7928 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:24.668327  8237 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.668702  8237 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.669530  8237 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.670804  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.671660  7940 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:24.672901  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.673578  8237 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.673969  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.674265  8247 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.657626  8239 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:24.673616  6451 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.675119  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.675237  8239 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:24.675700  8239 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.676595  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.677495  8124 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.678236  8163 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:24.662040  8136 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.679692  8136 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:24.679582  8185 consensus_queue.cc:1175] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4268023ecea149cd83cba77db53c3100" responder_term: 7 status { last_received { term: 6 index: 7 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:24.676543  8239 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.682273  8239 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.683462  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.690099  6451 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:24.690706  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.690912  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.692337  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.694926  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.695797  8238 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.696674  8238 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:24.696537  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.697160  8238 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.697454  8233 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.698662  8172 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.699415  8172 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.698027  8238 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.701198  8238 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.703783  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:24.703718  8124 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:24.704802  7940 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.705354  8237 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.705766  8237 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.706135  7940 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:24.707135  7940 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:24.706712  8237 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.707517  7940 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:24.708137  8237 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.698318  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.708374  7940 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.709811  7940 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.710314  8173 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.711795  8173 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:24.714962  8254 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.708137  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.717236  8233 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:24.731645  8252 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:24.732568  6302 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.717887  6453 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:24.720468  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.720374  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:24.722394  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.738440  8267 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:24.723048  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.744575  6450 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:24.751187  6451 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.755913  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.757522  6381 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.763909  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.764202  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.771171  6455 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:24.774425  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.774627  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.777417  6381 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:24.778743  6454 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:24.781831  6376 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.782018  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.783602  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.785454  6376 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:24.787592  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.790217  6385 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.791977  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:24.792740  6452 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.793803  6307 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:24.800920  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:24.802659  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.802999  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:24.804911  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.807060  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.807324  6305 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:24.808780  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.810448  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.811403  6305 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:24.812770  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.813009  8264 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:24.814059  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.814942  6307 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:57668
I20260709 12:25:24.815210  6381 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:24.817783  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:24.819828  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.820804  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:24.821604  6377 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:24.821477  8254 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.827082  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.828817  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.827813  6304 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:24.832273  6304 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:24.834707  6304 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:24.837281  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 143617us
I20260709 12:25:24.838287  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:24.842993  6304 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.843502  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.844485  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:24.816747  8264 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:24.838095  8235 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:25:24.846604  8264 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.847756  8264 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:24.848598  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.850234  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.852131  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.830647  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.849416  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.857337  8261 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.857728  8209 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.849406  8197 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.858126  8209 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:24.853255  8252 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:25:24.859454  8252 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:24.859176  8209 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.859845  8252 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:24.860638  8209 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.860939  8252 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.861599  6302 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
W20260709 12:25:24.845507  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 152647us
I20260709 12:25:24.882709  6310 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:24.862375  8252 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:24.905221  8185 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.905922  8185 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:24.854846  8124 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.907294  8185 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:24.855127  8237 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.908631  8266 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:24.908743  8163 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:24.909596  8200 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:24.909637  8266 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:24.908948  8185 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.910986  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:24.911458  8266 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:24.913306  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.916441  8058 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:24.915542  8093 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:24.926817  8093 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:24.927277  8093 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:24.931211  6303 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:24.918246  8058 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:24.933887  8058 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:24.935823  8058 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:24.937840  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:24.917326  8265 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:24.917598  8267 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:24.856676  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:24.863358  6307 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:24.943303  6307 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:24.943739  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 87132.9us
I20260709 12:25:24.870069  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:24.945057  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75077.2us
I20260709 12:25:24.863668  8208 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:24.946202  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.952346  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.952569  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:24.954015  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.956035  6201 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 3, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.958385  6201 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 4, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:24.954469  6305 raft_consensus.cc:2376] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4. Re-sending same reply.
I20260709 12:25:24.973910  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.946686  8216 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.977280  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:24.982398  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:24.984272  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 192357us
I20260709 12:25:24.952991  8209 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:25:25.044916  8163 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.045552  8163 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.046464  8163 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.047192  8173 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:24.981528  8191 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:25:25.047675  8173 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.048518  8173 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.049742  8173 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.054557  8198 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:25.055359  8221 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:25.056607  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76629us
I20260709 12:25:25.056666  6307 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.061425  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:25.062245  8237 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:25.014655  7332 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:25.067289  6305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2483d7801887403e873adc62429fe046"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:24.962898  8217 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:25.068181  6305 raft_consensus.cc:606] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:25.069674  6307 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:25.070134  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49686.7us
W20260709 12:25:25.073455  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 228999us
I20260709 12:25:25.074385  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:24.979516  8246 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.076125  6377 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:25.079511  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95757us
I20260709 12:25:24.980234  8250 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:24.984061  8216 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:24.985499  8249 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.081152  6455 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:25:25.088908  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.089573  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.119052  6378 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:25.118471  6309 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:25.135203  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:25.058861  8228 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
I20260709 12:25:24.986676  8243 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
W20260709 12:25:25.181061  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50749.7us
I20260709 12:25:25.088606  8267 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.085517  8217 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:25.222119  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 81916.9us
I20260709 12:25:25.088397  8211 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 12:25:25.119837  8266 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
W20260709 12:25:25.235484  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 98406.2us
I20260709 12:25:25.138245  8237 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 12:25:25.258776  6301 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:25.232059  7317 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:25.290117  8163 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.290715  6379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:25.291424  6379 raft_consensus.cc:606] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:25.291747  6379 raft_consensus.cc:614] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:25.292707  8271 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:25.296036  8265 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:25.296506  8265 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.297354  8265 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.298777  8265 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.299263  8221 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:25.299824  8221 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:25.300848  8221 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.305748  8198 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.306170  8198 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:25.307102  8198 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:25.309087  8198 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:25.250440  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 115332us
I20260709 12:25:25.311822  8163 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:25.313781  8163 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.314136  8163 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.314766  8235 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:25.315171  8254 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:25.315011  8163 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.315685  8254 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:25.316654  8254 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:25.317834  8254 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:25.318529  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.319985  6378 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:25.324434  8227 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:25.324924  8227 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:25.319001  6206 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:25.325784  8227 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.327157  8227 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.328008  8163 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.258095  8216 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:25:25.333961  8246 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:25.343905  6455 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:25.357895  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.362254  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 281801us
I20260709 12:25:25.365000  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:25:25.377738  8093 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.364185  8217 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:25.383138  8093 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:25.371702  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:25.385881  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:25.386190  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:25.386713  6378 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:25.388005  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:25.398213  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:25.398551  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:25.399120  6452 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:25.400758  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:25.399600  6452 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:25.403311  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:25.405237  6452 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.364076  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:25.414613  7317 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:25.419708  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:25.421224  6309 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:25.398579  8275 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.363036  8207 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:25:25.402829  8093 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.404470  8136 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:25.425627  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:25.424242  8124 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:25.422566  8271 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:25.430267  8271 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:25.430718  8271 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:25.431035  8136 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:25.431481  8136 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:25.423158  8270 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:25.437120  8270 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.438203  8270 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.441094  8271 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.419442  8267 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:25:25.443099  8267 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:25.443486  8267 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.443639  8271 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:25.444345  8267 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.444809  6454 raft_consensus.cc:606] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:25.445370  6454 raft_consensus.cc:614] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:25.445802  8267 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:25.410069  8238 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:25.450877  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:25:25.419760  8266 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:25:25.452582  8185 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:25.452818  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:25.452317  6201 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:25.453038  8185 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.453711  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:25.453917  8185 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.455166  8185 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.457096  7317 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:25.457811  6452 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:25.457949  6303 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:25.461489  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:25.461620  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:25.461781  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:25.462352  6458 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:25.463624  7329 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:25.471844  6303 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:25.464128  8093 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:25.470202  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:25.480417  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:25.496338  8198 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:25.495927  8163 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.496138  8271 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:25.513212  8198 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:25.500254  8093 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.525179  6452 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:25.531054  8261 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:25.531562  8261 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:25.532475  8261 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.532277  8093 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.535010  8198 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.537166  6202 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:25.519017  8163 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.544691  6452 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:25.550896  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:25.523165  8271 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.552891  6377 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:25.550944  8265 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 12:25:25.561458  8261 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:25.563819  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:25.564834  6303 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:25.572037  8221 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.587191  8271 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.592792  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:25.549504  8163 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:25.617919  8265 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.632961  6377 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:25:25.630896  8265 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.636090  8265 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.650708  8163 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:25.655151  8284 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:25:25.680711  8136 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.683686  6204 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:25.714815  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:25.723496  8136 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:25.724958  8265 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.726184  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:25.727134  6378 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:25.733596  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:25.734252  6301 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.741737  8238 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.742403  8238 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.744782  8238 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:25.746685  6301 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:25.747138  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:25.748508  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:25.766685  6202 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 4, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:25.769716  8239 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.769119  6202 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:25.770313  8239 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:25.760099  8136 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:25.756958  8227 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.761924  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:25.778509  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:25.778810  8265 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:25.783367  6378 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.783324  6303 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:25.788357  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:25.786662  7316 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:25.784416  8239 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:25.855931  6379 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:25.785732  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:25.856886  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:25.856460  6206 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 2, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:25.860385  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:25.861138  6309 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:25.862118  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:25.783571  8227 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:25.868328  8227 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:25.869490  8238 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:25.869985  8238 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.870333  8238 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:25.870867  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:25.871541  6458 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:25.873020  8267 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:25.873600  8267 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.874575  8267 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.873064  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:25.875972  8216 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:25.876391  8216 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.876736  8216 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:25.897020  8295 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:25.898782  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:25.899462  6303 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:25.878520  8239 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:25.900552  8239 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.900911  8239 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.782828  8136 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:25.906414  6378 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:25.918295  8216 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:25.942914  8254 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.943579  8254 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:25.946158  8254 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:25.947374  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:25.947978  8136 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.950096  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:25.950853  6378 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:25.951891  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:25.956857  8208 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.957917  8208 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.960038  8208 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:25.960105  8216 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:25.962664  8254 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.963276  8254 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.963708  6309 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:25.968025  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:25.968688  6378 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:25.969631  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:25.968178  8250 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.969539  8265 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:25.970638  8250 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.973150  8250 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:25.973417  8239 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.978751  6458 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:25.978948  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:25.982973  8228 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:25.985996  8302 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:25.986449  8302 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.986850  8302 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.988205  8228 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.988729  8228 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:25.991180  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:25.991894  6309 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.994422  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:25.995018  7332 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:25.981542  7316 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:25.996722  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:25.998260  8198 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:25.998723  8198 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.999080  8198 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:25.999259  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.000384  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.001083  6305 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:26.003789  7332 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:26.007328  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.017969  6378 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:26.018230  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:26.011201  6381 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:26.003873  6379 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:26.005049  6309 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:26.022245  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:26.026391  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56844.3us
I20260709 12:25:25.991748  8239 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.033562  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:26.037545  8216 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:26.035914  8227 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:26.043238  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.035899  8305 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.026273  8185 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:25:26.005761  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.043830  6381 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:26.047911  6305 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.048796  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:26.046415  8305 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.038245  8227 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.058743  8305 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.058837  8227 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.056782  8222 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:26.059608  8305 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:26.059677  8222 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.060127  8222 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.054024  8271 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:25:26.038195  8216 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:26.065263  8216 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.056559  8274 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 12:25:26.067703  7317 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:26.033727  8254 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.060128  8305 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:26.057439  8239 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:26.074311  8239 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.074716  8239 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.077939  6377 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:26.081208  6452 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.083936  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.086443  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:26.087098  6452 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.084745  6377 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:26.088595  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.106019  6452 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
W20260709 12:25:26.110596  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 92443.3us
I20260709 12:25:26.113113  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.114027  7332 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:26.082003  8198 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.084985  8228 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.122973  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.081914  8305 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.107164  8254 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:26.116158  8136 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.138695  8136 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.141402  8136 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.148615  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.116667  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.088058  8266 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:25:26.152103  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.153599  6303 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:26.094399  8270 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:25:26.156643  6377 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:26.158627  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:26.136420  8254 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.132313  8302 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.160048  7317 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:26.172271  8254 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.156461  8238 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.197196  6452 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.162077  8267 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:25:26.181697  8246 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.207543  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:26.208436  6377 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:26.186647  8200 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.209806  8200 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.207520  8246 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.211979  8200 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.212651  8246 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.192101  8136 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:26.202620  8228 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:26.215492  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.216235  6455 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:26.217299  8136 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.217535  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.217769  8136 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.218768  8227 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.220321  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.220155  6206 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.220932  6309 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:26.224602  8238 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.230501  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.203187  8239 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.231822  8302 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.234932  8227 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.239670  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.240248  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.240798  6303 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.241415  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.242297  6307 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.239598  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:26.243835  7332 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:26.253396  8266 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:26.253823  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.253957  8266 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.255775  8245 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:26.256160  8245 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.256515  8245 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.257560  8266 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.261685  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:26.262825  6381 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.237382  8239 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.263209  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:26.263890  6378 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.264149  8198 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.265115  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:26.265689  6379 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.271117  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.272219  6452 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:26.273762  8246 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.273600  8274 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.293798  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:26.290444  6381 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.284199  6307 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:26.290441  7332 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:26.292111  6378 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:26.280031  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:26.292001  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.304351  6376 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:26.304965  8261 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.313395  8261 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.313608  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:26.314535  7317 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:26.315645  8261 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.316701  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:26.317348  7316 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.321233  8211 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:26.321722  8211 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.322021  8208 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.322811  8208 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.324931  8208 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.325500  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.322072  8211 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.326522  8302 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:26.333915  6303 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:26.332237  7332 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.282622  8221 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.350641  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:26.339526  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:25:26.358875  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78951.5us
I20260709 12:25:26.363500  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.367949  6379 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:26.304742  8239 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:26.377138  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:26.378453  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:26.370666  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.359352  7316 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.329195  8308 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.389909  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.398452  6376 raft_consensus.cc:2368] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:26.292634  8222 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.304113  8246 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.404327  8246 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.311098  8274 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.377147  8185 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:26.376971  8307 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:26.407433  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.376161  8302 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:26.393983  8207 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:26.376590  8250 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.328286  8245 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.383561  8228 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:26.424484  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47418.4us
I20260709 12:25:26.413421  6451 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:26.415104  8305 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.419420  6305 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:26.352849  8254 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.373435  8221 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.393702  8239 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:26.430233  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53216.5us
I20260709 12:25:26.428082  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:26.440183  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 138254us
I20260709 12:25:26.455010  6274 leader_election.cc:400] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:26.411303  8185 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:26.457587  8185 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.429749  8216 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.397694  8303 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:26.432605  8207 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:26.444105  8238 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.460848  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.415805  8308 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.456722  6274 leader_election.cc:403] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.454985  8058 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.425500  8250 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.474159  8322 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:25:26.477433  8198 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:26.429610  8211 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.433190  8228 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.459968  8243 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:26.487684  8323 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:26.442370  8305 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.519289  6452 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:26.521871  8228 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.522450  8243 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.522936  8243 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.525524  8308 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.526678  8323 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.529224  8323 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.529783  8323 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:26.530324  8207 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:26.531381  8316 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:26.530252  8323 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.534966  8319 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.483289  8058 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.534611  8323 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.540179  8250 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.541457  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.541849  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.542111  6303 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:26.542516  6301 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.542856  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.543615  6310 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:26.545576  8211 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.546099  8198 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:26.547080  8198 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.549635  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.558925  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.550311  6458 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:26.474372  8271 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.596230  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.538254  8058 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.580590  8320 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.550782  8163 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.582135  8324 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.577981  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.583624  8307 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:25:26.584092  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 142587us
I20260709 12:25:26.592175  8243 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.475725  8303 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:26.593672  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.522202  8305 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.576148  8332 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.526307  8308 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:26.577303  8331 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.603960  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.621213  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:26.592710  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.606633  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.622422  6307 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.623980  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.624769  7314 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:26.626055  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:26.603209  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.469460  8216 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.476037  8238 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.607055  6310 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:26.610093  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.550311  7317 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:26.593142  8254 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.616317  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:26.627238  8245 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.628229  8274 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.478026  8302 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.635365  7329 raft_consensus.cc:2368] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:26.607162  6301 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:26.639570  6307 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:26.656080  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:26.600993  8271 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:26.659849  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52107.2us
I20260709 12:25:26.661728  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:26.674281  6453 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:26.674670  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:26.663340  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:26.676033  6450 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.664466  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.677989  6456 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:26.666675  6275 leader_election.cc:400] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:26.671382  6385 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.679427  6275 leader_election.cc:403] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.655019  8250 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.612488  8334 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.612285  8163 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.658169  8222 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.684840  6376 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:26.631717  8331 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.691972  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.694425  8321 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.696959  8185 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.698627  8323 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:26.699118  8323 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:26.699911  8323 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.700335  8221 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.704875  6310 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:26.707820  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.708837  6301 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:25:26.676781  8254 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:26.616784  8320 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.717209  6425 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:25:26.721909  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.716758  6453 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:26.737689  6450 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:26.738914  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.739382  7332 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.741053  6425 leader_election.cc:403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.653579  8308 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.702783  8334 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.725849  8267 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.703567  8331 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.644248  8332 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.728168  8254 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:26.631306  8324 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:26.730366  8239 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.691474  8136 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.774660  8267 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.740265  8340 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.755739  8305 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:25:26.794773  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.778680  8254 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.795709  6379 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:26.796411  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.797966  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.798138  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:26.800469  6377 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.802177  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.799647  6385 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.802940  6376 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.788946  8340 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.809280  7314 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:26.811478  6377 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:26.799592  8216 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.815856  8334 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.816807  8163 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.820077  6450 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:26.800777  8271 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.793484  8185 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.825455  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.829445  6301 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.831096  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.830866  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.831569  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:26.832099  6379 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:26.801327  8339 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:26.835647  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.836365  6376 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:26.836180  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.837477  6385 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:26.808961  8340 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.843369  6377 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:26.824054  8227 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:26.846570  8320 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.848551  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.849288  6450 raft_consensus.cc:2368] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:26.851222  6272 leader_election.cc:400] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:26.851593  6272 leader_election.cc:403] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.741492  8307 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.841432  8327 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:26.840847  8185 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.858251  8267 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.859529  8185 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.861840  8185 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.756927  8303 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.880110  8243 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.883424  8332 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.884514  8308 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:26.885532  8136 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.887161  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.888067  8324 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.894941  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.895022  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.896211  6453 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:26.896872  8200 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.898387  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:26.907835  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 157652us
I20260709 12:25:26.838680  8163 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:25:26.890610  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.917512  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:26.841850  8271 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:25:26.918689  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:26.856704  8320 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.920684  8334 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:26.854879  8238 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:26.897321  8200 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.922358  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:26.919579  8338 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:26.857862  8327 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.923122  6305 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:26.897466  8324 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.917311  8216 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:26.916766  8332 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:26.919207  8308 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:26.943944  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:26.944626  7316 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:26.944425  8308 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:26.947190  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 272678us
I20260709 12:25:26.853896  8227 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:26.931674  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:26.915213  8305 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:26.927246  6301 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:26.940119  7332 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:26.926052  8245 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:25:26.943491  6307 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:26.939128  8319 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:26.949605  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:26.950649  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:26.973632  6453 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.936820  8334 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:26.928524  8200 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.967877  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.975914  6379 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:26.938428  8327 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:26.927881  8342 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:26.687458  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:26.942587  8332 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.980927  8332 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:26.974309  6204 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:26.977353  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.947487  8321 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:26.966081  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.946365  8216 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:26.963311  8058 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.990784  8058 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.993446  8058 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:26.982393  6376 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:26.981053  8334 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:26.946167  8324 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:26.984220  8348 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.002876  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:27.008375  6384 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:26.999348  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:27.009658  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:27.010105  6379 raft_consensus.cc:2368] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:27.003547  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:27.003693  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:27.012197  6382 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:27.002286  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:27.005690  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:26.974222  8305 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.013062  6381 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:27.010694  6380 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:27.016018  6274 leader_election.cc:400] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:27.022092  6310 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:25:27.023113  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55805us
I20260709 12:25:27.032223  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.032958  6456 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:27.039530  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:27.039844  6274 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:27.040412  6274 leader_election.cc:403] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.043191  6384 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:27.044559  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:27.069070  8353 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.074522  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:27.075320  6456 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:27.086946  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.091144  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.091989  6452 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:27.093116  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.093796  6452 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:27.057510  6453 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:27.015184  8342 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.069870  8353 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.089797  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.087834  6456 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:27.037508  8348 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.990044  8331 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.111033  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.112715  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.106288  6458 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:27.014321  8271 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:25:26.973111  8338 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.054798  8058 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:27.122754  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.055663  8221 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:27.069015  8361 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.103968  8136 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:27.115257  8266 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:27.124963  8308 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.172699  6457 raft_consensus.cc:2368] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:27.127300  8058 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.119289  8348 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:27.179808  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69478us
I20260709 12:25:27.127653  8274 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:27.128669  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:27.185245  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 291720us
W20260709 12:25:27.172067  8315 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:27.193005  8058 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:27.178576  8267 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.173812  8208 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.200959  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.201373  8354 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:27.205339  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76734.5us
I20260709 12:25:27.168877  8345 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.214903  8346 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.215394  8346 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.215754  8346 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.122390  8216 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.187740  8136 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.208688  8274 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:27.213467  8208 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.191428  8266 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.210199  8345 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.213655  8267 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.221903  8354 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.221529  8227 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.184990  8361 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.283732  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:27.519691  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 235921us
I20260709 12:25:27.521227  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2017) took 1062 ms. Trace:
I20260709 12:25:27.527051  6451 rpcz_store.cc:276] 0709 12:25:26.458851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.460558 (+  1707us) service_pool.cc:224] Handling call
0709 12:25:27.520757 (+1060199us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54002c9054. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:27.520993 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2411777920}
I20260709 12:25:27.496155  6201 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.521421  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.547271  6458 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:27.547791  6272 leader_election.cc:400] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:27.501629  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:27.538643  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:27.540359  7332 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.041s	sys 0.058s
W20260709 12:25:27.549198  7332 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.041s	sys 0.058s
I20260709 12:25:27.549453  6307 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:27.551834  6454 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
W20260709 12:25:27.541064  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 346504us
I20260709 12:25:27.552731  8136 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.542299  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.555226  8198 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.556409  8198 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.524874  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.414050  8305 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:27.430452  8314 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.799s	user 0.007s	sys 0.000s
I20260709 12:25:27.560456  8251 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:27.548363  6272 leader_election.cc:403] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.600803  6451 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:27.602129  8221 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.603108  8221 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.605916  8221 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.607318  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.437786  8348 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:27.596277  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.004s	sys 0.007s
W20260709 12:25:27.636093  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.006s	sys 0.007s
W20260709 12:25:27.432588  8320 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
W20260709 12:25:27.640650  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 116210us
W20260709 12:25:27.419524  8356 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
W20260709 12:25:27.433314  8351 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:25:27.647166  6454 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:27.647491  7329 raft_consensus.cc:2368] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:27.647642  8222 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.647785  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.355693  8354 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:27.640547  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 93503.1us
I20260709 12:25:27.606087  8251 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:27.611477  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.000s	sys 0.007s
W20260709 12:25:27.654775  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60191.5us
I20260709 12:25:27.350136  8345 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:27.651078  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 608149us
I20260709 12:25:27.429293  8353 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.347100  8250 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:27.658418  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.000s	sys 0.014s
W20260709 12:25:27.667940  8340 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.815s	user 0.011s	sys 0.004s
I20260709 12:25:27.432154  8331 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:27.672389  8228 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:27.672551  8368 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:27.432098  8338 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.437081  8361 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.436331  8324 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.417765  8342 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:27.678715  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 125364us
I20260709 12:25:27.682994  8359 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:27.683236  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 117) took 1458 ms. Trace:
I20260709 12:25:27.661286  8208 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.688756  6455 rpcz_store.cc:276] 0709 12:25:26.222599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.223317 (+   718us) service_pool.cc:224] Handling call
0709 12:25:26.223666 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:26.224071 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.224096 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.224570 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.411662 (+187092us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:26.411698 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.411757 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.429181 (+1017424us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:27.680164 (+250983us) raft_consensus.cc:1685] finished
0709 12:25:27.680215 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.680533 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":23586048,"thread_start_us":186709,"threads_started":1,"wal-append.queue_time_us":326238}
I20260709 12:25:27.690773  8251 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.687920  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.692612  7316 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:27.693886  8320 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:27.443133  8346 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.695139  8250 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.696519  8266 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.659129  8363 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.644891  6201 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.426828  8274 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.703382  8228 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.700728  7332 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
W20260709 12:25:27.701625  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 89026.8us
W20260709 12:25:27.715615  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58020.3us
I20260709 12:25:27.716351  8308 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:27.716852  8308 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:27.717357  8308 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:27.698752  8359 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.429597  8332 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:27.732758  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.703277  8267 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.707942  8228 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.706984  8198 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.739697  6455 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:25:27.726320  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 681816us
W20260709 12:25:27.733659  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.027s	sys 0.025s
I20260709 12:25:27.741240  6206 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 268) took 1446 ms. Trace:
I20260709 12:25:27.723328  8305 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:25:27.749135  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:27.751355  8270 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:27.752111  8270 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:27.752537  8265 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.732275  8359 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.677982  8340 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:27.755249  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.027s	sys 0.026s
I20260709 12:25:27.761665  8265 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:27.764096  8058 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.006s
I20260709 12:25:27.764333  8265 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.764729  8058 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.767711  8058 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.767750  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:27.721819  8335 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.738920  8368 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.439064  8327 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.778112  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.781515  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.791119  8305 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.791433  8228 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.792092  8228 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.792047  8305 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.792618  6455 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:27.792824  8228 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.794689  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:27.796949  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.721804  8363 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.800750  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:27.798326  8265 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.799031  8270 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.816123  6450 raft_consensus.cc:2368] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:27.801169  8348 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.812323  8265 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.827252  6378 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:27.827898  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.828778  7316 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:25:27.829355  8331 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:25:27.832235  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.835960  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.842851  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:27.845024  8360 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.511s	user 0.000s	sys 0.003s
I20260709 12:25:27.844560  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.848230  7314 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:27.844884  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:27.850790  8337 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:27.855190  6454 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.835021  8265 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:27.852933  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:27.860947  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47314.2us
I20260709 12:25:27.863344  7332 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:27.850809  8340 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:27.866026  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132295us
I20260709 12:25:27.870072  8347 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:27.870297  8346 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.870762  8347 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.870990  7314 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:27.871388  8347 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.871740  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:27.875093  8320 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.875248  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.875933  8320 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.879122  8328 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.879396  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.853770  8372 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:27.885179  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67439.2us
I20260709 12:25:27.744149  6206 rpcz_store.cc:276] 0709 12:25:26.195812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.195979 (+   167us) service_pool.cc:224] Handling call
0709 12:25:27.635148 (+1439169us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:26.312260 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:26.358923 (+ 46663us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:26.358980 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:25:26.359007 (+    27us) tablet.cc:662] Decoding operations
0709 12:25:26.359463 (+   456us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:26.359516 (+    53us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:26.359542 (+    26us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:26.359742 (+   200us) tablet.cc:701] Row locks acquired
0709 12:25:26.359761 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:25:26.359900 (+   139us) write_op.cc:270] Start()
0709 12:25:26.360001 (+   101us) write_op.cc:276] Timestamp: P: 1783599926359859 usec, L: 0
0709 12:25:26.360020 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:25:26.360652 (+   632us) log.cc:844] Serialized 532 byte log entry
0709 12:25:26.574249 (+213597us) op_driver.cc:464] REPLICATION: finished
0709 12:25:26.577732 (+  3483us) write_op.cc:301] APPLY: starting
0709 12:25:26.577826 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:26.577957 (+   131us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:26.578128 (+   171us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:26.578410 (+   282us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:26.578682 (+   272us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:26.578759 (+    77us) write_op.cc:312] APPLY: finished
0709 12:25:26.579149 (+   390us) log.cc:844] Serialized 23 byte log entry
0709 12:25:26.582648 (+  3499us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:26.582762 (+   114us) write_op.cc:454] Released schema lock
0709 12:25:26.582814 (+    52us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:26.584419 (+  1605us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:26.584463 (+    44us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1359,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":3181,"apply.run_cpu_time_us":2295,"apply.run_wall_time_us":6663,"num_ops":1,"prepare.queue_time_us":9382,"prepare.run_cpu_time_us":4289,"prepare.run_wall_time_us":56531,"replication_time_us":213981,"thread_start_us":9491,"threads_started":1}]]}
I20260709 12:25:27.887539  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:27.883057  6425 leader_election.cc:400] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:27.895435  6425 leader_election.cc:403] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.892977  6458 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
W20260709 12:25:27.892102  8314 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:27.900730  8368 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:27.901005  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:27.905786  8342 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.906527  8342 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.908860  8342 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.884516  8198 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:27.914742  8353 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.915496  8353 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.890477  8245 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:27.851804  8337 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.925799  8353 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.926394  6455 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:27.927839  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:27.883796  8328 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.932108  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.858343  8335 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.917588  8198 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.942190  8338 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.945071  8221 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.952087  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:27.952514  8221 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.953276  6309 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:27.935263  8305 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.945389  8198 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:27.909319  8372 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.944109  8274 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:27.945170  8245 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:27.967456  6309 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:27.968348  8323 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:27.967904  8245 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.968823  8323 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.969224  8323 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.966260  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:27.955155  8254 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.132s	user 0.009s	sys 0.011s
I20260709 12:25:27.951520  8222 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:27.978154  8250 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:27.929972  8359 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:27.980706  8305 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:27.990000  6454 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:27.997514  8337 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:27.970455  8221 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:27.965504  8274 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:28.006213  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:27.965080  8332 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:28.012144  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118863us
I20260709 12:25:28.015800  8305 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.018611  8324 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.019441  8324 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.020226  8345 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.024533  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.026315  8185 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.026953  8185 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:28.028012  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:28.028187  8238 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 12:25:28.028868  6300 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.029505  8335 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.030620  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:28.030977  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.031463  6309 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:27.993103  8368 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:28.021219  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1481) took 1212 ms. Trace:
W20260709 12:25:28.035187  8371 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.217s	user 0.002s	sys 0.002s
I20260709 12:25:27.962744  8338 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.043545  6455 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:28.027953  8185 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.039371  6425 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:28.047093  6425 leader_election.cc:403] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.039896  6378 rpcz_store.cc:276] 0709 12:25:26.797262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.797722 (+   460us) service_pool.cc:224] Handling call
0709 12:25:27.988310 (+1190588us) spinlock_profiling.cc:243] Waited 981 ms on lock 0x7b54003d1154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:28.009283 (+ 20973us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2255552896}
I20260709 12:25:28.045073  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.009656  8274 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:28.018667  8200 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.028s	user 0.014s	sys 0.006s
I20260709 12:25:28.023496  8222 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.043980  8363 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.050977  7332 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:28.040690  7314 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:28.052433  8207 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:28.051780  8250 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.002079  8254 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.052634  8331 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:28.052824  8305 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.076802  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.077375  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:28.080668  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.081112  8345 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:28.081913  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46000.6us
I20260709 12:25:28.083117  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.057801  8320 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.086194  6455 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:28.052256  8200 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.088490  8222 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.096781  8222 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.097587  8308 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.099035  8324 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.099850  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.100632  7317 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:25:28.093530  8207 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.103153  8338 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.103585  8207 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.108855  6304 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:28.112936  6300 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:28.115821  6382 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:28.116933  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 171) took 1065 ms. Trace:
I20260709 12:25:28.118992  8345 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.118818  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.122642  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.122663  6301 rpcz_store.cc:276] 0709 12:25:27.051470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.052143 (+   673us) service_pool.cc:224] Handling call
0709 12:25:27.052557 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.186441 (+133884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.186478 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.192947 (+  6469us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.697284 (+504337us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:27.713975 (+ 16691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.714108 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:27.988661 (+274553us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b240017b058. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:25:28.116274 (+127613us) raft_consensus.cc:1685] finished
0709 12:25:28.116309 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.116670 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9579,"raft.queue_time_us":67742,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":7721,"spinlock_wait_cycles":66934144,"thread_start_us":290719,"threads_started":2,"wal-append.queue_time_us":254980}
I20260709 12:25:28.123970  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.125077  6384 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:28.130692  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:28.132656  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.133132  6383 raft_consensus.cc:2368] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:28.134769  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.135390  6272 leader_election.cc:400] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:28.135730  6378 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:28.135893  6272 leader_election.cc:403] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.128752  8328 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.140079  8372 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.142755  8346 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.145332  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.145430  8346 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:28.146198  6382 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:28.147495  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.149833  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.150568  6454 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.148801  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.104581  8340 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.149348  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.160550  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.161298  8207 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:28.164223  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.165349  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:28.183032  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.185732  8200 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.187146  8254 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.188083  8254 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:28.188563  8254 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.188982  8254 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:28.190659  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.182788  8345 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.193145  8340 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.195657  8303 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.196395  8303 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.197011  8345 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.201532  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.202346  6376 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:28.203353  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.203583  8327 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.204229  6376 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:28.204281  8327 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.204910  8327 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.205191  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.195667  8340 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.207396  6305 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
W20260709 12:25:28.207823  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 72660.3us
I20260709 12:25:28.208802  8368 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.209084  6424 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:28.209486  6424 leader_election.cc:403] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.210212  8305 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.213449  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.121711  8324 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:25:28.214958  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.215797  6458 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:28.209488  8368 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.217247  8207 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.218559  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.219192  6453 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:28.219672  8368 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.219755  8207 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.206300  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.231050  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:28.232656  8308 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.233377  8308 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:28.235502  6450 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:28.158471  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.238417  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.239053  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.239632  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.214962  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.241545  6456 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:28.242265  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 1922) took 1242 ms. Trace:
I20260709 12:25:28.242823  6383 rpcz_store.cc:276] 0709 12:25:27.000200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.005422 (+  5222us) service_pool.cc:224] Handling call
0709 12:25:28.241949 (+1236527us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400064d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:28.242170 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2562744064}
I20260709 12:25:28.244235  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.244359  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.240840  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.323288  8305 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:28.318728  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.273386  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:28.319746  7335 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:28.325047  6309 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:28.325433  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.317438  6454 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
W20260709 12:25:28.329207  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62715.8us
I20260709 12:25:28.336412  8222 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:28.245298  8327 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.339710  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.340962  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:28.338806  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:28.341929  8386 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 12:25:28.342751  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 111812us
I20260709 12:25:28.342927  8271 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:25:28.343659  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58058.4us
I20260709 12:25:28.344394  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.344576  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.344907  8207 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:28.344295  8345 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:28.341985  8354 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.647s	user 0.010s	sys 0.005s
I20260709 12:25:28.243644  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.337385  8359 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:28.345695  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 72402.5us
I20260709 12:25:28.274560  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.345890  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.385430  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.247648  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.386817  8198 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.387961  8274 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.389133  8198 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.390821  6302 raft_consensus.cc:2368] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3.
I20260709 12:25:28.391891  6204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 261) took 2013 ms. Trace:
I20260709 12:25:28.273744  8346 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.359922  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:28.382696  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45384us
I20260709 12:25:28.395431  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.381409  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.290138  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.346446  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.245590  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.383155  8345 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:28.267793  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:28.325526  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.351114  6453 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:28.345427  8207 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.326956  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.352725  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.353297  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.405717  6380 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:28.406168  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52955.2us
I20260709 12:25:28.354357  8348 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:28.354390  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 107619us
I20260709 12:25:28.341068  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.355049  6457 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.359993  6376 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.359812  8363 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:25:28.361434  8228 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.568s	user 0.007s	sys 0.007s
I20260709 12:25:28.361598  8347 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.411219  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.363478  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.344416  6309 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:28.320714  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:28.264549  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.266705  8250 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.414265  8228 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.427177  6305 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:28.401942  6301 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.408485  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.426738  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:28.412892  6385 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:28.398244  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.266209  8368 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 12:25:28.432196  6383 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.406723  8207 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:28.367208  8308 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.411173  8271 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.423931  6377 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:25:28.413421  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.435598  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:28.399811  8305 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.431627  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:28.433761  6310 raft_consensus.cc:2368] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:28.425009  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:28.383723  8354 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.436661  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.292347  8303 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.437198  6457 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:28.436972  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.441700  6307 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:28.423949  8228 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.466995  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.467190  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:28.392338  6204 rpcz_store.cc:276] 0709 12:25:26.373627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.373779 (+   152us) service_pool.cc:224] Handling call
0709 12:25:28.386637 (+2012858us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:27.649606 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:27.723688 (+ 74082us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:27.723758 (+    70us) write_op.cc:435] Acquired schema lock
0709 12:25:27.723790 (+    32us) tablet.cc:662] Decoding operations
0709 12:25:27.724404 (+   614us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:27.724498 (+    94us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:27.724548 (+    50us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:25:27.724992 (+   444us) tablet.cc:701] Row locks acquired
0709 12:25:27.725037 (+    45us) write_op.cc:260] PREPARE: finished
0709 12:25:27.725253 (+   216us) write_op.cc:270] Start()
0709 12:25:27.725417 (+   164us) write_op.cc:276] Timestamp: P: 1783599927725158 usec, L: 0
0709 12:25:27.725443 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:25:27.745311 (+ 19868us) log.cc:844] Serialized 527 byte log entry
0709 12:25:27.958408 (+213097us) op_driver.cc:464] REPLICATION: finished
0709 12:25:28.109969 (+151561us) write_op.cc:301] APPLY: starting
0709 12:25:28.110107 (+   138us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:28.110278 (+   171us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:28.110537 (+   259us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:28.111043 (+   506us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:28.111362 (+   319us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:25:28.111444 (+    82us) write_op.cc:312] APPLY: finished
0709 12:25:28.111824 (+   380us) log.cc:844] Serialized 23 byte log entry
0709 12:25:28.150545 (+ 38721us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:28.173112 (+ 22567us) write_op.cc:454] Released schema lock
0709 12:25:28.174885 (+  1773us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:28.346153 (+171268us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:28.352701 (+  6548us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":698243,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":135136,"apply.run_cpu_time_us":3195,"apply.run_wall_time_us":156043,"num_ops":1,"prepare.queue_time_us":139760,"prepare.run_cpu_time_us":12743,"prepare.run_wall_time_us":133214,"raft.queue_time_us":93237,"raft.run_cpu_time_us":26994,"raft.run_wall_time_us":140489,"replication_time_us":223906,"thread_start_us":169391,"threads_started":4,"wal-append.queue_time_us":59989}]]}
I20260709 12:25:28.443658  6379 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:28.434191  8331 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:28.471117  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.473014  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:28.449230  8265 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.576s	user 0.008s	sys 0.004s
I20260709 12:25:28.436717  8274 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.427429  8198 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:28.484962  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.449434  8200 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.489739  6456 raft_consensus.cc:2368] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:28.491647  6381 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:28.497609  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 66051.5us
I20260709 12:25:28.498617  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:28.504069  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.504756  6458 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:28.505330  6274 leader_election.cc:400] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:28.475683  8320 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:28.474875  8270 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:28.482344  8327 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.486137  8349 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.450928  8250 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.482446  8332 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.519459  6380 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:28.512208  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.520661  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:28.457365  8308 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.532816  6381 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:28.511641  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:28.456391  8305 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.487254  8384 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.532816  6455 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
W20260709 12:25:28.525143  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118201us
I20260709 12:25:28.510190  8320 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:28.536654  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.536321  6308 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:28.488596  8265 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.521744  8359 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:28.514434  8185 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:28.543762  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.502562  8200 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:28.529026  8323 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.006s
I20260709 12:25:28.545802  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.543483  8327 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:28.545598  6459 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:28.540894  8384 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:28.552064  6378 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:28.490331  8345 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.541352  8320 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:28.554688  7329 raft_consensus.cc:2368] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:28.556449  6274 leader_election.cc:400] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:28.559831  6385 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:28.545074  8335 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:28.550014  6351 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.508s (SENT) [suppressed 9 similar messages]
I20260709 12:25:28.525825  8270 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.565198  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.550019  8338 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:28.550978  8328 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.550860  8331 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.553483  8222 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:28.552505  8359 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:28.582106  6451 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
W20260709 12:25:28.584329  8389 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:25:28.552960  8323 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.596864  8327 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.601580  8305 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.601981  8363 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.602967  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.603807  8347 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.604207  6451 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:28.603614  8384 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:28.587626  8322 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:28.608907  8331 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.611385  8354 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.611557  6301 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:28.563210  8368 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:25:28.615150  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.617583  6201 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 139) took 2380 ms. Trace:
I20260709 12:25:28.562995  8185 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.572046  8303 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.602967  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.644347  6379 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:28.596931  8394 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:28.650216  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.650409  6376 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:28.650164  6377 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
W20260709 12:25:28.648377  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 145462us
W20260709 12:25:28.650521  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95399.5us
I20260709 12:25:28.652983  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:28.653685  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:28.709517  6451 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:28.658108  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.669378  8308 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.691219  6454 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:28.691020  6300 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:28.692902  6458 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:28.611379  8250 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:28.712026  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 159414us
I20260709 12:25:28.712488  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:28.714061  6307 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:28.717356  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.718600  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:28.732587  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.741611  8359 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.744737  8323 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.748028  8185 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.750005  8331 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:28.754981  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 204988us
I20260709 12:25:28.727869  8328 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:28.723575  8337 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.699527  8267 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:28.672025  6201 rpcz_store.cc:276] 0709 12:25:26.236793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.236988 (+   195us) service_pool.cc:224] Handling call
0709 12:25:28.617476 (+2380488us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:27.817770 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:27.888561 (+ 70791us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:27.895490 (+  6929us) write_op.cc:435] Acquired schema lock
0709 12:25:27.897600 (+  2110us) tablet.cc:662] Decoding operations
0709 12:25:27.951257 (+ 53657us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:27.954636 (+  3379us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:27.954690 (+    54us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:27.955521 (+   831us) tablet.cc:701] Row locks acquired
0709 12:25:27.955548 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:25:27.958344 (+  2796us) write_op.cc:270] Start()
0709 12:25:28.097385 (+139041us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b60001051c8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa6c11161 00007f6fa6d7cae3 00007f6fa6d7f67c 00007f6fa6c12e93 00007f6fa6bf0246 00007f6fa6cf263a 00007f6fa6cdd286 00007f6fa6cdcd58 00007f6fa6ce19f1 00007f6fa6ce19a9 00007f6fa6ce1939
0709 12:25:28.097667 (+   282us) write_op.cc:276] Timestamp: P: 1783599927955670 usec, L: 0
0709 12:25:28.097702 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:25:28.129632 (+ 31930us) log.cc:844] Serialized 901 byte log entry
0709 12:25:28.200463 (+ 70831us) spinlock_profiling.cc:243] Waited 8.61 ms on lock 0x7b1800384b80. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa6cde41d 00007f6fa6ce071a 00007f6fa6ce06a8 00007f6fa6ce0628 00007f6fa6ce05e3 00007f6fa6cdf7fc 00007f6fa91122f6 00007f6fa910caa8 00007f6fa32b3310
0709 12:25:28.200531 (+    68us) op_driver.cc:464] REPLICATION: finished
0709 12:25:28.398105 (+197574us) write_op.cc:301] APPLY: starting
0709 12:25:28.404490 (+  6385us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:28.420636 (+ 16146us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:28.421712 (+  1076us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:28.442850 (+ 21138us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:28.477138 (+ 34288us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:25:28.481085 (+  3947us) write_op.cc:312] APPLY: finished
0709 12:25:28.481331 (+   246us) log.cc:844] Serialized 29 byte log entry
0709 12:25:28.489503 (+  8172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:28.489614 (+   111us) write_op.cc:454] Released schema lock
0709 12:25:28.489677 (+    63us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":17963,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":183694,"apply.run_cpu_time_us":2463,"apply.run_wall_time_us":93875,"num_ops":2,"prepare.queue_time_us":68805,"prepare.run_cpu_time_us":12037,"prepare.run_wall_time_us":335740,"replication_time_us":62335,"spinlock_wait_cycles":39787136,"thread_start_us":306306,"threads_started":1}]]}
I20260709 12:25:28.764060  6274 leader_election.cc:400] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:28.767410  6274 leader_election.cc:403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.710621  8303 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.707238  8363 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.775357  8207 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:28.753407  8265 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.780411  6383 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:28.781322  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:28.786782  6310 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:28.744786  8200 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.758041  8228 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:28.823992  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 103183us
I20260709 12:25:28.798615  8250 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:28.842664  6384 raft_consensus.cc:2368] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:28.832597  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:28.799835  8322 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:28.843751  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:28.845405  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.803341  8337 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.880190  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:28.898439  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53176.8us
I20260709 12:25:28.901582  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:28.902733  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75755.5us
I20260709 12:25:28.903452  7288 leader_election.cc:400] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3.
I20260709 12:25:28.903707  7288 leader_election.cc:403] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.906596  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:28.840216  8328 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:25:28.915426  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:28.920189  6272 leader_election.cc:400] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:28.937892  6272 leader_election.cc:403] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.872797  8363 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:28.932873  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:28.934027  7317 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.007s
I20260709 12:25:28.934341  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:28.934818  8274 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:28.935034  8359 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.935995  8325 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:28.873843  8346 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:28.899580  8228 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:28.975256  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55304.7us
I20260709 12:25:28.908046  8327 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.977283  8207 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:28.979017  8254 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.789s	user 0.011s	sys 0.004s
I20260709 12:25:28.979507  7317 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:28.980675  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:28.981269  8348 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:25:28.981727  6303 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:28.983803  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51065.6us
I20260709 12:25:28.983021  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:28.910810  8372 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:28.939754  8267 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:28.880900  8399 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.218s
I20260709 12:25:28.975883  8058 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:28.994825  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:28.995201  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:28.979530  8270 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:28.997414  6272 leader_election.cc:400] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
W20260709 12:25:29.063057  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65735.2us
I20260709 12:25:29.063572  7332 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:29.063843  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.064882  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:28.999877  7331 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:28.987284  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:28.973344  8331 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:28.881239  8322 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:29.050714  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 113738us
I20260709 12:25:28.988957  8346 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:29.055315  6305 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:28.984510  8347 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:25:28.987138  8261 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:29.002853  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:28.989188  8245 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:29.058269  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.086844  8322 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:28.999451  8325 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:29.087908  8363 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:29.087939  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:29.088892  6457 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.088517  8325 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.090813  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.048445  8348 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:29.007441  8327 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:28.992865  8359 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.074368  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.075438  6302 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:29.077504  8331 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.069473  8274 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.072046  8324 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.080832  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.081136  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.082240  6452 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:29.001420  8254 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.075757  8346 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:29.104489  6385 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:29.071043  6425 leader_election.cc:400] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:29.124032  6425 leader_election.cc:403] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:29.084390  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81615.9us
I20260709 12:25:29.096686  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.075042  8058 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:29.100778  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.103191  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.106043  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:29.125852  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.109285  8327 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:29.110338  8337 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.127118  6382 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:29.127372  7317 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:29.110918  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.127574  8058 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.111236  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.111521  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.130811  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:29.132076  8410 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.291s	user 0.000s	sys 0.002s
I20260709 12:25:29.112525  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.116178  8359 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:29.116995  6458 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:29.133757  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.111985  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.135143  6302 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:29.135841  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.120220  8254 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:29.120725  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.121212  6453 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:29.140573  6305 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:29.123450  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:29.093510  8303 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:29.131999  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.136185  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.142161  6300 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:29.120072  8331 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:29.144603  6309 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:29.145182  8250 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
W20260709 12:25:29.145337  8398 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.420s	user 0.001s	sys 0.001s
I20260709 12:25:29.150921  7319 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:29.154850  6303 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:29.093358  8245 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:29.138826  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67986.5us
I20260709 12:25:29.169359  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.172267  7335 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:29.174697  8305 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.155798  8327 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:29.108054  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:29.177783  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.143424  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.178411  6202 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 294) took 2724 ms. Trace:
I20260709 12:25:29.139619  8324 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:29.178810  6202 rpcz_store.cc:276] 0709 12:25:26.454387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.454748 (+   361us) service_pool.cc:224] Handling call
0709 12:25:29.178224 (+2723476us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":2}
I20260709 12:25:29.179651  8324 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.163121  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.182790  6303 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:29.164800  6457 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:29.183403  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:29.184144  6376 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.191035  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:29.191908  6456 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:29.192268  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:29.192518  6457 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:29.193853  6457 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:29.197683  8228 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:29.201589  8348 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:29.202980  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:29.104123  8320 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.540s	user 0.009s	sys 0.003s
I20260709 12:25:29.205593  8320 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.206082  8245 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.155413  8401 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:25:29.207680  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.208374  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.169414  6310 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:29.216876  8254 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:29.217690  8327 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:29.217958  8267 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3.
I20260709 12:25:29.218196  8327 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.218592  8327 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:29.220511  8414 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.313s	user 0.002s	sys 0.000s
I20260709 12:25:29.225940  7320 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:29.228251  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.228451  6301 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:29.230863  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.230105  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:29.236536  6275 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2483d7801887403e873adc62429fe046. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:29.237272  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.238049  6304 raft_consensus.cc:2376] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7. Re-sending same reply.
I20260709 12:25:29.249154  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:29.233743  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:29.262655  6424 leader_election.cc:400] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:29.265162  6377 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:29.267056  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:29.272941  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.275683  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.276649  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.267238  8303 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:29.288787  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.289212  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.288839  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.281657  8185 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:25:29.288779  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:29.261121  8363 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:25:29.260257  8399 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:25:29.281809  8200 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:29.254144  8346 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.266899  8305 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:25:29.283842  8323 raft_consensus.cc:2764] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260709 12:25:29.275015  8267 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:25:29.303659  8331 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:29.297369  8303 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:25:29.334993  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47515.4us
I20260709 12:25:29.338621  7329 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:29.328023  8331 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:29.351011  7335 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:29.350775  8331 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.353552  7319 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:29.286208  8396 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
W20260709 12:25:29.352597  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67515.4us
I20260709 12:25:29.354048  8303 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.367602  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.337637  8348 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:29.424095  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56634.3us
I20260709 12:25:29.414582  8328 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.425089  8328 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.424141  8348 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.429059  8328 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.430990  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.431955  6458 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:29.433715  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:29.434336  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 157767us
W20260709 12:25:29.435330  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 151543us
I20260709 12:25:29.436851  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.437167  8433 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:29.441895  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.442673  8267 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:25:29.407943  8404 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.562s	user 0.013s	sys 0.000s
W20260709 12:25:29.417193  8424 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:25:29.439465  8433 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:29.451757  8345 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:29.449470  8324 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:29.465020  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59931.2us
W20260709 12:25:29.466212  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 182170us
I20260709 12:25:29.470536  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.457453  8320 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:29.463630  8200 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:29.468165  8324 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:29.499207  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.469575  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:25:29.512533  8418 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 12:25:29.514474  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44107.5us
I20260709 12:25:29.490597  8337 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.502002  8440 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:29.467989  8327 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.528429  6459 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:29.530001  6383 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:29.531430  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 178) took 1323 ms. Trace:
I20260709 12:25:29.504029  8324 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.532002  6381 rpcz_store.cc:276] 0709 12:25:28.200237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.581156 (+380919us) service_pool.cc:224] Handling call
0709 12:25:28.594897 (+ 13741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.595431 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:28.595459 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:28.595968 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:25:28.697395 (+101427us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:28.697448 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.697537 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.275169 (+577632us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:29.520277 (+245108us) raft_consensus.cc:1685] finished
0709 12:25:29.520304 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:29.523709 (+  3405us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":470,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"spinlock_wait_cycles":14701440,"thread_start_us":100980,"threads_started":1,"wal-append.queue_time_us":161601}
I20260709 12:25:29.537865  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:29.522985  8328 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:29.532752  8327 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:29.542646  6458 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:29.531872  8346 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:29.554664  8348 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:25:29.512600  8200 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.565459  8324 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.534538  8440 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:29.531600  8337 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.591991  8440 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.554708  8427 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:29.602466  8427 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.604699  8427 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.601711  8337 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:29.552942  8328 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.610466  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.621001  8331 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.625861  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.635076  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:29.639036  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 96648.5us
I20260709 12:25:29.648049  6309 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:29.662629  8425 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:29.650631  8058 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:29.680070  8324 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.689442  8431 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:29.688000  8331 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:29.706302  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 433411us
I20260709 12:25:29.712699  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:29.714720  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:29.716048  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:29.717710  6425 leader_election.cc:400] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:29.718664  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:29.711670  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:29.750702  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.752303  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.674958  8058 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.763075  8428 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:29.765506  8338 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.718434  8327 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:29.807211  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50705.2us
I20260709 12:25:29.814324  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:29.817137  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.824921  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:29.774952  8328 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:29.766459  8338 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:29.778038  8446 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:29.768615  8416 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:29.794937  8428 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:29.718669  8431 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.844996  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.850328  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:29.851590  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.788304  8254 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:29.783507  8432 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:29.853238  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:29.829684  8363 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:29.828075  8368 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:29.867503  8254 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.873543  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.781420  8325 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
I20260709 12:25:29.868690  6308 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:29.874993  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.879326  8446 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:29.869133  8363 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:29.888466  6308 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:25:29.866520  8432 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:29.895891  7320 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:29.896128  8432 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:29.869882  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.872061  8431 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.880867  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:29.905397  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:29.871847  8399 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:25:29.909626  7314 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:25:29.902940  8245 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.005s
I20260709 12:25:29.870386  8428 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.915377  6308 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:29.898402  8363 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:29.868003  8416 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:29.936858  8058 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.946136  7318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:34546
I20260709 12:25:29.946981  7318 raft_consensus.cc:606] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:29.948259  8416 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:29.914250  8245 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:29.969938  8308 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:29.976068  8245 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:29.980608  8426 raft_consensus.cc:993] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:25:29.981307  8426 raft_consensus.cc:1081] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Signalling peer 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:25:29.987980  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:29.989570  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:29.990187  6453 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:29.993804  6452 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:30.027920  8363 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.012854  8274 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:30.025239  8254 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:30.036543  8354 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:30.002180  6202 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:30.044852  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:30.056053  6452 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:29.995950  8432 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.074769  6202 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 5 to 7, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 7 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:30.078948  6202 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:30.070458  8200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.030s	sys 0.102s
I20260709 12:25:30.079906  8185 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:30.071938  8354 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:30.114326  8267 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:30.069636  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 409857us
I20260709 12:25:30.092255  8305 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.106156  6202 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 5, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:30.083601  8250 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:30.142136  6381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.017s	sys 0.008s
W20260709 12:25:30.235080  6381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.018s	sys 0.008s
I20260709 12:25:30.106384  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:30.085100  8200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.030s	sys 0.102s
W20260709 12:25:30.210897  6377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.013s
I20260709 12:25:30.228915  6452 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:30.238752  6377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.003s	sys 0.013s
I20260709 12:25:30.240923  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
W20260709 12:25:30.240919  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 134664us
I20260709 12:25:30.245141  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:30.245894  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:30.252812  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:30.241642  6384 raft_consensus.cc:606] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:30.207108  8222 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:30.217888  8332 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.252640  8384 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:30.282537  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 316061us
I20260709 12:25:30.298636  6453 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:30.394068  6457 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:30.294059  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:30.249199  8250 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:30.230899  8267 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:30.269918  8440 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:30.273579  8347 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:30.272850  8222 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.394057  6451 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:30.469034  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:30.411870  8261 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.002s	sys 0.020s
W20260709 12:25:30.415684  8303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.027s	sys 0.027s
W20260709 12:25:30.464653  7335 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.007s	sys 0.065s
W20260709 12:25:30.476289  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60414.1us
I20260709 12:25:30.459193  6382 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:30.461390  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 167621us
I20260709 12:25:30.277254  8332 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:30.490338  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.019s
I20260709 12:25:30.414127  8200 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:30.292560  8198 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.177s
I20260709 12:25:30.385447  8436 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.395s
W20260709 12:25:30.471580  8338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.021s	sys 0.021s
W20260709 12:25:30.494300  7335 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.007s	sys 0.067s
W20260709 12:25:30.497579  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.009s	sys 0.019s
I20260709 12:25:30.470839  8308 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.295305  8349 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:30.519423  6349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.011s	sys 0.016s
W20260709 12:25:30.526237  6349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.011s	sys 0.016s
W20260709 12:25:30.526734  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 676496us
I20260709 12:25:30.519896  8473 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:30.528929  8447 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 12:25:30.529273  8473 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.490437  8363 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:30.396967  8384 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.095s
I20260709 12:25:30.466589  8409 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.423s
W20260709 12:25:30.534202  8448 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.270s	user 0.000s	sys 0.000s
I20260709 12:25:30.459225  8347 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:30.490283  8261 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.002s	sys 0.031s
W20260709 12:25:30.493886  8303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.030s	sys 0.030s
I20260709 12:25:30.580816  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
W20260709 12:25:30.490743  8444 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.017s	sys 0.030s
W20260709 12:25:30.528929  8450 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 12:25:30.523249  8338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.022s	sys 0.022s
I20260709 12:25:30.530423  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1609) took 1292 ms. Trace:
I20260709 12:25:30.463038  8440 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.527501  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:30.524736  8432 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:30.592386  6384 raft_consensus.cc:606] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:30.525537  8337 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:30.592310  8444 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.018s	sys 0.030s
I20260709 12:25:30.531689  8473 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:30.595008  8348 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:30.671207  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 137842us
I20260709 12:25:30.640071  6450 rpcz_store.cc:276] 0709 12:25:29.238039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.239163 (+  1124us) service_pool.cc:224] Handling call
0709 12:25:29.239428 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.442833 (+203405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.442869 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.444183 (+  1314us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.786811 (+342628us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:29.786858 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.786942 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.530187 (+743245us) raft_consensus.cc:1685] finished
0709 12:25:30.530211 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.530353 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11413,"raft.queue_time_us":23343,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6250,"thread_start_us":498267,"threads_started":2,"wal-append.queue_time_us":631695}
W20260709 12:25:30.682035  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 154627us
I20260709 12:25:30.694013  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:30.530588  8200 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.661192  8436 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:30.672300  8428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.023s	sys 0.050s
I20260709 12:25:30.709638  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 145) took 1174 ms. Trace:
I20260709 12:25:30.676539  8245 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:30.714146  6454 rpcz_store.cc:276] 0709 12:25:29.535383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.535801 (+   418us) service_pool.cc:224] Handling call
0709 12:25:29.536140 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.542507 (+  6367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.542540 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.573719 (+ 31179us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.791729 (+218010us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:29.791795 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.791909 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.709411 (+917502us) raft_consensus.cc:1685] finished
0709 12:25:30.709436 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.709574 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3988,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":4262,"thread_start_us":203744,"threads_started":1,"wal-append.queue_time_us":527418}
I20260709 12:25:30.680660  8467 ts_tablet_manager.cc:933] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:30.663679  8349 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.678114  8348 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:30.773943  8198 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:30.721129  8436 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:30.775249  8453 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 12:25:30.775382  8466 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:25:30.775631  8332 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:30.713038  8444 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:25:30.776325  8222 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:30.714038  8428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.025s	sys 0.050s
I20260709 12:25:30.707278  8274 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:30.776628  8323 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:30.708817  8270 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:30.678560  8337 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:30.785307  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 287980us
I20260709 12:25:30.691056  8399 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:30.772253  8401 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.814298  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1515) took 1291 ms. Trace:
I20260709 12:25:30.798565  8245 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.842849  8267 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:30.794955  8435 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.826128  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:30.852274  8323 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:30.871902  6376 rpcz_store.cc:276] 0709 12:25:29.515015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.515176 (+   161us) service_pool.cc:224] Handling call
0709 12:25:29.528952 (+ 13776us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.535477 (+  6525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.535507 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.545353 (+  9846us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.894019 (+348666us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b48001f0158. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:25:30.073751 (+179732us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:30.073795 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.073858 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.798859 (+725001us) raft_consensus.cc:1685] finished
0709 12:25:30.798883 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.806614 (+  7731us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2004,"raft.queue_time_us":3882,"raft.run_cpu_time_us":3316,"raft.run_wall_time_us":356368,"spinlock_wait_cycles":2302848,"thread_start_us":493488,"threads_started":1,"wal-append.queue_time_us":686077}
W20260709 12:25:30.830015  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.027s	sys 0.016s
I20260709 12:25:30.831274  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:30.817765  8474 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.401s	user 0.001s	sys 0.001s
I20260709 12:25:30.819329  8198 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.854030  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:30.876013  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 189) took 1329 ms. Trace:
I20260709 12:25:30.879253  6453 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:30.880602  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 150) took 1213 ms. Trace:
I20260709 12:25:30.877471  6308 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:30.895856  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:30.881376  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.027s	sys 0.016s
I20260709 12:25:30.853947  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:30.883059  6381 rpcz_store.cc:276] 0709 12:25:29.546646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.546934 (+   288us) service_pool.cc:224] Handling call
0709 12:25:29.572676 (+ 25742us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.577971 (+  5295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.578015 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.578556 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.235685 (+657129us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:30.235746 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.235847 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.875776 (+639929us) raft_consensus.cc:1685] finished
0709 12:25:30.875802 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.875952 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4075,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":3488,"thread_start_us":656685,"threads_started":1,"wal-append.queue_time_us":831056}
I20260709 12:25:30.929641  6455 rpcz_store.cc:276] 0709 12:25:29.667197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.667375 (+   178us) service_pool.cc:224] Handling call
0709 12:25:29.671969 (+  4594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.675552 (+  3583us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.675595 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.696286 (+ 20691us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.263211 (+566925us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:30.263250 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.263324 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.880390 (+617066us) raft_consensus.cc:1685] finished
0709 12:25:30.880410 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.880541 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":6905,"thread_start_us":448967,"threads_started":1,"wal-append.queue_time_us":768258}
I20260709 12:25:30.864089  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:30.849056  8332 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:30.874513  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:30.876192  8477 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:25:30.951905  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 77542.5us
I20260709 12:25:30.881309  8347 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:30.852417  8222 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:30.956931  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92934.5us
I20260709 12:25:30.877226  8436 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:30.972607  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 278691us
I20260709 12:25:30.973759  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:30.957865  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 240732us
W20260709 12:25:30.936108  8458 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 12:25:30.957890  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:31.029789  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71988.7us
I20260709 12:25:31.030759  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:30.880000  8323 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:30.954607  8332 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.877420  8348 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:30.876776  8245 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:30.885456  8198 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:30.876467  8440 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:30.892100  8200 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:30.852530  8274 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:30.958413  8428 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:30.877097  8435 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.032820  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:30.871018  8270 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:30.964434  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:30.960351  8399 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:31.040874  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 67202.7us
I20260709 12:25:31.042129  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:31.029415  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:25:30.894718  8349 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.056398  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:31.061511  6453 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:30.994043  8469 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:31.069947  6457 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:31.032922  8364 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:31.082145  8476 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.098351  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:31.099684  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:31.062736  8348 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:31.068063  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.008s	sys 0.011s
I20260709 12:25:31.083184  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:30.986151  8222 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.040369  8482 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:31.105690  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:31.106189  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:31.106606  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 187) took 1592 ms. Trace:
I20260709 12:25:31.106947  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:31.100208  8308 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:25:31.107230  6377 rpcz_store.cc:276] 0709 12:25:29.514245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.514393 (+   148us) service_pool.cc:224] Handling call
0709 12:25:29.521511 (+  7118us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.647333 (+125822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.647367 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.656511 (+  9144us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.239474 (+582963us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:30.239530 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.239620 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.106355 (+866735us) raft_consensus.cc:1685] finished
0709 12:25:31.106373 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.106532 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4586,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":5642,"thread_start_us":679741,"threads_started":2,"wal-append.queue_time_us":1184739}
W20260709 12:25:31.087699  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45703.6us
I20260709 12:25:31.095495  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:31.051963  8444 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:25:31.051949  8368 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.036s
I20260709 12:25:31.086560  8440 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.096240  8436 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.097530  8323 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:31.176373  8486 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:31.197525  8486 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:31.198556  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:31.177484  8481 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:31.065555  8409 consensus_queue.cc:1175] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 6 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: [5.7-5.7]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:25:31.206480  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 93024.8us
I20260709 12:25:31.109059  8480 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:31.160199  8449 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.660s	user 0.002s	sys 0.000s
I20260709 12:25:31.175112  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:31.175851  8478 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.185642  8274 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.186831  8332 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.105556  8198 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:25:31.128422  7319 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:31.099633  8399 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.188638  8270 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:31.125833  8475 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:31.109048  8364 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.100486  8432 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:31.088048  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:31.103031  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.008s	sys 0.012s
I20260709 12:25:31.196332  8436 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:31.200079  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 230653us
W20260709 12:25:31.200719  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 102507us
I20260709 12:25:31.217418  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:31.218845  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:31.199491  7318 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:31.210466  8349 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:31.241741  8489 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.243134  8364 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:31.244194  8435 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.198900  8486 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:31.255911  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 299576us
I20260709 12:25:31.259601  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1418) took 1365 ms. Trace:
I20260709 12:25:31.214668  8363 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.032s
I20260709 12:25:31.261662  6309 rpcz_store.cc:276] 0709 12:25:29.894105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.901756 (+  7651us) service_pool.cc:224] Handling call
0709 12:25:29.910540 (+  8784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.910824 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:30.291363 (+380539us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.259026 (+967663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:31.259094 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:31.259158 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.259247 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.259389 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10447,"prepare.queue_time_us":795950,"prepare.run_cpu_time_us":6658,"prepare.run_wall_time_us":67255,"raft.queue_time_us":11596,"thread_start_us":1327911,"threads_started":2,"wal-append.queue_time_us":795373}
I20260709 12:25:31.268589  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:31.269181  6302 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:31.270638  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:31.276276  6308 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:31.210500  8440 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.206424  8476 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.235705  8478 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.236191  8480 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.294373  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:31.304252  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:25:31.294797  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45337.3us
I20260709 12:25:31.283946  8245 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:25:31.276481  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 483) took 1158 ms. Trace:
I20260709 12:25:31.316393  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:31.322765  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:31.336457  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:31.337244  6382 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:25:31.305337  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:31.277062  8489 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.338860  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:31.267627  8467 tablet_copy_client.cc:323] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:31.340253  8349 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:31.340150  7318 rpcz_store.cc:276] 0709 12:25:30.102598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.104668 (+  2070us) service_pool.cc:224] Handling call
0709 12:25:31.198303 (+1093635us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:31.260574 (+ 62271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20089216}
I20260709 12:25:31.341058  8482 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:31.317569  8436 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.355238  8482 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.307116  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:31.355755  8482 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:31.339552  6307 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:31.356182  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:31.370991  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:31.291507  8222 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:31.374379  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 318670us
I20260709 12:25:31.377784  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:31.376977  6308 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:25:31.382790  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.323350  8332 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.393152  8399 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.394058  8478 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.393219  8489 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:31.395586  8459 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.621s	user 0.000s	sys 0.002s
I20260709 12:25:31.316383  8198 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.398278  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:31.353556  8487 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:31.365509  8349 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:31.316587  8490 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.149s
I20260709 12:25:31.306274  8364 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 12:25:31.322167  8444 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:25:31.403985  8348 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.409636  8476 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.413705  8444 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:25:31.426730  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55309.4us
I20260709 12:25:31.428529  8503 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:31.429356  8503 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.405055  8399 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:31.404549  8435 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:31.433415  8435 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:31.434712  8435 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.404731  8324 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:31.437505  8324 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.439617  8480 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.440397  8324 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:31.411811  8471 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.623s	user 0.000s	sys 0.001s
I20260709 12:25:31.450299  6302 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:31.458035  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:31.466159  8499 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:31.466810  8499 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.469549  8499 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.471736  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:31.472456  7316 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:31.403040  8332 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.426136  8444 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:31.404233  8198 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.456851  8436 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.462374  8222 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:31.464938  8363 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:31.427011  8348 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.406396  8487 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.475888  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:31.485421  8444 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.488384  8338 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.488930  8338 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.495044  8503 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.503681  8348 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.506059  8222 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.507958  8363 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:31.479791  8399 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.480086  8436 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.484612  8482 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.517721  8440 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.724714  8305 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.761480  8250 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:31.754922  8404 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.235s	user 0.003s	sys 0.000s
I20260709 12:25:31.741369  8338 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.515960  7319 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:25:31.723676  8354 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.764313  8354 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:31.752889  8328 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:31.767494  8328 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:31.768280  8328 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.769280  8328 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:31.769555  8432 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.770076  8432 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.772008  8432 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.762053  8305 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.774585  8305 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.758344  8460 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:31.778735  8323 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.779033  8460 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:31.779281  8323 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.779951  8460 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.781541  8323 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:31.729038  8487 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:31.783929  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:31.785691  8428 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.786155  8428 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.788228  8428 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.789167  8428 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.789687  8428 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.791927  8428 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.600996  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:31.512698  8456 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.166s	user 0.000s	sys 0.003s
I20260709 12:25:31.516441  8436 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.516681  8254 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:31.511730  8468 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.733s	user 0.001s	sys 0.000s
I20260709 12:25:31.807941  6452 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:31.808182  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 2483d7801887403e873adc62429fe046 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:25:31.548686  8444 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:31.813263  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:31.835762  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 151) took 1007 ms. Trace:
I20260709 12:25:31.836210  6454 rpcz_store.cc:276] 0709 12:25:30.828323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.828473 (+   150us) service_pool.cc:224] Handling call
0709 12:25:30.828761 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.880202 (+ 51441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.880236 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.880293 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:30.880332 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.835526 (+955194us) spinlock_profiling.cc:243] Waited 51.2 ms on lock 0x7b5400343854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:31.835574 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.835698 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117819264}
I20260709 12:25:31.837615  8308 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:31.838224  8308 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.840749  8308 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.841249  8308 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.841300  8460 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:31.841719  8308 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.842953  8481 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.843592  8481 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.503681  8337 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:31.526937  8368 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.847939  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:31.722270  8452 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.259s	user 0.002s	sys 0.002s
I20260709 12:25:31.823465  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 2483d7801887403e873adc62429fe046 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-2483d7801887403e873adc62429fe046
I20260709 12:25:31.823688  6305 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:31.826179  8198 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.826845  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:31.809651  8482 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.514323  8332 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.517493  8325 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:31.855962  8325 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.512548  8363 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.512374  8399 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.831444  8254 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:31.848730  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:31.850628  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:31.852959  8481 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.854969  8368 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:31.862735  6457 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:25:31.919235  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70606.7us
I20260709 12:25:31.926600  8338 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:31.929281  8368 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:31.929978  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 147) took 2362 ms. Trace:
I20260709 12:25:31.930349  6458 rpcz_store.cc:276] 0709 12:25:29.567545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.567711 (+   166us) service_pool.cc:224] Handling call
0709 12:25:29.568138 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.576700 (+  8562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.576738 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.580269 (+  3531us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.077547 (+497278us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:30.077597 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.089312 (+ 11715us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:31.721603 (+1632291us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:31.924162 (+202559us) raft_consensus.cc:1685] finished
0709 12:25:31.924193 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:31.929878 (+  5685us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":133,"spinlock_wait_cycles":74925952,"thread_start_us":460688,"threads_started":1,"wal-append.queue_time_us":674517}
I20260709 12:25:31.932293  8308 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:31.933012  8222 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:31.934477  8325 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:31.971714  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.977326  8058 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.977973  8058 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:31.981478  6303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.004s	sys 0.032s
W20260709 12:25:31.982096  6303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.004s	sys 0.032s
W20260709 12:25:31.995931  6381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.029s	sys 0.032s
W20260709 12:25:31.996595  6381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.030s	sys 0.032s
W20260709 12:25:31.946871  8261 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.021s	sys 0.044s
W20260709 12:25:31.946691  8346 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.026s	sys 0.034s
W20260709 12:25:32.053894  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.077656  8476 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:31.980837  8409 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:32.081434  8505 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:32.087009  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:32.090416  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.102545  6377 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:25:32.090690  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.091135  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.091940  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.105572  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.092702  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.094435  8469 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:31.981977  8337 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:31.995136  8271 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.054564  8346 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.027s	sys 0.034s
W20260709 12:25:32.055365  8261 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.995s	user 0.021s	sys 0.045s
I20260709 12:25:32.120781  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.121572  6377 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:25:32.104158  8058 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:32.106318  6384 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:32.082006  8267 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.117106  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:32.104949  8463 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:25:32.116322  8271 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.151865  8271 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:32.119040  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.160440  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 486) took 1211 ms. Trace:
I20260709 12:25:32.160516  6378 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:25:32.161141  7335 rpcz_store.cc:276] 0709 12:25:30.941697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.941960 (+   263us) service_pool.cc:224] Handling call
0709 12:25:30.942358 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.058270 (+115912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.058310 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.078952 (+ 20642us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.449380 (+370428us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:31.449442 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.449521 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.152672 (+703151us) raft_consensus.cc:1685] finished
0709 12:25:32.152701 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.152869 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86860,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":25896,"thread_start_us":457462,"threads_started":2,"wal-append.queue_time_us":1054694}
W20260709 12:25:32.128015  6380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.014s	sys 0.033s
I20260709 12:25:32.198995  8368 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:32.210865  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.215813  6380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.014s	sys 0.035s
I20260709 12:25:32.107211  8409 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.211095  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.220417  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:32.221812  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.078460  8476 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.223101  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.233966  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:32.273804  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:32.278146  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44275.2us
I20260709 12:25:32.279807  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:32.285864  6383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.010s	user 0.016s	sys 0.028s
W20260709 12:25:32.286759  6383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.017s	sys 0.028s
W20260709 12:25:32.287317  6310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.963s	user 0.020s	sys 0.031s
W20260709 12:25:32.287814  6310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.020s	sys 0.031s
I20260709 12:25:32.152930  8337 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.154021  8267 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:32.295662  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.033s	sys 0.051s
W20260709 12:25:32.297433  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.033s	sys 0.051s
I20260709 12:25:32.228730  8481 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:32.289333  8323 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:32.147900  8469 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.117740  8505 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:32.336859  6304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.035s	sys 0.057s
I20260709 12:25:32.307229  8482 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:32.223830  8368 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:32.289942  8349 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:32.289750  8463 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:32.338198  6455 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.099s	user 0.022s	sys 0.045s
W20260709 12:25:32.358642  6455 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.023s	sys 0.045s
W20260709 12:25:32.339190  6304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.035s	sys 0.057s
W20260709 12:25:32.359766  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.010s	sys 0.025s
W20260709 12:25:32.360306  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.010s	sys 0.025s
I20260709 12:25:32.311292  8481 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:32.365842  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.094s	user 0.038s	sys 0.017s
W20260709 12:25:32.367695  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.096s	user 0.038s	sys 0.018s
W20260709 12:25:32.345803  8396 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.021s	sys 0.013s
W20260709 12:25:32.306962  8479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.399s	user 0.027s	sys 0.069s
W20260709 12:25:32.346151  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.305042  6320 tablet_copy_source_session.cc:215] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:25:32.337352  8486 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.007s	sys 0.034s
I20260709 12:25:32.328023  8323 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:32.335352  8254 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:32.381008  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.347517  8482 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:32.384100  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:32.480324  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.519421  6378 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
W20260709 12:25:32.518190  6275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.020s	sys 0.022s
I20260709 12:25:32.521456  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:32.521584  6275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.163s	user 0.021s	sys 0.022s
I20260709 12:25:32.345134  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1527) took 1182 ms. Trace:
W20260709 12:25:32.418073  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.003s	sys 0.027s
I20260709 12:25:32.422623  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:32.408802  6301 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:32.386389  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:32.370826  8435 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.014s	sys 0.032s
I20260709 12:25:32.368640  8481 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.491597  8323 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:25:32.523231  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 1.16503e+06us
W20260709 12:25:32.384843  8396 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.009s	user 0.022s	sys 0.013s
W20260709 12:25:32.441691  8200 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.005s	sys 0.005s
I20260709 12:25:32.490042  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 205) took 1206 ms. Trace:
W20260709 12:25:32.523304  8486 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.010s	sys 0.040s
W20260709 12:25:32.487408  6302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.015s	user 0.003s	sys 0.011s
W20260709 12:25:32.559973  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.012s	sys 0.030s
I20260709 12:25:32.561247  6383 rpcz_store.cc:276] 0709 12:25:31.163051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.187345 (+ 24294us) service_pool.cc:224] Handling call
0709 12:25:31.187637 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.257919 (+ 70282us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:31.275719 (+ 17800us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.305753 (+1030034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.305811 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:32.305863 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.344525 (+ 38662us) spinlock_profiling.cc:243] Waited 70 ms on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:32.344723 (+   198us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.345041 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36114,"prepare.queue_time_us":21703,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":52,"raft.queue_time_us":23489,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":14304,"spinlock_wait_cycles":195350912,"thread_start_us":1011177,"threads_started":1,"wal-append.queue_time_us":1032878}
W20260709 12:25:32.524055  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.033s	sys 0.036s
W20260709 12:25:32.477005  8479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.563s	user 0.027s	sys 0.070s
I20260709 12:25:32.552006  8338 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:32.493404  8274 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:32.586169  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.035s	sys 0.037s
W20260709 12:25:32.581873  6302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.005s	sys 0.013s
I20260709 12:25:32.576789  6310 rpcz_store.cc:276] 0709 12:25:31.283403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.290046 (+  6643us) service_pool.cc:224] Handling call
0709 12:25:31.290424 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.290689 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:31.323746 (+ 33057us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.489454 (+1165708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.489546 (+    92us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:32.489616 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.489722 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.489947 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204226,"prepare.queue_time_us":45219,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":200844,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":31352,"thread_start_us":964216,"threads_started":1}
I20260709 12:25:32.374137  8463 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.593695  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:32.597949  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.409327  8399 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:32.595844  6378 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:32.383081  8349 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.598654  6385 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
W20260709 12:25:32.585440  8200 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.076s	user 0.005s	sys 0.006s
I20260709 12:25:32.593050  8335 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.609673  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:32.581032  8267 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:32.599557  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:32.517251  8444 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:32.479698  8409 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:32.601609  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:32.604709  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:32.580826  8509 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:32.525082  8482 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:32.516090  7319 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:25:32.605662  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.576189  8271 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:32.563580  8435 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.018s	sys 0.033s
I20260709 12:25:32.616150  8058 raft_consensus.cc:993] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:32.592375  8338 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:32.672888  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:32.614909  8335 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:32.614317  8464 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:32.672719  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.603600  8486 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:32.615880  8399 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:32.685796  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:32.692765  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.015s	sys 0.016s
I20260709 12:25:32.693231  8349 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:32.602303  8274 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:32.694803  8510 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:25:32.695031  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1429) took 1341 ms. Trace:
I20260709 12:25:32.619810  6385 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:32.759976  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.754943  6452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.029s	sys 0.049s
I20260709 12:25:32.695199  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2090) took 1853 ms. Trace:
W20260709 12:25:32.797143  6382 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.012s	sys 0.013s
I20260709 12:25:32.697870  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1633) took 1609 ms. Trace:
W20260709 12:25:32.839668  6376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.014s	sys 0.017s
W20260709 12:25:32.893680  6376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.797s	user 0.024s	sys 0.058s
W20260709 12:25:32.893329  6452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.036s	user 0.032s	sys 0.050s
W20260709 12:25:32.893330  6382 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.014s	sys 0.013s
I20260709 12:25:32.894292  6455 rpcz_store.cc:276] 0709 12:25:31.088721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.094917 (+  6196us) service_pool.cc:224] Handling call
0709 12:25:31.110021 (+ 15104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.184331 (+ 74310us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.192743 (+  8412us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.193415 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.359227 (+1165812us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:32.359288 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.359381 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.697594 (+338213us) raft_consensus.cc:1685] finished
0709 12:25:32.697627 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.697796 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":1122566,"threads_started":1,"wal-append.queue_time_us":1484728}
I20260709 12:25:32.894371  6450 rpcz_store.cc:276] 0709 12:25:30.841722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.842051 (+   329us) service_pool.cc:224] Handling call
0709 12:25:30.842639 (+   588us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:31.104503 (+261864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.104546 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:31.204950 (+100404us) log.cc:844] Serialized 511 byte log entry
0709 12:25:31.716279 (+511329us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:31.716344 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.784604 (+ 68260us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b54001c1e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:31.784675 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.694841 (+910166us) raft_consensus.cc:1685] finished
0709 12:25:32.694865 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.695128 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":20928,"spinlock_wait_cycles":600653056,"thread_start_us":467376,"threads_started":1,"wal-append.queue_time_us":1170422}
I20260709 12:25:32.893381  6309 rpcz_store.cc:276] 0709 12:25:31.353383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.353525 (+   142us) service_pool.cc:224] Handling call
0709 12:25:31.353896 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.354121 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.354151 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.354653 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.299463 (+944810us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:32.299518 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.299608 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.694811 (+395203us) raft_consensus.cc:1685] finished
0709 12:25:32.694834 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.694976 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":944335,"threads_started":1,"wal-append.queue_time_us":1338062}
W20260709 12:25:32.894265  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.016s	sys 0.040s
I20260709 12:25:32.840413  6459 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:32.827798  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.379s	user 0.035s	sys 0.082s
W20260709 12:25:32.900206  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.452s	user 0.036s	sys 0.082s
W20260709 12:25:32.900589  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.45219e+06us
W20260709 12:25:32.898619  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.017s	sys 0.040s
W20260709 12:25:32.898929  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.382s	user 0.023s	sys 0.024s
W20260709 12:25:32.901990  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 613175us
W20260709 12:25:32.902323  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.899521  6310 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:32.838138  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1530) took 1516 ms. Trace:
I20260709 12:25:32.903041  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:32.907056  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.907460  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.679811  8506 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:32.839227  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:32.825922  8521 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:25:32.912608  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.903514  6380 rpcz_store.cc:276] 0709 12:25:31.321652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.321807 (+   155us) service_pool.cc:224] Handling call
0709 12:25:31.322225 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.322480 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.322529 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.323125 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.216339 (+893214us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:32.216382 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.216452 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:32.715965 (+499513us) raft_consensus.cc:1685] finished
0709 12:25:32.715996 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.838072 (+122076us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":892786,"threads_started":1,"wal-append.queue_time_us":1273602}
I20260709 12:25:32.686019  8337 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:32.691393  8476 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:32.696039  8512 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:25:32.913283  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:32.917678  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.946836  6309 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:32.918018  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.918272  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.925902  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.695773  8500 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:32.831758  8338 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:32.826259  8368 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:32.839908  8399 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:32.692773  8254 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:32.956559  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.833637  8463 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:32.837533  8505 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:32.962765  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.963172  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.963644  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:32.914129  8460 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.072s	user 0.010s	sys 0.005s
I20260709 12:25:32.837335  8490 raft_consensus.cc:1081] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:32.679952  8444 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:32.986115  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:32.894106  8435 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:32.986781  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.986938  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.927687  6320 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:53970 (request call id 210) took 1112 ms. Trace:
I20260709 12:25:32.915000  8506 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:32.964056  8522 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:33.011621  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.967356  6310 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:57670
W20260709 12:25:33.046944  6377 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.015s	sys 0.064s
I20260709 12:25:33.048177  6310 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:25:33.048385  6377 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.015s	sys 0.064s
I20260709 12:25:33.048641  6310 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:33.017364  6457 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:33.050266  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.051895  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.053086  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.053720  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:33.056080  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.057490  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:33.037241  8354 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.272s	user 0.009s	sys 0.005s
I20260709 12:25:33.062847  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:32.989190  6309 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:33.035609  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.035673  6378 raft_consensus.cc:2410] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:32.836894  8271 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:33.015269  8516 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.299s	user 0.000s	sys 0.001s
W20260709 12:25:33.038779  6454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.185s	user 0.040s	sys 0.078s
W20260709 12:25:33.092460  6454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.239s	user 0.040s	sys 0.079s
I20260709 12:25:33.092700  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.015573  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.059906  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.032433  6385 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:33.034910  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.037045  8505 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:33.085054  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.085693  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.086062  6309 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:33.047221  8506 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:33.098294  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.014263  8481 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:32.965941  8500 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.063896  8486 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.107400  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.067211  8526 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:33.109933  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1521) took 2037 ms. Trace:
I20260709 12:25:33.045753  8254 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.115159  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.045611  8435 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.065778  8530 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:33.116777  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.084517  8347 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:33.092350  8436 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.084964  8327 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:33.111711  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.118065  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.119578  6380 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:33.122807  6384 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:33.111421  6310 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.116346  6381 rpcz_store.cc:276] 0709 12:25:31.072083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.072210 (+   127us) service_pool.cc:224] Handling call
0709 12:25:31.095057 (+ 22847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.210997 (+115940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.211026 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.221450 (+ 10424us) log.cc:844] Serialized 26 byte log entry
0709 12:25:31.997164 (+775714us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:31.997237 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.085774 (+ 88537us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54001946d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:32.085855 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.101261 (+1015406us) raft_consensus.cc:1685] finished
0709 12:25:33.101280 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.109866 (+  8586us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38191,"raft.queue_time_us":24975,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":15619,"spinlock_wait_cycles":38633344,"thread_start_us":739097,"threads_started":1,"wal-append.queue_time_us":1283080}
I20260709 12:25:33.107707  8528 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.127904  6451 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:33.126343  8527 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:25:33.038791  6320 rpcz_store.cc:276] 0709 12:25:31.807672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.807936 (+   264us) service_pool.cc:224] Handling call
0709 12:25:32.905306 (+1097370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64896}
I20260709 12:25:33.064358  8522 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.133716  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.126107  8347 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.135824  6310 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.148743  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.150131  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.150975  6380 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:33.148324  6453 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 1.
I20260709 12:25:33.157188  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.151510  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:33.159729  8511 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:25:33.160332  8500 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.161094  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1427) took 1879 ms. Trace:
I20260709 12:25:33.161619  6304 rpcz_store.cc:276] 0709 12:25:31.281252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.281448 (+   196us) service_pool.cc:224] Handling call
0709 12:25:31.281819 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.294170 (+ 12351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.294198 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.303144 (+  8946us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.359825 (+1056681us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:32.359879 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.359955 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.160845 (+800890us) raft_consensus.cc:1685] finished
0709 12:25:33.160871 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.161017 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11949,"raft.queue_time_us":113078,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":1056102,"threads_started":1,"wal-append.queue_time_us":1529451}
I20260709 12:25:33.163398  8463 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:33.163944  8463 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:33.164419  8463 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:33.172462  6378 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:33.176210  6308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.009s	sys 0.037s
W20260709 12:25:33.177906  6308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.009s	sys 0.037s
I20260709 12:25:33.141160  8349 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.113152  8486 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.121472  8526 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:33.189505  6300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.015s	sys 0.046s
I20260709 12:25:33.093976  8399 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:33.198113  6300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.015s	sys 0.046s
I20260709 12:25:33.130371  8436 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.130522  8327 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.156374  8460 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:25:33.140936  8530 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.138999  8528 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:33.234524  8524 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:25:33.190326  8349 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:33.328207  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.018s	sys 0.003s
W20260709 12:25:33.372725  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.018s	sys 0.003s
I20260709 12:25:33.373189  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1643) took 1525 ms. Trace:
W20260709 12:25:33.344964  8323 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.748s	user 0.006s	sys 0.006s
I20260709 12:25:33.373780  6459 rpcz_store.cc:276] 0709 12:25:31.847855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.848693 (+   838us) service_pool.cc:224] Handling call
0709 12:25:31.849015 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.900184 (+1051169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.372912 (+472728us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b540009a6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:33.373120 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279495040}
I20260709 12:25:33.374760  8323 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:33.284318  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.376969  8323 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.377583  8323 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:33.240475  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1640) took 1395 ms. Trace:
I20260709 12:25:33.378844  6454 rpcz_store.cc:276] 0709 12:25:31.845467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.847984 (+  2517us) service_pool.cc:224] Handling call
0709 12:25:31.848310 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.851415 (+  3105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.851460 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.853291 (+  1831us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.093109 (+1239818us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:33.093173 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.093274 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.240131 (+146857us) raft_consensus.cc:1685] finished
0709 12:25:33.240173 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.240389 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2571,"raft.queue_time_us":2744,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":6642,"thread_start_us":1239357,"threads_started":1,"wal-append.queue_time_us":1363601}
I20260709 12:25:33.381290  6454 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:33.384521  8327 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.385274  8327 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.385946  8327 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.387203  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1641) took 1541 ms. Trace:
I20260709 12:25:33.387372  6459 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:33.389194  8530 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.391757  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1430) took 1931 ms. Trace:
W20260709 12:25:33.340930  8534 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:25:33.343566  8522 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:33.399823  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 496852us
I20260709 12:25:33.424922  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:25:33.400511  6382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.000s	sys 0.035s
I20260709 12:25:33.337052  8399 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:33.232538  8444 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.235713  8335 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.237067  8347 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.359265  8338 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.347121  8254 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.342449  8526 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:33.426225  6382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.000s	sys 0.035s
I20260709 12:25:33.411831  6302 rpcz_store.cc:276] 0709 12:25:31.460201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.469132 (+  8931us) service_pool.cc:224] Handling call
0709 12:25:31.469654 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.470141 (+   487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.470185 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.471899 (+  1714us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.592412 (+1120513us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:32.592484 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.592585 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.391546 (+798961us) raft_consensus.cc:1685] finished
0709 12:25:33.391567 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.391696 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":576,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":163,"thread_start_us":1119987,"threads_started":1,"wal-append.queue_time_us":1491751}
I20260709 12:25:33.343672  8435 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.406683  6452 rpcz_store.cc:276] 0709 12:25:31.845807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.845954 (+   147us) service_pool.cc:224] Handling call
0709 12:25:31.846367 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:31.846694 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:31.846734 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:31.855079 (+  8345us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.895495 (+1040416us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:32.895528 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.895585 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.386914 (+491329us) raft_consensus.cc:1685] finished
0709 12:25:33.386945 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.387131 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1038389,"threads_started":1,"wal-append.queue_time_us":1316812}
I20260709 12:25:33.404402  8271 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.350895  8463 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.378093  8323 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.404752  8368 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:33.453739  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1532) took 1364 ms. Trace:
W20260709 12:25:33.451683  8482 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 12:25:33.403079  8506 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.457366  8481 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:33.471182  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.447295  8409 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:33.475376  8482 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.480549  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1537) took 1389 ms. Trace:
I20260709 12:25:33.490970  6376 rpcz_store.cc:276] 0709 12:25:32.091333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.094392 (+  3059us) service_pool.cc:224] Handling call
0709 12:25:32.094844 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.095293 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.095331 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.095878 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:25:32.894356 (+798478us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:32.894415 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.894482 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.480316 (+585834us) raft_consensus.cc:1685] finished
0709 12:25:33.480337 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.480489 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1488,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":128,"thread_start_us":797990,"threads_started":1,"wal-append.queue_time_us":1082090}
I20260709 12:25:33.481189  6301 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:33.482263  8349 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:33.483911  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.461772  6454 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:33.478117  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1543) took 1382 ms. Trace:
I20260709 12:25:33.494184  6377 rpcz_store.cc:276] 0709 12:25:32.095283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.147510 (+ 52227us) service_pool.cc:224] Handling call
0709 12:25:32.276928 (+129418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.277211 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:33.048882 (+771671us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.477650 (+428768us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.477735 (+    85us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.477809 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.477899 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.478044 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":710,"prepare.queue_time_us":925043,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"thread_start_us":1181701,"threads_started":2}
I20260709 12:25:33.480727  8506 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:33.503391  6274 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.923s	user 0.015s	sys 0.019s
I20260709 12:25:33.474185  8368 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.490398  8409 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.508092  6452 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:33.510941  6422 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.005s	sys 0.023s
I20260709 12:25:33.501024  8335 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
W20260709 12:25:33.511664  6274 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.015s	sys 0.019s
W20260709 12:25:33.517563  6422 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.845s	user 0.006s	sys 0.023s
I20260709 12:25:33.518430  8327 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.511126  8347 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.525846  8347 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.526317  8327 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.526938  8327 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.527957  8347 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.529072  8327 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.529462  8486 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.503391  8338 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:33.526636  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 980433us
I20260709 12:25:33.531370  8482 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.529256  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:33.521972  8543 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:33.517333  8409 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:33.523216  8469 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.523368  8528 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.529531  8270 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:33.511427  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.536089  8271 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:33.535430  8506 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.548856  8506 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:33.550058  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.762s (SENT) [suppressed 5 similar messages]
I20260709 12:25:33.551118  8506 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.551641  8506 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:33.518980  8464 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.552098  8506 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.552531  8464 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:33.553217  8436 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.554160  8506 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.534832  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:33.536942  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:33.559612  7335 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:25:33.521854  8523 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 12:25:33.556559  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:33.546815  8508 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.029s	user 0.009s	sys 0.013s
I20260709 12:25:33.560092  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:33.560528  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:33.563625  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:33.567788  8486 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:33.568205  8486 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.568384  8433 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:33.569120  8433 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.569607  8433 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:33.561177  8522 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:33.570976  8522 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.571432  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.573421  8522 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.572707  8347 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:33.576131  8368 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.576687  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.577358  6304 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:33.546314  8463 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:33.578632  8323 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.578769  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:33.557523  8538 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:25:33.579349  6304 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
I20260709 12:25:33.580240  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1621) took 1195 ms. Trace:
I20260709 12:25:33.546654  8543 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.580627  6308 rpcz_store.cc:276] 0709 12:25:32.384952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.385091 (+   139us) service_pool.cc:224] Handling call
0709 12:25:32.432143 (+ 47052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.432461 (+   318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.432518 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.433098 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.181947 (+748849us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:33.182009 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.182109 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.579996 (+397887us) raft_consensus.cc:1685] finished
0709 12:25:33.580015 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.580177 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":657264,"threads_started":1,"wal-append.queue_time_us":1025647}
I20260709 12:25:33.559195  8270 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.583695  7332 raft_consensus.cc:2410] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:25:33.584494  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.577431  8490 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:33.568606  8486 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:33.595505  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.582237  8543 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.608620  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.621569  6450 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:33.622773  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.623538  6450 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:33.624476  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.625134  6450 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:33.562855  8271 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:33.656008  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:33.653607  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.657326  6308 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:33.597046  8436 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:33.662668  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.664150  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:33.665907  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:33.598747  8490 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:33.670158  8368 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:33.677873  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.678861  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.680137  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:33.661182  8363 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.326s	user 0.013s	sys 0.026s
I20260709 12:25:33.681111  8541 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:33.685014  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.689724  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1625) took 1277 ms. Trace:
I20260709 12:25:33.673847  8338 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.673110  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:33.675510  8327 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.747484  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:33.676115  8270 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.713363  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.715143  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.748615  7335 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:33.750895  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.755620  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.755774  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 1965) took 1154 ms. Trace:
I20260709 12:25:33.756657  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.762780  6380 rpcz_store.cc:276] 0709 12:25:32.601558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.158567 (+557009us) service_pool.cc:224] Handling call
0709 12:25:33.159407 (+   840us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.159620 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:33.547133 (+387513us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.755102 (+207969us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.755183 (+    81us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:33.755294 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.755431 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.755634 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":73724,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":151,"thread_start_us":582765,"threads_started":2}
W20260709 12:25:33.724582  8363 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.396s	user 0.013s	sys 0.026s
I20260709 12:25:33.769749  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.698546  8506 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.771319  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.769855  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.773310  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.786192  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.787006  6308 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:33.791728  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.794072  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.794853  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:33.795658  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.796630  6385 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:33.690752  8368 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.724136  8469 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:33.795650  7335 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:33.798748  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.805168  6381 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
W20260709 12:25:33.801158  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69456us
I20260709 12:25:33.802032  8058 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:33.806458  8058 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.827140  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.828593  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.829631  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.764828  8327 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.832679  8327 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.724529  8490 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.460496  6382 rpcz_store.cc:276] 0709 12:25:32.089549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.092980 (+  3431us) service_pool.cc:224] Handling call
0709 12:25:32.093330 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:32.093656 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:32.896740 (+803084us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.453253 (+556513us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.453332 (+    79us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.453400 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.453490 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.453651 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1055293,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":2119,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4151,"thread_start_us":1358712,"threads_started":2,"wal-append.queue_time_us":499720}
I20260709 12:25:33.834192  8490 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:33.836570  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.837212  6452 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
W20260709 12:25:33.798285  8537 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 12:25:33.797479  8515 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:25:33.851234  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:33.838881  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.839480  6378 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:33.853569  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 1960) took 1254 ms. Trace:
I20260709 12:25:33.853607  6378 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:33.843606  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.854067  6379 rpcz_store.cc:276] 0709 12:25:32.599318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.608876 (+  9558us) service_pool.cc:224] Handling call
0709 12:25:32.609289 (+   413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.614806 (+  5517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.614842 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.687575 (+ 72733us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.480108 (+792533us) spinlock_profiling.cc:243] Waited 36.8 ms on lock 0x7b180010a260. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:25:33.694187 (+214079us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:33.694223 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.694287 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.834407 (+140120us) raft_consensus.cc:1685] finished
0709 12:25:33.834427 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.853506 (+ 19079us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49956,"raft.queue_time_us":362,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":72821,"spinlock_wait_cycles":84594048,"thread_start_us":213890,"threads_started":1,"wal-append.queue_time_us":1134754}
I20260709 12:25:33.859252  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.762022  8338 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.864509  6450 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:33.871388  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:33.872148  6381 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:33.872737  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.873966  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.875051  8058 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:33.876482  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.877313  6452 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:33.841835  8528 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:33.882961  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:33.883488  8363 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:33.883667  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:33.884104  8363 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:33.884848  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.886705  8551 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:25:33.730141  6300 rpcz_store.cc:276] 0709 12:25:32.412625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.412848 (+   223us) service_pool.cc:224] Handling call
0709 12:25:32.413409 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.413927 (+   518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.413972 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.423024 (+  9052us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.389974 (+966950us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:33.688956 (+298982us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.689318 (+   362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.689420 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.689475 (+    55us) raft_consensus.cc:1685] finished
0709 12:25:33.689493 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.689641 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":468,"raft.queue_time_us":4097,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":51830,"thread_start_us":1171685,"threads_started":2,"wal-append.queue_time_us":1017865}
I20260709 12:25:33.892100  8338 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.893435  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.894655  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.895259  6459 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:33.883697  6452 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:33.896200  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.798580  8506 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.895933  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.897401  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.900702  6304 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:33.902707  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:33.896111  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.905704  6459 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:33.879402  6378 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.913592  6378 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.915287  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.863674  8481 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:33.920436  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.920614  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:33.913043  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:33.837774  8469 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:33.954082  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:33.844041  8409 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.956967  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.957356  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:33.958843  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.959239  6453 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
W20260709 12:25:33.964844  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.886404  8363 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:33.969714  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:33.975200  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:33.975369  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.975984  6453 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:33.883036  8528 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:33.981410  6450 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:33.967226  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:33.976068  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:33.983578  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:34.005824  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:34.009583  6458 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:25:34.010345  6458 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:34.010797  6458 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:33.972692  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:33.958662  8481 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.015854  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:34.024070  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.024322  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:34.002300  8540 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.036026  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:34.036108  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.009131  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.014930  8498 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:34.014444  8271 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:34.012584  8323 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:34.072666  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115398us
I20260709 12:25:34.077903  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:34.073465  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:34.016511  8566 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:34.079221  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:34.082237  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:34.085527  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.086983  6450 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:25:34.035537  8543 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:34.091149  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:34.095237  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.096607  8335 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:34.097224  8335 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:34.097586  8433 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.002s
I20260709 12:25:34.096969  6453 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:34.097774  8335 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:34.098048  8433 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:34.099153  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.100153  8433 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:34.075621  8498 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.076884  8271 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:34.104616  8553 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:25:34.104725  8274 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:34.093786  8543 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:34.105690  8543 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.106007  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2102) took 1183 ms. Trace:
I20260709 12:25:34.080782  8323 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:34.106539  6451 rpcz_store.cc:276] 0709 12:25:32.922184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.180461 (+258277us) service_pool.cc:224] Handling call
0709 12:25:33.372204 (+191743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.372849 (+   645us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.372879 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.373580 (+   701us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.607760 (+234180us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:33.607802 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.607876 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.105680 (+497804us) raft_consensus.cc:1685] finished
0709 12:25:34.105712 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.105916 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":250,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":7369,"thread_start_us":233593,"threads_started":1,"wal-append.queue_time_us":522621}
I20260709 12:25:34.108176  8323 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.110399  8368 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:34.111227  8368 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:34.112169  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.112617  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.107808  8562 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:25:34.120638  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 162) took 1202 ms. Trace:
I20260709 12:25:34.121039  6455 rpcz_store.cc:276] 0709 12:25:32.918328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.063800 (+145472us) service_pool.cc:224] Handling call
0709 12:25:33.103520 (+ 39720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.121343 (+ 17823us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:33.507007 (+385664us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.963055 (+456048us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.963153 (+    98us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:33.963264 (+   111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.120437 (+157173us) spinlock_profiling.cc:243] Waited 17.6 ms on lock 0x7b54002c8154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:34.120478 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.120586 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40572,"prepare.queue_time_us":389696,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":126801,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":9134,"spinlock_wait_cycles":40520448,"thread_start_us":793929,"threads_started":3,"wal-append.queue_time_us":405923}
I20260709 12:25:34.123898  8435 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:34.132011  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46853us
I20260709 12:25:34.133283  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:34.133785  8274 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:25:34.093443  7319 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:34.090981  8526 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:34.090731  8347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:53964 (request call id 11) took 9655 ms. Trace:
I20260709 12:25:34.103418  8498 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:34.101091  8554 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:34.166575  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.103534  8271 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:34.169016  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86892.9us
I20260709 12:25:34.131567  8547 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:25:34.219120  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:34.170378  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1631) took 1742 ms. Trace:
I20260709 12:25:34.222370  6301 rpcz_store.cc:276] 0709 12:25:32.427589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.033503 (+605914us) service_pool.cc:224] Handling call
0709 12:25:33.055146 (+ 21643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.494865 (+439719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.170056 (+675191us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b54002071d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:34.170305 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":979399040}
I20260709 12:25:34.223573  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.224135  6301 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:34.226046  6301 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:34.171519  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2105) took 1248 ms. Trace:
I20260709 12:25:34.161968  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:34.181689  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.100987  8566 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:34.231578  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84508.3us
I20260709 12:25:34.186774  6305 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:34.102705  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.204586  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2096) took 1284 ms. Trace:
I20260709 12:25:34.103026  8506 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:34.103228  8338 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:34.209066  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.210220  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.211201  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.370203  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:34.212251  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.124686  8435 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.152846  6458 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.165194  8526 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:34.343762  6308 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:34.227267  6454 rpcz_store.cc:276] 0709 12:25:32.923262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.382273 (+459011us) service_pool.cc:224] Handling call
0709 12:25:33.405898 (+ 23625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.496542 (+ 90644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.171121 (+674579us) spinlock_profiling.cc:243] Waited 55.6 ms on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:34.171437 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":127828736}
I20260709 12:25:34.338249  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:34.398164  6381 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.351068  6456 rpcz_store.cc:276] 0709 12:25:32.920124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.096453 (+176329us) service_pool.cc:224] Handling call
0709 12:25:33.096846 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.388230 (+291384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.388278 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.388940 (+   662us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.632293 (+243353us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:33.632348 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.204188 (+571840us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b5400163ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:34.204230 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.204277 (+    47us) raft_consensus.cc:1685] finished
0709 12:25:34.204290 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.204514 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":669179264,"thread_start_us":242749,"threads_started":1,"wal-append.queue_time_us":333731}
I20260709 12:25:34.389139  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:34.390262  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:34.390607  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 171458us
I20260709 12:25:34.401165  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:34.384835  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.386053  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:34.352725  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 273583us
I20260709 12:25:34.405320  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.405750  6458 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:34.406505  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.410064  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:34.411648  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 277544us
I20260709 12:25:34.428905  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.328027  8569 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:34.337492  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.386870  7318 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:25:34.431561  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:34.354815  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.441565  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:34.433801  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.441972  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.444509  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.368048  8338 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:34.450755  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.451259  8583 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:34.435854  8463 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.452713  8572 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:34.453449  8557 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:34.383051  8469 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:34.406741  8526 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:34.456177  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:34.430275  8556 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:34.468047  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:34.472539  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.438037  8529 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:34.433923  8541 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:34.434799  8569 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:34.492671  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.459321  8338 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.458016  8583 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.505237  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.457334  8572 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:34.516364  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.479316  8469 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:34.500550  8274 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:34.530429  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.501319  8582 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:34.539448  8582 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:34.540565  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.205816  8347 rpcz_store.cc:276] 0709 12:25:24.420444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.420715 (+   271us) service_pool.cc:224] Handling call
0709 12:25:24.481701 (+ 60986us) log.cc:844] Serialized 485 byte log entry
0709 12:25:24.587161 (+105460us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b18002bee20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3227b26 00007f6fa322149a 00007f6fa3220714
0709 12:25:24.587592 (+   431us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b54003168a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3227b26 00007f6fa322149a 00007f6fa3220714 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634
0709 12:25:24.591003 (+  3411us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b54003168a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3220142 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634 0000000000388479 00007f6fa01ba475
0709 12:25:24.604797 (+ 13794us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b18002bee20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:25:24.607494 (+  2697us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b18002bee20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240
0709 12:25:24.608319 (+   825us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18002bee20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3220142 00007f6fa3200eaa 00007f6fa3207d53
0709 12:25:24.608868 (+   549us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b54003168a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3220142 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634 0000000000388479 00007f6fa01ba475
0709 12:25:24.643394 (+ 34526us) spinlock_profiling.cc:243] Waited 5.05 ms on lock 0x7b54003168a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3227b26 00007f6fa322149a 00007f6fa3220714 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634
0709 12:25:34.053539 (+9410145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36839,"raft.run_cpu_time_us":10902,"raft.run_wall_time_us":194825,"spinlock_wait_cycles":139679744,"thread_start_us":41336,"threads_started":1,"wal-append.queue_time_us":72999,"wal-append.run_cpu_time_us":2129,"wal-append.run_wall_time_us":1013082}
I20260709 12:25:34.525775  8572 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.547734  8435 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:34.533675  8274 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:34.538511  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.560640  8274 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.565771  8467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2483d7801887403e873adc62429fe046. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:25:34.538797  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.559154  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.508050  8583 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:34.562172  8274 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:34.496874  8569 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.537323  6380 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:34.576869  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:34.579713  6378 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.666s	user 0.016s	sys 0.026s
I20260709 12:25:34.593761  7317 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:25:34.592742  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 213060us
I20260709 12:25:34.529723  8433 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:34.588629  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.588371  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:34.507613  8368 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:34.528060  8481 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:34.582944  6452 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.671s	user 0.009s	sys 0.014s
I20260709 12:25:34.542275  8582 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:34.598915  6384 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:34.603317  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.604091  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.608522  6452 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:34.615669  6451 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:34.610829  6384 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:34.591162  8529 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:34.629768  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.635360  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:34.592483  8435 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:34.599087  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.645624  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:34.619690  8486 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.960s	user 0.013s	sys 0.006s
I20260709 12:25:34.582545  8585 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:25:34.599810  8433 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:34.679418  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.625289  6384 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.681864  6384 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.658524  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:34.664449  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.604476  8368 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:34.642570  8490 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.808s	user 0.009s	sys 0.010s
W20260709 12:25:34.683676  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.688481  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 244067us
I20260709 12:25:34.693679  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.699174  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.699630  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:34.697479  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 265499us
I20260709 12:25:34.642314  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.715555  6377 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:34.717162  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:34.686060  8433 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.715492  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:34.698791  8368 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:34.747056  8585 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:34.749802  8271 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:34.751157  6384 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:25:34.753989  8335 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.005s
W20260709 12:25:34.739883  8540 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.702s	user 0.006s	sys 0.009s
I20260709 12:25:34.665486  8435 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:34.761716  6381 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
W20260709 12:25:34.770480  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.683123  8486 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.644538  8529 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:34.776017  8323 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.652s	user 0.010s	sys 0.000s
I20260709 12:25:34.755226  8556 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:34.685734  8586 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:25:34.800030  6307 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:34.801198  8583 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:34.801952  8556 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:34.802520  8592 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:34.802793  8560 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.781s	user 0.004s	sys 0.001s
I20260709 12:25:34.803038  8323 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.807614  8526 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:34.809435  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1667) took 1001 ms. Trace:
I20260709 12:25:34.814736  8347 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:34.815120  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 207) took 1099 ms. Trace:
I20260709 12:25:34.863636  6377 rpcz_store.cc:276] 0709 12:25:33.716092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.786227 (+ 70135us) service_pool.cc:224] Handling call
0709 12:25:34.814875 (+1028648us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b540010a4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:34.815049 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121932672}
I20260709 12:25:34.819682  8545 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:34.821009  8593 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:34.828361  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.828341  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2129) took 1035 ms. Trace:
I20260709 12:25:34.858395  6451 rpcz_store.cc:276] 0709 12:25:33.808034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.108951 (+300917us) service_pool.cc:224] Handling call
0709 12:25:34.615387 (+506436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.616616 (+  1229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.809190 (+192574us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b540009b350. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:34.809359 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1163576576}
I20260709 12:25:34.796628  8540 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:34.789984  8594 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:34.872233  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 155471us
I20260709 12:25:34.827901  8556 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:34.795953  8591 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:34.885784  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 240274us
W20260709 12:25:34.880214  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 241212us
W20260709 12:25:34.866406  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 368573us
I20260709 12:25:34.890908  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:34.860858  8464 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.308s	user 0.014s	sys 0.000s
I20260709 12:25:34.889200  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:34.827562  8592 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:34.794767  8335 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:34.867767  6453 rpcz_store.cc:276] 0709 12:25:33.793192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.096360 (+303168us) service_pool.cc:224] Handling call
0709 12:25:34.096641 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.096852 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:34.390105 (+293253us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.510925 (+120820us) log.cc:844] Serialized 14 byte log entry
0709 12:25:34.816706 (+305781us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b18000c49c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1
0709 12:25:34.817943 (+  1237us) spinlock_profiling.cc:243] Waited 5.44 ms on lock 0x7b18000c49c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:25:34.827976 (+ 10033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:34.828039 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:34.828089 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.828154 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.828285 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22625,"prepare.queue_time_us":23405,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":33500,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":35380,"spinlock_wait_cycles":21338368,"thread_start_us":391195,"threads_started":2,"wal-append.queue_time_us":337957}
I20260709 12:25:34.914968  8540 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:34.916529  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.863530  8347 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:34.919387  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:34.866636  8545 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:34.884640  8571 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:34.926750  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:34.870875  8593 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:34.885324  8594 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:34.940466  8590 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:25:34.949469  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:34.934585  8347 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.950323  7317 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:34.922631  8592 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:34.939320  8593 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:34.937183  8530 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:34.967820  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 251637us
W20260709 12:25:34.968350  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 257815us
I20260709 12:25:34.920866  8464 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:34.971259  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:34.971619  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1639) took 2521 ms. Trace:
I20260709 12:25:34.941423  8594 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:34.986379  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:34.946740  8591 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:34.947481  8363 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:34.958460  8589 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:34.999462  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:34.989070  6309 rpcz_store.cc:276] 0709 12:25:32.449823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.098865 (+649042us) service_pool.cc:224] Handling call
0709 12:25:33.099197 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.473716 (+374519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.473745 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.477125 (+  3380us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.697563 (+220438us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:33.807635 (+110072us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.807852 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.807919 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.802050 (+994131us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:34.971322 (+169272us) raft_consensus.cc:1685] finished
0709 12:25:34.971415 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.971555 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1361,"prepare.queue_time_us":157380,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":76912,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":22023,"spinlock_wait_cycles":27685632,"thread_start_us":656758,"threads_started":3,"wal-append.queue_time_us":494067}
I20260709 12:25:35.006623  6459 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:25:35.022804  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96041.2us
I20260709 12:25:35.000111  8591 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:35.009161  8557 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.026979  8614 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:35.001521  8586 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:25:35.036758  8557 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:35.031975  8486 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:34.970808  8545 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:35.068254  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 155198us
I20260709 12:25:35.036881  8271 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:35.079126  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 108092us
I20260709 12:25:35.083643  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:35.084889  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:35.085348  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58469.3us
W20260709 12:25:35.071755  8572 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.003s
I20260709 12:25:35.067914  8557 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:35.101223  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.105372  8601 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:35.081751  8529 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.110972  8529 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:35.111393  8529 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:35.111760  8529 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:35.074257  8435 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.074736  8323 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:35.082098  8585 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:35.121268  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.122301  8545 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.101763  8486 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.117430  8335 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.126295  8585 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.103121  8572 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.183851  8586 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:25:35.185420  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.186295  8464 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:35.157023  8486 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:35.182771  8585 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.190924  8605 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:35.163285  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:35.209059  8486 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:35.225790  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:35.228783  8543 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.123s	user 0.008s	sys 0.012s
I20260709 12:25:35.229238  8543 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.230356  7329 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:35.232265  8338 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.713s	user 0.011s	sys 0.000s
W20260709 12:25:35.249356  8498 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.030s	user 0.002s	sys 0.012s
I20260709 12:25:35.250188  8498 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.250893  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1447) took 1508 ms. Trace:
I20260709 12:25:35.251596  7335 raft_consensus.cc:2368] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:25:35.253284  8528 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.254154  6310 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:35.156719  8591 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.257031  8498 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.251291  6307 rpcz_store.cc:276] 0709 12:25:33.742894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.771034 (+ 28140us) service_pool.cc:224] Handling call
0709 12:25:35.250661 (+1479627us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002df0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:35.250825 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2357847936}
W20260709 12:25:35.259020  8569 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.011s
I20260709 12:25:35.261590  8566 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:35.260879  8569 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.265354  8526 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:35.267696  8612 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:35.268448  8409 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:35.273816  8605 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:35.274430  8605 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:35.275745  7320 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:35.275529  8605 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.277375  8363 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:35.279537  7335 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:35.280462  8274 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.681s	user 0.006s	sys 0.012s
I20260709 12:25:35.232726  8338 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.288862  8338 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:35.289075  8433 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.005s
I20260709 12:25:35.290254  8433 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.274088  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:35.292586  8433 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.291998  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.296094  6453 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.303264  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1640) took 1124 ms. Trace:
I20260709 12:25:35.303722  6310 rpcz_store.cc:276] 0709 12:25:34.178968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.182645 (+  3677us) service_pool.cc:224] Handling call
0709 12:25:35.253950 (+1071305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.254592 (+   642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.303043 (+ 48451us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54002df0d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:35.303203 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2378112256}
W20260709 12:25:35.318130  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 216798us
I20260709 12:25:35.260118  8572 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:35.330549  8565 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.317s	user 0.006s	sys 0.001s
I20260709 12:25:35.324869  6453 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:35.318971  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.327039  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.333709  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.334790  6458 raft_consensus.cc:2368] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4.
I20260709 12:25:35.334818  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:35.335493  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.339437  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.340081  6452 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:35.343600  7285 leader_election.cc:400] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4.
I20260709 12:25:35.344024  7285 leader_election.cc:403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:25:35.346764  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:35.348076  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.350059  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.350841  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.351130  6301 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:35.351195  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.352777  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.352861  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.295915  8605 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.354065  6309 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:35.354890  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.349236  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.360050  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.360987  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.327170  8409 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:35.356170  8436 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.337172  8338 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:35.376276  8592 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.380048  8569 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:35.380348  8585 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.384339  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:35.384145  8529 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.357386  8581 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:35.385396  8409 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.385875  8581 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:35.386660  8543 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.386895  8581 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.387750  8409 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:35.389372  8368 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.599s	user 0.004s	sys 0.009s
I20260709 12:25:35.389329  8594 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.359249  8624 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:35.391731  8347 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.392565  8624 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:35.392855  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.366955  8556 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.393575  8624 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.359170  8363 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:35.394730  8467 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43752 (request call id 480) took 5606 ms. Trace:
I20260709 12:25:35.394843  8363 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:35.395303  8363 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.395212  8467 rpcz_store.cc:276] 0709 12:25:29.812811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.824006 (+ 11195us) service_pool.cc:224] Handling call
0709 12:25:30.772737 (+948731us) ts_tablet_manager.cc:934] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
0709 12:25:34.517880 (+3745143us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:34.941625 (+423745us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:34.946569 (+  4944us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:35.290349 (+343780us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:35.338074 (+ 47725us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:35.394543 (+ 56469us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:35.394622 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":51653,"tablet-copy.queue_time_us":672248,"thread_start_us":743809,"threads_started":2}
I20260709 12:25:35.396277  8588 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:35.396745  8588 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:35.397043  8467 tablet_copy_client.cc:806] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:35.397712  8467 tablet_copy_client.cc:670] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:35.345793  8572 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:35.366016  8593 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.373854  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.399590  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2124) took 1607 ms. Trace:
I20260709 12:25:35.399991  6459 rpcz_store.cc:276] 0709 12:25:33.792015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.920089 (+128074us) service_pool.cc:224] Handling call
0709 12:25:35.399396 (+1479307us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:35.399524 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2490944896}
W20260709 12:25:35.401794  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54177.1us
I20260709 12:25:35.405925  6452 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:35.407266  6456 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:35.408591  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.409257  6456 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.411648  6454 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:35.414079  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.415791  6451 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.417500  6458 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:35.425920  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:35.430855  6309 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:35.432678  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.433247  6301 raft_consensus.cc:2368] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4.
I20260709 12:25:35.397641  8588 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.448343  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.449790  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.446105  6302 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:35.447153  6305 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:35.452077  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.465714  6307 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.390081  8368 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:35.417182  8547 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.989s	user 0.014s	sys 0.036s
I20260709 12:25:35.464666  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.439265  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.475692  6302 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:35.469714  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.484447  6451 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
W20260709 12:25:35.485982  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.487814  6302 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:35.417996  8585 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:35.489137  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 410024us
I20260709 12:25:35.447685  8610 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.440184  8436 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4.
I20260709 12:25:35.444763  8556 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.505255  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:35.475293  8547 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.016s	sys 0.037s
I20260709 12:25:35.522356  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.519943  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.434862  8486 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.366720  8607 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.137s
I20260709 12:25:35.452525  8529 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.441483  8409 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:35.496243  8592 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.501689  8610 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.526855  6459 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:35.526191  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.540603  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.543644  6307 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:35.544137  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:35.545094  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.547114  6379 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:35.548607  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.548940  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.549117  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:35.540547  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.530028  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.549602  6378 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:35.534289  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:35.527763  6302 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:35.550380  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:35.495692  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 410859us
I20260709 12:25:35.533826  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.551800  6459 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:35.553224  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.527062  8463 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:25:35.553162  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:35.557542  8499 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:35.557837  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.557993  8499 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:35.558368  8499 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.551743  8338 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:35.559707  8338 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:35.560202  8338 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.563506  8463 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:35.564527  8490 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:35.564895  8490 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:35.558933  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.565687  8490 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.566610  6457 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:35.566875  8490 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.557063  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.571365  6378 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:35.572734  8433 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:35.568802  6457 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:35.570108  8584 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:35.574033  8584 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:35.574946  8584 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.576303  8584 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.577999  8338 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.582803  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:35.583636  8617 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:35.583894  8347 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.584194  8543 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:35.584671  8594 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.584771  8543 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:35.586131  8593 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.583909  6303 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:25:35.587749  8368 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.545045  8552 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:25:35.585847  8543 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.542905  8582 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:25:35.590704  8569 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:35.591176  8569 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:35.591638  8486 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.530360  8557 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.591989  8569 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.593274  8569 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.593883  8557 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.588991  8552 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:35.594765  8617 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:35.552793  8254 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:25:35.595748  8617 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.596549  8254 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:35.596999  8583 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.597136  8617 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.554746  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:35.547492  8409 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:35.598348  8363 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.599520  6450 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:35.599212  8409 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.600943  8588 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:35.603129  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62660.2us
I20260709 12:25:35.608807  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.609517  6379 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:35.610803  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.611546  6384 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:35.612195  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:35.612740  6381 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:35.594767  8552 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:35.628021  8335 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.596954  8254 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:35.630764  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:35.631296  8254 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.632045  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:35.632545  8254 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.633127  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:35.635371  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.636169  8335 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:35.636559  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.638700  8613 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:35.641469  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.643435  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:35.645612  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.645844  6459 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:35.647681  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.648156  6456 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:35.649559  6453 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:35.649811  6383 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:25:35.653589  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 327273us
I20260709 12:25:35.655411  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.628981  8552 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.656145  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:35.660327  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.662516  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:35.662642  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.663638  6380 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:35.666571  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.666831  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.667323  6459 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:35.666886  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.668828  6303 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:35.669885  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:35.666095  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.676406  8572 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.676892  8572 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:35.677276  8572 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.678954  7316 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
W20260709 12:25:35.684458  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 305413us
I20260709 12:25:35.685711  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:35.687017  6456 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:35.661580  8591 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.688719  8469 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:25:35.683320  8583 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:35.683544  8582 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:25:35.656661  6381 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
W20260709 12:25:35.701404  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 144494us
I20260709 12:25:35.702136  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.702559  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.708034  6303 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:35.708128  6301 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.702896  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.704391  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.709273  6305 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:35.709270  6309 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 1.
W20260709 12:25:35.714363  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.712402  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:35.722175  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:35.723035  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.723145  6305 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:35.725855  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.726629  6310 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.734963  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:35.729816  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:35.737771  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:35.743220  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.736800  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:35.745281  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:35.706562  8552 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:35.706966  8556 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:35.718914  8541 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:35.762164  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.763512  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.764730  6307 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:35.765738  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.768599  6307 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:25:35.780578  8577 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.009s	sys 0.024s
I20260709 12:25:35.769577  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.769927  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:35.786399  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.787837  6458 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:35.791096  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:35.791687  6305 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:35.752315  8594 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:35.752550  8632 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.753374  8499 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:35.804320  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:35.805193  6310 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
W20260709 12:25:35.781143  8577 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.009s	sys 0.025s
I20260709 12:25:35.789726  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.810081  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.811054  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:35.794672  8635 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:25:35.741577  8605 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:25:35.740837  8631 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:25:35.753038  8556 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:35.794173  6458 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:35.781749  8469 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:25:35.758000  8584 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:35.767907  8641 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:35.770682  8347 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:35.792943  8338 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:35.831036  6301 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:35.799837  8594 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:35.739032  8464 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:25:35.832135  8363 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:35.820334  8632 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:25:35.848800  8499 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:35.853678  6274 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.530s (SENT)
W20260709 12:25:35.858734  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 337930us
I20260709 12:25:35.817371  8409 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.819453  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:35.834671  8556 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:35.840863  6305 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:35.872540  8584 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:35.834264  8541 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.922603  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.874655  8594 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:35.932331  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 194672us
I20260709 12:25:35.929816  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:35.879352  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 101119us
W20260709 12:25:35.933030  8636 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:25:35.960992  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:35.935983  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 213865us
I20260709 12:25:35.977200  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:35.915748  8569 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:25:35.961243  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 150219us
I20260709 12:25:35.961138  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:35.977975  8569 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:35.978397  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:35.965447  6453 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:25:35.975553  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 121893us
I20260709 12:25:35.979130  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:35.980013  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:35.929692  8338 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:25:35.980711  6274 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.589s (SENT)
I20260709 12:25:35.980927  8338 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:35.981181  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:35.982378  6274 leader_election.cc:336] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.637s (SENT)
I20260709 12:25:35.982674  8605 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:35.982890  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:35.978549  8569 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:35.982120  8338 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.983717  8605 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:35.985065  6274 leader_election.cc:336] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.735s (SENT)
I20260709 12:25:35.926611  8610 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:35.985489  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:35.935889  8363 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:35.874367  8347 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:35.992862  6274 leader_election.cc:336] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.093s (SENT)
I20260709 12:25:36.001591  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:35.955122  8583 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.005434  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.006065  8467 tablet_copy_client.cc:538] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:25:36.011355  6274 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.117s (SENT)
I20260709 12:25:35.975736  8603 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:36.029593  8603 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:35.985687  8498 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:36.019250  8271 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:36.019548  8644 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:25:36.019941  8499 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:36.027786  8639 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:25:35.997232  8605 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.033059  6454 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:25:36.033219  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 72296.4us
I20260709 12:25:36.002799  8541 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:25:36.017021  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.035272  6450 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:36.017657  8610 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:25:36.037568  6351 connection.cc:256] RPC call timeout handler was delayed by -1.1179218949996539s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16200000000000003s for a response
I20260709 12:25:36.077765  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:36.039796  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60778.9us
I20260709 12:25:36.028393  8584 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.034236  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.026063  8433 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:25:36.006021  8572 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:35.994719  8622 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.042094  8271 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:36.071269  8650 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:25:36.053287  8656 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:36.072885  8643 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:25:36.074898  8582 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:36.080809  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.065703  8649 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:36.094288  8663 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.053813  8463 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.118743  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.120062  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:25:36.120427  8647 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:25:36.092024  8604 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:36.092123  8271 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:36.053225  8631 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:36.062590  8469 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:36.082710  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65824.7us
I20260709 12:25:36.030865  8603 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.117394  8638 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:36.038120  8653 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:25:36.120791  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115815us
I20260709 12:25:36.122066  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.128571  7288 leader_election.cc:400] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 3 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4.
I20260709 12:25:36.129462  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:36.086305  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74990.3us
I20260709 12:25:36.102319  8541 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:36.121524  8464 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.125730  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:36.123502  8490 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.553s	user 0.009s	sys 0.002s
I20260709 12:25:36.170215  8490 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:36.170739  8566 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:36.171236  8566 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.172219  8566 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.173641  8566 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.132650  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:36.175247  8254 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.004s
I20260709 12:25:36.127364  8499 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.179622  8529 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:36.180116  8529 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.181121  8529 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:36.184182  8617 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.586s	user 0.009s	sys 0.000s
I20260709 12:25:36.184882  8617 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:36.185447  8617 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:36.110019  8656 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.188596  8656 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.127753  8632 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:36.189675  8632 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.191891  8632 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:36.202857  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 119781us
I20260709 12:25:36.193482  6204 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:36.209935  6204 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:36.213653  6274 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.946s (SENT)
I20260709 12:25:36.123538  8663 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.123793  8649 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.136178  8409 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:36.137068  8347 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:36.141230  8603 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:36.143155  8631 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:36.143958  8557 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:36.134465  8469 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.242815  8469 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.243863  8469 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.245317  8469 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:36.245800  8469 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.145088  8583 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.146999  8436 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:36.147795  8581 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:36.247635  8581 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:36.248670  8581 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.250202  8581 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.151351  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.152300  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.256247  6450 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:36.124432  8463 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.154362  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.261392  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.156466  8620 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:36.156908  8618 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:36.274194  8618 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.274439  8632 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:36.275210  8618 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.275877  8632 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:36.276434  8632 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.276710  8618 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:36.277356  8632 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.157413  8541 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.278750  8622 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:36.279220  8541 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.230808  8569 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.283957  8662 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:36.284415  8662 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.285463  8529 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.285249  8662 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.285998  8663 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:36.287006  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 135759us
I20260709 12:25:36.287307  8271 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.287714  8469 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.288661  8338 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:36.289228  8338 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.289608  8271 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:36.227277  8556 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.290200  8338 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.289057  8469 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.292063  8499 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:36.292212  8556 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:36.293196  8662 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:36.293437  8649 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.157428  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:36.296665  6385 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.158741  6305 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:36.297118  6385 raft_consensus.cc:614] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:36.132757  8641 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:36.159469  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.159834  8650 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:36.302024  8641 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:36.305301  8641 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:36.163969  8610 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:36.167294  8368 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:25:36.169682  8588 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.568s	user 0.004s	sys 0.004s
I20260709 12:25:36.217146  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.190153  6202 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 269) took 7002 ms. Trace:
I20260709 12:25:36.215081  6204 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:36.310117  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96496.9us
W20260709 12:25:36.311252  6274 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.986s (SENT)
W20260709 12:25:36.312175  6274 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.049s (SENT)
I20260709 12:25:36.312695  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:36.234537  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.241345  8631 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:25:36.152500  8624 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:36.155036  8659 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.264735  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.316447  6457 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:36.264855  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:36.265758  8591 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.000s
I20260709 12:25:36.186363  8617 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.267881  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.327821  6458 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:36.327642  6450 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:36.267895  8498 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:25:36.273154  8572 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:36.296918  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.302285  6459 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:36.302429  6301 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.282032  8569 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:36.304731  8650 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:36.162087  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.311865  6204 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:36.335923  6204 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:36.314052  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.240155  8557 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:36.314121  6274 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.084s (SENT)
I20260709 12:25:36.319217  7335 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:36.268568  8620 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:36.331041  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.340029  6304 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.339782  8557 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.341216  8557 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:36.275893  8463 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:36.343731  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 181738us
I20260709 12:25:36.343351  6204 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:36.344678  6274 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.102s (SENT)
I20260709 12:25:36.350931  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.356323  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.358716  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:36.359006  6301 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:36.359543  7335 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:36.350670  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:36.316355  8363 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.327404  8622 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:36.326864  8624 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:36.327179  8638 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:36.370909  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:36.397701  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34bae875d7104f298673c223da6caf2e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:36.398455  6385 raft_consensus.cc:606] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:36.372933  8672 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:25:36.394842  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.395565  6304 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:36.327623  8335 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:36.395797  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:36.357576  8591 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:36.412147  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.447216  6307 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:36.356940  8588 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:36.436883  8528 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:36.355758  8648 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:36.438228  8594 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.360786  8368 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:36.440043  7331 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
W20260709 12:25:36.440078  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45329.2us
I20260709 12:25:36.386765  8593 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.309753  6202 rpcz_store.cc:276] 0709 12:25:29.188222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.188411 (+   189us) service_pool.cc:224] Handling call
0709 12:25:36.190017 (+7001606us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:30.844004 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:30.877057 (+ 33053us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:30.877112 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:25:30.877132 (+    20us) tablet.cc:662] Decoding operations
0709 12:25:30.877860 (+   728us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:30.877946 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:30.877994 (+    48us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:30.878377 (+   383us) tablet.cc:701] Row locks acquired
0709 12:25:30.878395 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:25:30.878583 (+   188us) write_op.cc:270] Start()
0709 12:25:30.878712 (+   129us) write_op.cc:276] Timestamp: P: 1783599930878519 usec, L: 0
0709 12:25:30.878739 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:25:30.879492 (+   753us) log.cc:844] Serialized 1665 byte log entry
0709 12:25:31.344618 (+465126us) spinlock_profiling.cc:243] Waited 7.02 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b
0709 12:25:33.798585 (+2453967us) op_driver.cc:464] REPLICATION: finished
0709 12:25:34.114644 (+316059us) write_op.cc:301] APPLY: starting
0709 12:25:34.114727 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:34.205085 (+ 90358us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:34.213992 (+  8907us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:34.754739 (+540747us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b5c001e568c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9499331 00007f6fa9499221 00007f6fa9493f6c 00007f6fa6bf430b 00007f6fa6bf3fd3 00007f6fa6cf2e34 00007f6fa6cde97c 00007f6fa6ce2d61 00007f6fa6ce2d19 00007f6fa6ce2ca9 00007f6fa6ce2c71
0709 12:25:34.873328 (+118589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:34.934805 (+ 61477us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:34.938455 (+  3650us) write_op.cc:312] APPLY: finished
0709 12:25:34.990440 (+ 51985us) log.cc:844] Serialized 41 byte log entry
0709 12:25:35.602025 (+611585us) spinlock_profiling.cc:243] Waited 2.13 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b
0709 12:25:35.808198 (+206173us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:35.808315 (+   117us) write_op.cc:454] Released schema lock
0709 12:25:35.808359 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:35.851215 (+ 42856us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:35.851254 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":7,"child_traces":[["op",{"apply.queue_time_us":309978,"apply.run_cpu_time_us":42486,"apply.run_wall_time_us":1716284,"num_ops":4,"prepare.queue_time_us":530584,"prepare.run_cpu_time_us":107812,"prepare.run_wall_time_us":1784612,"raft.queue_time_us":512639,"raft.run_cpu_time_us":11953,"raft.run_wall_time_us":278440,"replication_time_us":2885447,"spinlock_wait_cycles":45023232,"thread_start_us":3295193,"threads_started":5,"wal-append.queue_time_us":2584430,"wal-append.run_cpu_time_us":23318,"wal-append.run_wall_time_us":1991291}]]}
I20260709 12:25:36.374567  8667 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.093s
W20260709 12:25:36.451653  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.451629  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.453229  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:25:36.405243  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.481622  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.483539  8593 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.464216  8467 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:36.464087  8528 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:36.484565  8433 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:36.503014  6307 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:36.482111  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:36.343045  8659 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.486080  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:36.488531  6379 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:36.489337  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.506511  6385 raft_consensus.cc:606] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.507428  6385 raft_consensus.cc:614] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:36.508019  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 430341us
I20260709 12:25:36.508320  6377 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:36.508351  6380 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:36.509291  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.505328  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.509511  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.509863  6382 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.510396  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.472844  8648 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:36.511605  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.512319  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.511734  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.507032  6376 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:36.490417  8498 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:36.514569  8676 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:36.519956  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:36.520629  7331 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
W20260709 12:25:36.516057  8674 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:36.520277  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:36.522547  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71019.7us
I20260709 12:25:36.508838  8581 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:36.518996  8648 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.409421  8638 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.528824  6451 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:36.506107  8528 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.491156  8663 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:36.531719  8663 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.533901  8663 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.535624  8594 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:36.518818  8613 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:36.536324  8659 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:36.467454  8588 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.540366  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.537993  6457 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:36.518353  8657 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:36.542276  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.540280  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.550805  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:36.551654  6385 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.552070  6385 raft_consensus.cc:614] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:36.486379  8624 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.522727  8490 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.538888  8569 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:36.557796  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.560819  6376 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:25:36.566373  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.479974  8363 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.538678  8581 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:36.546344  8603 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.544955  8651 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:36.545424  8657 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.544927  8613 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:36.619406  8654 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:36.527480  8498 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.647044  8498 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.564196  8347 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:25:36.646903  8648 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:36.647790  8347 consensus_queue.cc:1243] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:36.651162  8463 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:36.651593  8463 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:36.607841  8566 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:36.623145  8581 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.624406  8335 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.621925  8528 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.627414  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:36.629341  6382 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:36.633592  8591 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.554986  8668 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.107s
I20260709 12:25:36.635893  6456 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:36.665196  6458 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:36.638317  8490 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:36.642863  8613 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:36.643684  8657 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.654950  8556 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:36.564567  8572 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.591933  8659 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.660041  7329 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:36.611519  8583 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:36.665633  8569 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.668860  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.669955  6301 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.700064  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.659193  8651 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:36.661942  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.700405  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:36.682240  6302 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:36.698175  6381 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:36.665010  8593 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.702908  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.659154  8335 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.664742  8591 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.653012  8463 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.670073  8613 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.706868  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.706372  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.706233  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.706458  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.708194  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.677027  8556 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:36.717208  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 159065us
I20260709 12:25:36.718166  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:36.727496  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.717829  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.682561  8583 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:36.722838  6274 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.596s (SENT)
I20260709 12:25:36.676744  8490 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.720096  6378 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:36.731936  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.715248  8659 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:36.717842  8653 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:36.682194  8685 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:25:36.682591  8572 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.736526  6351 leader_election.cc:400] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:25:36.736718  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:53964 (request call id 12) took 1777 ms. Trace:
I20260709 12:25:36.739563  6351 leader_election.cc:403] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:36.713673  8368 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:25:36.717684  8499 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 12:25:36.735554  8666 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:25:36.719545  8657 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:36.744236  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.745636  6377 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:36.739619  6308 rpcz_store.cc:276] 0709 12:25:34.959195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.959338 (+   143us) service_pool.cc:224] Handling call
0709 12:25:36.730598 (+1771260us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400140d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ad735 00007f6fa826dd29 00007f6fa17ec1d4 00007f6fa17ec162 00007f6fa17ec091 00007f6fa17ec00c 00007f6fa17eb2c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:36.736649 (+  6051us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3129291008}
I20260709 12:25:36.734721  8583 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:36.756044  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.737088  8463 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:36.768456  8656 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:25:36.769450  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.769554  8638 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.740607  8651 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:36.736371  8593 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:36.772282  8335 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.740036  8653 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.760073  8409 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:36.787673  8409 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:25:36.788693  8529 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.503s	user 0.010s	sys 0.001s
I20260709 12:25:36.789039  8409 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.760485  8591 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.763422  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.794099  8653 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.721669  8543 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:25:36.723865  8436 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:25:36.757118  8618 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:25:36.752650  8684 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:36.768601  8680 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:25:36.764485  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.741560  8588 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:36.739859  8557 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:36.799139  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:36.795007  8648 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:36.816217  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51821.8us
I20260709 12:25:36.845304  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.846987  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.847944  6380 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:36.857381  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.796248  8368 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:36.858352  6380 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:36.870929  6378 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:36.872206  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.872995  6378 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:36.874007  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.875415  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.876668  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.833695  8529 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.877393  6378 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:36.879010  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:36.879503  6274 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.779s (SENT)
I20260709 12:25:36.880316  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.881600  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.882905  6380 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:36.884281  6378 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:36.834214  8684 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:36.843650  8433 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:36.866318  8641 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.001s
I20260709 12:25:36.854207  8685 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:36.891512  6454 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:36.888931  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.891762  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.866145  8409 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:36.887447  6274 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.792s (SENT)
I20260709 12:25:36.893740  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.893999  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.892436  6380 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:36.895244  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:36.895208  6274 leader_election.cc:336] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.762s (SENT)
I20260709 12:25:36.797756  8569 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:36.900131  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet e44e2bcbe27f4a9192cc9acd16b966ad: Illegal state: Replica 4268023ecea149cd83cba77db53c3100 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:25:36.897348  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.776856  8556 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.888011  8588 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:36.909901  8469 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.000s
I20260709 12:25:36.870348  8368 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.928171  8368 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:36.911304  8490 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:36.917429  8695 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.099s	user 0.002s	sys 0.003s
I20260709 12:25:36.919462  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.930632  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.892880  8685 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.920789  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:36.896114  8584 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:36.896080  8641 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.944211  8593 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:36.945828  8641 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:36.946465  7331 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:36.925203  8684 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.932080  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.934212  8685 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:36.947885  6378 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:36.893905  8433 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.946650  8591 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:36.941612  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.948307  6455 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:36.949007  6450 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:36.948792  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.950103  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.951505  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:36.905351  8572 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:36.952904  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:36.953461  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.935127  8613 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:36.955665  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.958078  6451 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:36.958271  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:36.953711  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:36.981420  6451 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
W20260709 12:25:36.980823  6274 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.877s (SENT)
I20260709 12:25:36.982992  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.985141  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.986817  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.987413  6307 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:36.987573  6303 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:36.988443  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.988799  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.989071  6307 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:36.989920  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.990952  6309 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:36.991075  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:36.991297  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.992055  6309 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:36.992178  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:36.992691  6307 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:36.993613  6307 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:36.994257  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:36.995071  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.995481  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:36.995647  6305 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
W20260709 12:25:36.957536  8688 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:36.999657  6307 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:37.002664  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.003436  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.003810  6452 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:37.004531  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.004837  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:36.956235  8591 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:37.006378  6458 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.006420  6310 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:37.007541  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.011273  6308 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.000015  8588 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.015355  6382 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:37.000085  8529 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.014878  6301 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:37.010456  6379 raft_consensus.cc:2376] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2. Re-sending same reply.
I20260709 12:25:37.017408  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.018049  6379 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:37.021515  8529 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.023097  8662 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:37.023295  8433 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.023592  8662 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:37.023710  8529 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.000969  8593 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.026304  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.026648  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.027258  6379 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:25:37.006716  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.002615  8490 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:37.003111  8469 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:37.006659  8557 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:37.032817  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.036490  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.036989  6376 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:37.006812  8463 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.037497  8490 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.002882  8676 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:25:37.008498  8591 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:37.048264  8613 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:37.049001  8498 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:37.050617  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.051899  6379 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:37.052106  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.049446  8498 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:37.055549  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:37.055910  8498 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.057392  8624 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:37.058043  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.062542  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:37.077767  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.064391  6379 raft_consensus.cc:2368] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4.
I20260709 12:25:37.057720  8498 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.040712  8657 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.081347  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:37.082152  8594 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:37.083735  8656 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:37.087790  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.088560  6378 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:37.058570  8463 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:37.090070  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.090171  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.090904  6378 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:37.038734  8469 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.094333  8469 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.024605  8662 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:37.096463  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.087314  8557 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:37.098343  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.099890  6378 raft_consensus.cc:2376] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
I20260709 12:25:37.100764  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.101331  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.102936  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.103770  6378 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.107609  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.109587  8657 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:37.100556  8557 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.110663  8463 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:37.110958  8557 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.111152  8463 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:37.108134  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.106855  8594 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.113054  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.112480  8657 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:37.099141  8656 consensus_queue.cc:1243] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:37.128522  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:37.129547  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:37.129841  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:37.131006  6303 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:37.140201  6378 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:37.141273  6307 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:37.145578  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:37.147044  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:37.111827  8463 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.148686  6308 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:37.154158  8490 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.150753  8650 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.112356  8594 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:37.114609  8584 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.154418  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:37.151057  8335 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:37.106988  8583 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.157881  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.128947  8662 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.127477  8529 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:37.127812  8657 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:37.236140  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:37.228533  8584 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.166313  7335 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.148397  8617 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:25:37.210641  8433 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:37.211555  8335 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.245625  6450 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.212255  8668 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:37.165676  8594 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.141975  8610 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.215160  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.251266  6454 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 2: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:37.215410  8588 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:37.215425  6302 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.155781  8622 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:25:37.221223  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:37.221748  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.223960  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:37.223084  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:37.223709  6385 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:37.233912  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:37.240262  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.241091  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:37.239171  8657 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:37.242363  8584 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:37.243048  6304 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:25:37.111532  8591 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.148397  8603 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:25:37.248461  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:37.269098  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 122142us
I20260709 12:25:37.269982  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.270206  8651 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.270386  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:37.271693  8651 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:25:37.249559  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.242707  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:37.247284  8335 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.274519  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:37.276875  8651 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:37.278432  8651 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.289819  6307 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:37.289772  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.291507  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.247642  8668 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:37.252779  8588 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.288343  7320 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:37.303258  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:37.302372  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:37.309994  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.310925  6381 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:37.291138  8617 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:37.289620  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:37.319758  7335 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:37.319917  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:37.321780  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.322041  6307 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:37.265779  8610 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.324673  8469 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.301875  8556 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.324662  7318 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:37.303041  8588 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:37.315199  8703 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:25:37.329427  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.318311  8593 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:37.339984  6381 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:37.321903  8617 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:37.341846  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.307204  8368 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:25:37.324106  8584 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.347141  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.348726  8335 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.353232  8685 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.348619  8668 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.354947  6383 raft_consensus.cc:2368] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:37.355614  8668 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:37.356199  6305 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:37.361809  8588 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.364554  8588 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.367568  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.368496  6382 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:37.348395  6381 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:37.362738  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:37.371490  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.372054  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.372592  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.382761  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.383567  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.384773  6381 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:37.383838  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.385421  6382 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
I20260709 12:25:37.386086  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:37.386178  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.390344  6385 raft_consensus.cc:606] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.390761  6385 raft_consensus.cc:614] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:37.391892  6381 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:37.392032  6385 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:25:37.381603  8593 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:25:37.345275  8699 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.343s	user 0.000s	sys 0.007s
I20260709 12:25:37.337114  8710 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:37.394318  6381 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:37.398357  6382 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:37.408700  6382 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:37.412060  6382 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.413430  6381 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:37.328475  8572 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:25:37.368995  8584 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:37.367449  8711 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:37.390691  8588 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:37.424901  8588 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.426919  8588 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.430298  8588 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:37.430696  8528 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:37.434556  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:37.438094  8708 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:25:37.438720  8638 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:37.439270  8638 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:37.440232  8638 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.441303  8685 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.441556  8638 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.423473  8584 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.415509  8710 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:37.442811  8710 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:37.445673  8335 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.446502  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:37.447441  8610 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.448589  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:37.448918  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:37.449517  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:37.449828  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:37.438200  8583 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:37.459901  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.462929  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:37.465922  8543 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:37.463191  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.464046  8582 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:37.466810  6380 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:37.465205  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.468417  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:37.469272  6382 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:37.469360  8665 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:37.466598  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet f53146508e564f8aae279a61e6696e80: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:25:37.472740  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:37.475584  6382 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:37.477095  6382 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:37.478120  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.480674  6384 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:37.481455  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.425559  8711 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:37.455071  8552 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:37.514364  8552 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:37.514902  8552 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.492455  6310 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:37.523193  8676 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:37.523679  8676 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:37.524621  8676 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.525707  8676 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.527264  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.528764  6450 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:37.530103  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.531489  8711 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.532298  8591 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:37.536510  6457 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:37.537746  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.538419  6450 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:37.539750  8650 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:37.541342  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.499868  8653 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.000s
I20260709 12:25:37.470731  8368 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:25:37.540416  8433 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.551259  8368 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:37.551712  8368 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:37.552467  8707 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:37.552909  8707 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:37.552628  8368 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.554119  8368 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.506932  6380 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:37.555344  8653 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.541839  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:37.543176  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:37.558089  6451 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.544029  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:37.546886  6459 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:37.559614  8684 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:37.546921  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:37.553774  8707 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.547979  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.476336  8582 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:37.557510  6455 raft_consensus.cc:2368] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:37.566324  8660 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:37.560142  8684 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:37.433712  8594 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.574518  6451 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
W20260709 12:25:37.569783  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81783.4us
I20260709 12:25:37.575629  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:37.578871  6450 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.570050  6204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 262) took 7727 ms. Trace:
I20260709 12:25:37.551208  8433 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.582957  7319 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:25:37.585104  8624 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.000s
I20260709 12:25:37.585892  6300 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:25:37.582324  8684 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.586978  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 242) took 1428 ms. Trace:
I20260709 12:25:37.587309  6300 rpcz_store.cc:276] 0709 12:25:36.158295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.158565 (+   270us) service_pool.cc:224] Handling call
0709 12:25:36.158833 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.586241 (+1427408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.586791 (+   550us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000a9e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:37.586933 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3281288448}
I20260709 12:25:37.583055  8433 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.592106  6457 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.500958  7332 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:37.597293  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:37.599210  6457 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.601835  6450 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:37.603615  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.632644  6455 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:37.603781  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.557556  8653 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:37.612057  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 290371us
I20260709 12:25:37.647173  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.608664  6201 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 6 to 8, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 8 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:37.580119  6204 rpcz_store.cc:276] 0709 12:25:29.847070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.855262 (+  8192us) service_pool.cc:224] Handling call
0709 12:25:37.569946 (+7714684us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:36.715946 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:36.792647 (+ 76701us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:36.792700 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:25:36.792727 (+    27us) tablet.cc:662] Decoding operations
0709 12:25:36.793741 (+  1014us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:36.793829 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:36.793870 (+    41us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:25:36.813280 (+ 19410us) tablet.cc:701] Row locks acquired
0709 12:25:36.815064 (+  1784us) write_op.cc:260] PREPARE: finished
0709 12:25:36.854290 (+ 39226us) write_op.cc:270] Start()
0709 12:25:36.867080 (+ 12790us) write_op.cc:276] Timestamp: P: 1783599936854222 usec, L: 0
0709 12:25:36.870187 (+  3107us) op_driver.cc:348] REPLICATION: starting
0709 12:25:36.929165 (+ 58978us) log.cc:844] Serialized 2423 byte log entry
0709 12:25:37.089545 (+160380us) op_driver.cc:464] REPLICATION: finished
0709 12:25:37.381555 (+292010us) write_op.cc:301] APPLY: starting
0709 12:25:37.396128 (+ 14573us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:37.408371 (+ 12243us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:37.471567 (+ 63196us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:37.507534 (+ 35967us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:37.507747 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:25:37.507788 (+    41us) write_op.cc:312] APPLY: finished
0709 12:25:37.508028 (+   240us) log.cc:844] Serialized 53 byte log entry
0709 12:25:37.508198 (+   170us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:37.508324 (+   126us) write_op.cc:454] Released schema lock
0709 12:25:37.508375 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:37.508776 (+   401us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:37.508804 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":218074,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":245398,"apply.run_cpu_time_us":3456,"apply.run_wall_time_us":164059,"num_ops":6,"prepare.queue_time_us":319041,"prepare.run_cpu_time_us":5825,"prepare.run_wall_time_us":234928,"replication_time_us":211340,"thread_start_us":349353,"threads_started":2}]]}
I20260709 12:25:37.622519  8436 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:37.622710  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.623420  8583 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:37.624316  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:37.652379  6301 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:25:37.589867  8569 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:25:37.626106  8338 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.626696  6381 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:37.565989  8582 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.426030  8617 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.636452  8498 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.630304  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.696684  6452 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:37.631374  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:37.697849  6385 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:37.698062  6451 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:37.621053  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:37.648847  6201 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 5 to 7, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 7 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.648885  6454 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:37.598098  8624 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:25:37.663493  6376 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:37.663959  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:37.703895  6378 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:37.661682  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.619380  8707 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:37.636686  8433 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:37.651409  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.604421  8710 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.707489  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:37.722584  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:37.707779  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 86779.9us
I20260709 12:25:37.723737  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:37.708475  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46851.6us
W20260709 12:25:37.674764  8557 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.563s	user 0.009s	sys 0.002s
W20260709 12:25:37.717471  8463 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.554s	user 0.005s	sys 0.004s
I20260709 12:25:37.726231  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.717717  8584 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.645545  8588 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:25:37.649791  8556 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:25:37.707168  8593 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:37.696205  8363 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.711409  8683 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:25:37.735759  8617 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:37.743237  6272 leader_election.cc:400] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:37.736171  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:37.710526  6201 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 2 to -1, term changed from 2 to 4, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) evicted, now has a pending config: VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65), VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66), VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67), NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:37.751325  6272 leader_election.cc:403] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:37.721175  8433 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:37.755508  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.710294  8671 raft_consensus.cc:2764] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:37.747771  6301 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:37.695384  8338 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.734817  8335 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.660552  8583 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.661738  8714 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
W20260709 12:25:37.750754  8662 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.000s
I20260709 12:25:37.729478  8648 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.720665  8622 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:37.764631  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:37.737807  8347 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:25:37.757562  8433 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:37.749012  8593 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:37.792176  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 231902us
I20260709 12:25:37.735229  8569 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:25:37.752218  8613 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:37.765599  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:37.800474  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:37.736472  8557 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.825991  6307 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:37.794188  8610 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.740873  8463 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.746961  8436 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:25:37.744596  8657 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:37.821838  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96504.6us
I20260709 12:25:37.793779  8594 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.800544  8335 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:37.754135  8363 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.819309  8593 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:37.883889  8552 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.886041  8552 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:37.860153  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:37.865991  8718 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:25:37.866115  8463 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:37.866712  8715 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:25:37.867228  8717 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:25:37.871575  8557 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:37.874544  8584 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.874977  8498 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:37.877645  8684 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:37.882762  8671 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:37.861316  8622 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.851992  8610 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.862317  8648 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:25:37.931638  8710 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.921753  8684 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:37.789328  6201 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 4 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 6 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:37.776137  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:38.008330  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 264977us
I20260709 12:25:37.803489  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.011000  6304 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:37.754257  6274 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5.
W20260709 12:25:38.016166  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 263340us
I20260709 12:25:37.855021  6379 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:38.029667  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:38.024423  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:38.039386  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:38.040058  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:38.034405  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:38.041844  6307 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:38.042191  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 142622us
I20260709 12:25:38.035136  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:38.049003  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:38.036165  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:38.033600  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:38.051103  6304 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:38.052686  6308 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:38.040768  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:38.055433  6378 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:38.057773  6274 leader_election.cc:400] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:38.058805  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:38.064563  6304 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:38.071130  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:38.072176  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 2e4301d25f8649e496bcbcb9f954c74e: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:25:37.890877  7317 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:37.889081  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:38.087915  8338 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.091120  8463 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.092204  8463 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:37.923787  8622 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.096226  8557 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.096520  8622 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.097393  8557 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.096522  8528 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.098855  8588 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:38.099232  8622 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:25:38.099510  8588 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:38.100576  8528 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:25:38.138023  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.002s	sys 0.014s
W20260709 12:25:38.138624  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.003s	sys 0.014s
I20260709 12:25:38.097498  8368 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
W20260709 12:25:38.139045  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 504352us
I20260709 12:25:37.929651  8657 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:37.944999  8684 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.140650  8588 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.144966  8363 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:38.193611  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:38.211549  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 154191us
I20260709 12:25:38.183967  6385 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:38.144033  8463 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.283773  6309 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:38.187703  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.214012  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:38.208266  8528 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.203375  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:38.188957  8610 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.199462  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.200304  8467 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 2 replicates
I20260709 12:25:38.184106  8529 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:38.184583  8584 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:38.144627  8622 consensus_queue.cc:1243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 12:25:38.299925  6300 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.019s	sys 0.031s
I20260709 12:25:38.197633  8732 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:38.150550  8719 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:25:38.196878  8498 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.198122  8543 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:38.186259  8707 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:38.319854  6300 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.020s	sys 0.032s
I20260709 12:25:38.201547  8653 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:38.178599  8591 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:38.181934  8583 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.309057  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.311265  8528 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.313354  6451 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:38.191536  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:38.210115  8665 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.200338  8684 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.308236  6307 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:25:38.317962  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 104138us
I20260709 12:25:38.333014  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:38.333287  6380 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:38.335511  6380 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:38.337612  8711 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:38.338120  8711 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:38.196684  8613 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:38.329993  8594 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:38.309571  8584 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.185570  8714 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 12:25:38.210572  8648 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.109s
I20260709 12:25:38.328545  8638 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:38.329198  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 164243us
I20260709 12:25:38.314524  8732 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.182871  8710 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.140s
I20260709 12:25:38.338438  8528 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.326525  8498 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:38.339735  8617 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.582s	user 0.003s	sys 0.014s
I20260709 12:25:38.341885  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.342067  8543 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.343978  8737 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:38.344839  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.322373  8653 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:38.358888  8594 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:38.346647  8463 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.348588  8684 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.189256  8552 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.138s
I20260709 12:25:38.187849  8713 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:38.363507  8732 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.178893  8557 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.363363  6381 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:38.363890  7331 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.360414  8584 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.391029  6304 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:38.391736  8653 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.393641  8739 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.362246  8638 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.394394  8739 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.395273  8638 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.397315  8739 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.360541  8591 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:38.398439  8557 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.399739  8591 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.400406  8707 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.403235  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.404181  6380 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:38.405225  8467 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:25:38.406360  7331 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:38.406739  8467 ts_tablet_manager.cc:1403] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 2.020s	user 0.019s	sys 0.037s
W20260709 12:25:38.405453  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.387982  8498 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:38.410511  8593 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.405055  8584 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.376605  8737 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:38.440449  8737 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:38.441777  8543 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.444396  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:38.446677  8650 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:38.447278  8650 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:38.426700  8529 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:38.427353  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:38.432727  8720 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.568s	user 0.003s	sys 0.000s
W20260709 12:25:38.434417  8721 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.564s	user 0.001s	sys 0.000s
I20260709 12:25:38.435956  8463 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.378461  8594 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:38.439594  8593 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.393675  8653 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:38.473034  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47386.9us
W20260709 12:25:38.494803  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115202us
I20260709 12:25:38.403285  8707 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.510751  6305 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.511521  6309 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:38.511798  8433 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.684s	user 0.004s	sys 0.017s
I20260709 12:25:38.512594  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.512396  8433 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.513402  6309 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:38.514452  8726 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.622s	user 0.000s	sys 0.003s
I20260709 12:25:38.515693  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.472332  8738 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.085s
I20260709 12:25:38.517690  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:38.518857  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2023) took 1606 ms (client timeout 2058 ms). Trace:
W20260709 12:25:38.519344  6379 rpcz_store.cc:269] 0709 12:25:36.911956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.087499 (+175543us) service_pool.cc:224] Handling call
0709 12:25:38.518521 (+1431022us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:38.518657 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1686869248}
W20260709 12:25:38.521219  7317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 515) took 2125 ms (client timeout 1553 ms). Trace:
W20260709 12:25:38.521626  7317 rpcz_store.cc:269] 0709 12:25:36.395263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.395463 (+   200us) service_pool.cc:224] Handling call
0709 12:25:38.520880 (+2125417us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b540043a3d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:38.521034 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3298871680}
I20260709 12:25:38.523001  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:38.523306  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.493424  8584 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.510800  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.539148  6351 leader_election.cc:400] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:38.539815  6351 leader_election.cc:403] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.541250  6451 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:38.543818  6376 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:38.543251  8529 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.544853  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2016) took 1633 ms. Trace:
I20260709 12:25:38.546628  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.545320  6377 rpcz_store.cc:276] 0709 12:25:36.910828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.389816 (+478988us) service_pool.cc:224] Handling call
0709 12:25:37.396443 (+  6627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.406721 (+ 10278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.406752 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.492431 (+ 85679us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:25:37.492901 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.709286 (+216385us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:38.031351 (+322065us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.031897 (+   546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.032131 (+   234us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.431849 (+399718us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:38.544424 (+112575us) raft_consensus.cc:1685] finished
0709 12:25:38.544445 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.544583 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36557,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":63897,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":11585,"spinlock_wait_cycles":45212160,"thread_start_us":506235,"threads_started":2,"wal-append.queue_time_us":307208}
I20260709 12:25:38.547420  6455 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:38.529006  8719 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.504253  8583 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:38.551671  6382 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:38.555464  6309 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:25:38.537127  8498 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.420583  8665 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.558385  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:38.558913  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:38.564375  6305 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:38.448534  8650 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.564707  7331 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:38.564193  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.566632  6304 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:38.569191  8668 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:38.569808  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.570209  8541 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:38.570684  6456 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:38.570945  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:38.571224  8541 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.569235  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.573280  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.574090  8541 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:38.575412  8556 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:38.577147  8671 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:38.577566  8651 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.567516  8498 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.584007  6450 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:38.578122  8583 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.559206  8593 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.557838  8684 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.559530  8529 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:38.620831  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62040.3us
W20260709 12:25:38.597504  7332 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.008s
I20260709 12:25:38.598709  8719 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.600013  8671 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:38.600869  8725 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.010s	sys 0.000s
I20260709 12:25:38.621918  7317 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:38.602612  8335 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.603492  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.058s (SENT) [suppressed 36 similar messages]
I20260709 12:25:38.603709  8733 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:38.604915  8650 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:38.606045  8749 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:25:38.606467  8662 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.638455  8662 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.639701  8719 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.608985  8463 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.640858  8662 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.643023  8583 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.612020  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.652143  6450 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.615594  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:38.615758  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.655009  7320 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:38.655284  6379 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:38.655723  8648 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.656515  8648 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.657042  6455 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:38.618676  8651 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.660221  8651 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:38.618019  8744 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:38.625174  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:38.633159  8725 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.011s	sys 0.000s
I20260709 12:25:38.589483  8433 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.631706  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.662865  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:38.632375  8529 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:38.664379  8529 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:38.665869  8529 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:38.666096  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50915.9us
W20260709 12:25:38.666162  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49569.6us
I20260709 12:25:38.622974  7332 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:38.614221  8498 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.636770  6302 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:38.666697  8738 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:25:38.676306  8738 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.676883  8738 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.677893  8541 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:38.635737  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:38.678342  8541 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:38.678962  8541 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:38.679014  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57522us
I20260709 12:25:38.616534  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.681334  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.681953  6376 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:38.682888  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.683198  8657 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:38.683939  8657 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:38.684569  8338 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:38.685067  8657 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.685848  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.686527  6305 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
W20260709 12:25:38.686975  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50538.7us
I20260709 12:25:38.687734  8648 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:38.688490  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:38.656698  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 200) took 1027 ms. Trace:
I20260709 12:25:38.689679  8648 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:38.633198  8593 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.690471  8648 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:38.691818  8648 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.673815  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:38.677088  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1498) took 1446 ms (client timeout 1609 ms). Trace:
I20260709 12:25:38.646162  8733 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.619445  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:38.648703  8467 raft_consensus.cc:359] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 NON_PARTICIPANT]: Replica starting. Triggering 2 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:38.697178  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 101293us
W20260709 12:25:38.693536  6308 rpcz_store.cc:269] 0709 12:25:37.230784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.233685 (+  2901us) service_pool.cc:224] Handling call
0709 12:25:38.676820 (+1443135us) spinlock_profiling.cc:243] Waited 752 ms on lock 0x7b540010fc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:38.676990 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1730624768}
I20260709 12:25:38.692806  8593 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.720777  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:38.686200  8338 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:25:38.660475  8638 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.745327  8624 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:38.746182  8624 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:38.744513  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70839.5us
I20260709 12:25:38.714432  8335 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.689898  6458 rpcz_store.cc:276] 0709 12:25:37.629257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.631147 (+  1890us) service_pool.cc:224] Handling call
0709 12:25:37.650422 (+ 19275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.814188 (+163766us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:37.851903 (+ 37715us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.437664 (+585761us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b18002f9860. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:25:38.582620 (+144956us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:38.595771 (+ 13151us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:38.634650 (+ 38879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.656153 (+ 21503us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54001e19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:38.656200 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.656448 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2138,"prepare.queue_time_us":37537,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":13582,"raft.queue_time_us":11049,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":384270976,"thread_start_us":113295,"threads_started":1,"wal-append.queue_time_us":792547}
I20260709 12:25:38.742615  8662 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.774441  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 155283us
I20260709 12:25:38.669646  8653 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 12:25:38.777076  8463 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.788395  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 184914us
I20260709 12:25:38.777420  8593 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:38.779685  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:38.785756  8668 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.777029  8725 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:25:38.796875  8593 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.796202  8732 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.811789  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.813104  8733 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.813577  8732 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.797473  8738 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:38.803721  7318 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.820879  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.824805  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.824887  8743 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:25:38.825732  6379 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:38.825917  8638 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.826048  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.822253  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.835052  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.836444  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.837061  6308 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:38.815125  8529 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.815600  8593 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:38.795147  8552 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:38.833215  6305 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:25:38.835685  6454 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 2.
I20260709 12:25:38.798851  8662 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.844827  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.846098  8676 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:38.845927  8737 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.846462  8752 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:38.821630  8467 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:25:38.814428  8713 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:38.846830  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.806419  8463 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.855490  8335 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.856457  8665 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:38.857797  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.850226  6450 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:38.859671  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:38.863415  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:38.834245  8556 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
W20260709 12:25:38.884600  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64814.5us
I20260709 12:25:38.883698  8657 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:38.900803  8467 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:38.886770  8676 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.922444  6348 leader_election.cc:400] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:38.922839  8676 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.925299  8676 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.889963  8764 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.888980  8529 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.926796  8764 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.928144  8529 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.929147  8764 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:38.921909  8657 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.891067  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.891934  6383 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:38.892794  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:38.886013  8742 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:25:38.896155  8752 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:38.902940  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:38.903362  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.909587  6451 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:38.909646  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:38.908599  8463 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:38.899533  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:38.914058  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:38.928198  8594 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:38.901362  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.122807  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:39.137595  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 238155us
I20260709 12:25:38.996205  6379 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:39.137346  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.999454  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:39.139411  8363 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:38.915603  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:38.901733  8713 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:38.995887  6453 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:25:38.862581  8668 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.136663  7335 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:38.858570  8725 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:39.010828  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:38.907594  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.139635  6305 raft_consensus.cc:2368] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5.
I20260709 12:25:39.139122  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.139854  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.144654  6381 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:39.144979  6301 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:39.155723  6379 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:39.119361  8617 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.139509  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:38.911983  8735 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:38.931813  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:38.996348  8760 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:25:39.120260  8610 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.160010  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 249635us
I20260709 12:25:38.931416  8736 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.002719  8662 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.174769  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:39.174307  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.140829  8529 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.175686  6301 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:39.179337  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.151933  8684 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:38.911199  8624 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.141332  8752 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:38.911000  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.180150  6384 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:38.931943  8657 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.177253  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:39.184648  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.185076  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.080576  7319 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:39.187749  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.180220  8467 ts_tablet_manager.cc:1434] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.770s	user 0.007s	sys 0.007s
I20260709 12:25:39.162312  8735 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:39.179796  7320 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:39.140904  8541 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.186117  8684 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.197860  6377 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:39.184139  8588 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.207348  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.215709  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:39.177825  8764 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:39.194864  6305 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:39.218750  8684 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.207404  8662 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.217751  8736 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.199364  8363 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.218986  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:39.219758  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.219887  8735 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.222950  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.189396  8765 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.179661  8617 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.195721  8738 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.231258  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:39.232085  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.234228  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.234635  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.235041  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.235562  8498 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.236861  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:39.206730  8650 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.563s	user 0.010s	sys 0.004s
W20260709 12:25:39.243181  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 340726us
I20260709 12:25:39.244491  6307 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:39.246469  6382 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:25:39.311538  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.255149  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.246383  6454 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:39.240186  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:39.239063  6384 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:39.273098  6453 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:39.229471  8610 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.275315  6379 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:25:39.297688  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66491.4us
I20260709 12:25:39.246014  8363 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.296661  6459 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
W20260709 12:25:39.213233  8467 ts_tablet_manager.cc:1467] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Tablet startup took 792ms
W20260709 12:25:39.304932  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55948.9us
I20260709 12:25:39.259039  8463 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:39.272010  6308 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:39.177875  8725 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.304003  8738 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.240963  8648 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:39.318035  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 307305us
W20260709 12:25:39.318842  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 78761.1us
I20260709 12:25:39.247408  8541 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.237816  8368 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:39.199877  8529 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.245559  8594 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.258003  8668 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.245700  8736 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.313117  8638 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:39.238910  8588 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.370582  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.344492  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:39.303579  8765 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.243079  8662 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.321209  8363 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.349562  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.240669  8764 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.258553  8684 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.350811  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:39.356083  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.348997  8368 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:39.359266  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.329540  8552 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.318046  8556 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:39.256603  8593 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.368417  6376 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:39.307341  8735 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.338188  8738 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.378746  6381 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:39.341136  8676 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.362202  8638 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.328136  8498 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.387847  8338 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.364900  8648 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:39.395578  8552 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:39.397401  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46563us
I20260709 12:25:39.411801  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:39.412755  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.384007  8668 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.400713  6382 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:39.383841  8713 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.357846  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:39.376448  8650 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:25:39.381237  8541 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.391523  8737 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:39.419849  8767 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:25:39.420468  8662 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.403779  8638 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:39.422857  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:39.427055  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69308us
I20260709 12:25:39.402343  8774 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:39.414656  8552 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:39.424355  8582 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.434s	user 0.026s	sys 0.016s
I20260709 12:25:39.411648  8338 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.485935  8774 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:39.485743  8648 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.486838  8774 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.487212  8774 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.487571  8774 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.490643  8572 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.491190  8572 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.491562  8572 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.494510  8713 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.495548  8713 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.410542  8676 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:39.465548  8582 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.592s	user 0.027s	sys 0.016s
W20260709 12:25:39.537704  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 142159us
W20260709 12:25:39.542928  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 120223us
I20260709 12:25:39.543869  6451 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:39.547432  7288 leader_election.cc:400] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5.
I20260709 12:25:39.547531  6456 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:25:39.545475  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 153117us
I20260709 12:25:39.427235  8556 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.551301  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.547767  7288 leader_election.cc:403] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:25:39.530557  8463 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:39.539834  8557 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:39.482641  8668 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.559945  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.483781  8541 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.564692  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:39.571497  8778 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:25:39.563628  8594 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:39.576335  8725 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.576735  8594 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.577291  8764 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.577831  8610 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.578128  8676 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.578752  8593 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.580271  8338 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.580521  8463 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.584122  8541 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.583702  8711 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:39.587767  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:39.587908  8711 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:39.322923  8467 ts_tablet_manager.cc:1469] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Trace:
0709 12:25:29.812811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.824006 (+ 11195us) service_pool.cc:224] Handling call
0709 12:25:30.772737 (+948731us) ts_tablet_manager.cc:934] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
0709 12:25:34.517880 (+3745143us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:34.941625 (+423745us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:34.946569 (+  4944us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:25:35.290349 (+343780us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:35.338074 (+ 47725us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:35.394543 (+ 56469us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:35.394622 (+    79us) inbound_call.cc:177] Queueing success response
0709 12:25:35.580415 (+185793us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18003d3ba0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa005eed8 00007f6fa01b9de2 00007f6fa821ed23 00007f6fa821e55d 00007f6fa82ba3d8 00007f6fa82ebf4e 00007f6fa82d0fdd
0709 12:25:36.368146 (+787731us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:25:38.407312 (+2039166us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:25:38.598366 (+191054us) tablet_replica.cc:261] Starting instrumentation
0709 12:25:38.599292 (+   926us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":51653,"mutex_wait_us":47611,"raft.queue_time_us":3011,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"spinlock_wait_cycles":25443072,"tablet-copy.queue_time_us":672248,"tablet-download-pool.queue_time_us":19403,"tablet-download-pool.run_cpu_time_us":1648,"tablet-download-pool.run_wall_time_us":571676,"thread_start_us":743809,"threads_started":2}
I20260709 12:25:39.592013  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.599604  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.602213  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.603276  6300 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:25:39.603667  8781 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:25:39.602540  7318 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.775s	user 0.008s	sys 0.002s
I20260709 12:25:39.604620  8684 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.604980  7318 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:39.608438  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.612758  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.614298  8792 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5.
I20260709 12:25:39.582278  8338 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:25:39.620682  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:39.595367  6305 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 2.
I20260709 12:25:39.497933  8368 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.641959  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.650683  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.636361  8648 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 12:25:39.652563  8780 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:25:39.655248  8594 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.656980  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1522) took 1148 ms. Trace:
I20260709 12:25:39.684752  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:39.618479  8738 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.658762  8583 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.619450  8785 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:39.665525  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.663772  8774 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.666906  8569 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:39.643431  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:39.670831  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.670652  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:39.671638  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.697696  6305 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:39.584936  8752 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.675913  8648 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:39.676550  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.676956  6308 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:39.623471  8651 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.679104  8668 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.679917  8713 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.687995  8498 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.688468  8764 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.715703  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.631207  8541 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 12:25:39.683085  6302 rpcz_store.cc:276] 0709 12:25:38.508248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.512171 (+  3923us) service_pool.cc:224] Handling call
0709 12:25:39.656783 (+1144612us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54002df5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:39.656911 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283812736}
I20260709 12:25:39.618629  8782 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:39.625730  8529 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.628728  8556 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.714411  8594 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:39.735071  8594 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.735594  8594 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:39.710844  8338 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:39.684316  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1616) took 1381 ms. Trace:
I20260709 12:25:39.703255  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.709236  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.717571  6456 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:39.717151  8764 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.695747  8569 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:39.732379  8774 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.706979  8648 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:39.624347  8789 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:39.747251  8338 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.772802  8338 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.773712  8363 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.743255  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:39.744330  8723 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.880s	user 0.003s	sys 0.003s
I20260709 12:25:39.748916  6458 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:39.750173  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.710683  8651 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.741210  8638 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:39.775682  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 104974us
I20260709 12:25:39.784641  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:39.776094  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132783us
I20260709 12:25:39.739207  8588 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.780296  6453 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:39.781911  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.702668  8583 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:39.785724  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:39.747362  6385 rpcz_store.cc:276] 0709 12:25:38.303599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.928312 (+624713us) service_pool.cc:224] Handling call
0709 12:25:38.928696 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.929121 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.929152 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.929785 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.177552 (+247767us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:39.177598 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.177681 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.684087 (+506406us) raft_consensus.cc:1685] finished
0709 12:25:39.684111 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.684248 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":701,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":242924,"threads_started":1,"wal-append.queue_time_us":441659}
I20260709 12:25:39.770844  8789 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:39.789992  8789 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:39.775381  8368 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.793192  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:39.793921  6382 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.798030  8569 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.798746  8603 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:39.799144  8603 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.799505  8603 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:39.799661  8569 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.800002  8638 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.800707  8528 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.804688  8764 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.806015  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 256) took 2332 ms. Trace:
I20260709 12:25:39.807837  8572 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.806388  6303 rpcz_store.cc:276] 0709 12:25:37.473840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.474325 (+   485us) service_pool.cc:224] Handling call
0709 12:25:37.474676 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:37.476678 (+  2002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.476709 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:37.486926 (+ 10217us) log.cc:844] Serialized 26 byte log entry
0709 12:25:37.798716 (+311790us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:38.009224 (+210508us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.009808 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.009974 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.743501 (+1733527us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:39.805477 (+ 61976us) raft_consensus.cc:1685] finished
0709 12:25:39.805613 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.805785 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1611,"prepare.queue_time_us":355988,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":94540,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":20540,"spinlock_wait_cycles":54933248,"thread_start_us":242797,"threads_started":1,"wal-append.queue_time_us":276083}
I20260709 12:25:39.791996  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.777335  8752 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.814851  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.820724  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.821925  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:39.820292  8791 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:39.824065  8791 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.824453  8791 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:39.824503  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:39.825520  6303 raft_consensus.cc:2368] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:39.826328  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.821511  8556 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:25:39.829798  8785 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.842962  8785 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.844753  8583 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.830416  8794 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:39.846004  8794 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:39.846509  8794 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:39.814126  8648 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.847637  8648 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:39.830911  8651 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.845534  8785 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.816046  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.851393  8338 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:39.852102  7314 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:39.853780  8338 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:39.857712  8671 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:39.858608  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.858952  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.859270  6302 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:39.859773  7314 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:39.862576  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.862948  8738 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.863484  7318 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:39.864459  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.864840  8794 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.865677  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:39.866212  6378 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5. Re-sending same reply.
I20260709 12:25:39.866580  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1718) took 1377 ms. Trace:
I20260709 12:25:39.867035  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.850755  8552 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:39.863323  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.913669  6378 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:39.866947  6452 rpcz_store.cc:276] 0709 12:25:38.445751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.467112 (+ 21361us) service_pool.cc:224] Handling call
0709 12:25:38.510463 (+ 43351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.633598 (+123135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.633635 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.642356 (+  8721us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.771721 (+129365us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:38.771770 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.808503 (+1036733us) spinlock_profiling.cc:243] Waited 83.9 ms on lock 0x7b54001635d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:39.808581 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.808653 (+    72us) raft_consensus.cc:1685] finished
0709 12:25:39.808675 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.822796 (+ 14121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34335,"raft.queue_time_us":56319,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"spinlock_wait_cycles":192969472,"thread_start_us":128832,"threads_started":1,"wal-append.queue_time_us":178363,"wal-append.run_cpu_time_us":2185,"wal-append.run_wall_time_us":1004377}
I20260709 12:25:39.833461  6382 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:39.835973  8617 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.874037  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.876668  8794 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.877676  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.918979  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.881207  6458 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: []
W20260709 12:25:39.881990  8775 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.327s	user 0.000s	sys 0.002s
I20260709 12:25:39.882916  8572 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:39.885370  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.927032  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.886745  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 527) took 1697 ms. Trace:
I20260709 12:25:39.928010  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2052) took 1026 ms. Trace:
I20260709 12:25:39.886986  6303 raft_consensus.cc:2410] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:39.833616  8765 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:39.889727  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:39.931504  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:39.890682  8363 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:39.889724  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.893182  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.907147  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.905753  8556 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:39.941252  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:39.830389  8528 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:39.910605  6300 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:39.912518  6351 leader_election.cc:400] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:39.910308  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2042) took 1099 ms. Trace:
I20260709 12:25:39.917786  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:39.919097  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.919173  6308 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:39.920960  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 366) took 1043 ms. Trace:
I20260709 12:25:39.949414  6450 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:39.919279  6305 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
W20260709 12:25:39.925552  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 463386us
I20260709 12:25:39.930591  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:39.921934  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:39.934432  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:39.934376  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:39.932128  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:39.917356  8529 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:39.951617  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:39.949556  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:39.988091  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:39.988063  6456 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:39.928517  6384 rpcz_store.cc:276] 0709 12:25:38.901288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.338342 (+437054us) service_pool.cc:224] Handling call
0709 12:25:39.359711 (+ 21369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.387340 (+ 27629us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.387368 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.401588 (+ 14220us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.573688 (+172100us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:39.924713 (+351025us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.924957 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.925012 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.925064 (+    52us) raft_consensus.cc:1685] finished
0709 12:25:39.925080 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.927945 (+  2865us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25824,"raft.queue_time_us":114738,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":44535,"spinlock_wait_cycles":1473280,"thread_start_us":470012,"threads_started":2,"wal-append.queue_time_us":263043}
W20260709 12:25:39.986680  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54664.6us
I20260709 12:25:39.948962  6351 leader_election.cc:403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
W20260709 12:25:39.968271  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.935096  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:40.004184  8463 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:39.925449  8557 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:40.013614  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101233us
I20260709 12:25:39.949023  6376 rpcz_store.cc:276] 0709 12:25:38.811357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.811488 (+   131us) service_pool.cc:224] Handling call
0709 12:25:39.891203 (+1079715us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b54003e7e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:39.910227 (+ 19024us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":278367104}
I20260709 12:25:39.993432  7317 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:40.015491  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:40.016309  6376 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:40.020629  6378 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:39.951565  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:39.937682  8498 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:39.951143  7332 rpcz_store.cc:276] 0709 12:25:38.862606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.873677 (+ 11071us) service_pool.cc:224] Handling call
0709 12:25:38.892339 (+ 18662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.383716 (+491377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.383776 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.383842 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:39.395698 (+ 11856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.866910 (+471212us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b54003bc4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:39.905528 (+ 38618us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.905737 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1097981824}
I20260709 12:25:40.023669  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.945712  8794 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:39.944855  8735 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.944607  8617 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.951866  8765 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:40.027349  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:39.998912  8793 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:40.029515  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:40.030284  6301 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:40.021025  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:39.998437  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:40.032562  8588 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:40.032615  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81152.7us
I20260709 12:25:40.033582  8796 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.933946  8789 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:40.034601  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.994664  8738 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:39.995121  8363 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:40.039525  6378 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:39.938781  8594 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.040427  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:39.938624  8603 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.017839  8556 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.042344  8668 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.042281  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:39.928236  7319 rpcz_store.cc:276] 0709 12:25:38.189282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.190783 (+  1501us) service_pool.cc:224] Handling call
0709 12:25:39.869641 (+1678858us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b5400438ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:39.886623 (+ 16982us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390178048}
I20260709 12:25:40.026718  8794 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:40.067778  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.082561  7329 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:40.068735  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:40.035332  8765 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.088948  6450 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:40.018407  8529 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.089880  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:40.083784  6301 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:25:40.085696  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2045) took 1272 ms (client timeout 1559 ms). Trace:
W20260709 12:25:40.090873  6383 rpcz_store.cc:269] 0709 12:25:38.813520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.813629 (+   109us) service_pool.cc:224] Handling call
0709 12:25:40.085510 (+1271881us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b5400064d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:40.085637 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136468864}
I20260709 12:25:40.088425  6376 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:40.023229  8785 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:25:40.094519  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:40.095106  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:40.050985  8498 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:40.096529  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1619) took 1172 ms. Trace:
I20260709 12:25:40.099212  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:40.039181  8735 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.100461  6384 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 3: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:40.096925  6379 rpcz_store.cc:276] 0709 12:25:38.924492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.397189 (+472697us) service_pool.cc:224] Handling call
0709 12:25:39.406038 (+  8849us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.636233 (+230195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.636281 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.650983 (+ 14702us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.081094 (+430111us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:40.081131 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.081222 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.096319 (+ 15097us) raft_consensus.cc:1685] finished
0709 12:25:40.096339 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.096467 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40658,"raft.queue_time_us":141464,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":10472,"thread_start_us":596481,"threads_started":2,"wal-append.queue_time_us":402261}
I20260709 12:25:40.068076  8463 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3.
I20260709 12:25:40.100453  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:40.102008  6459 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:40.073308  8711 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.055691  7314 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:40.039379  8617 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:40.072618  8668 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.105438  6378 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:40.081395  8657 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:40.087466  8794 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:40.095872  8765 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:40.118021  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:40.110692  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:40.134189  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1626) took 1066 ms. Trace:
I20260709 12:25:40.149324  6457 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:40.093968  8738 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:40.086125  8588 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:40.139159  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:40.139560  8799 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:25:40.107872  8591 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:25:40.109217  8650 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:40.150239  6378 rpcz_store.cc:276] 0709 12:25:39.064750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.116592 (+1051842us) service_pool.cc:224] Handling call
0709 12:25:40.130992 (+ 14400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.131185 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.131215 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.131263 (+    48us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:40.131295 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.131350 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.131447 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.117167  8734 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:40.163513  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:40.160771  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 247) took 1751 ms. Trace:
I20260709 12:25:40.153682  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:40.078785  8796 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.129851  8668 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.168329  6301 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7.
I20260709 12:25:40.153463  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1625) took 1204 ms. Trace:
I20260709 12:25:40.174688  6379 rpcz_store.cc:276] 0709 12:25:38.952709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.103267 (+1150558us) service_pool.cc:224] Handling call
0709 12:25:40.130760 (+ 27493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.130967 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.131000 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.131078 (+    78us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:40.131135 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.131213 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.131373 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.149206  8552 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.181231  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:40.182197  6457 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:40.189999  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 267) took 1207 ms. Trace:
I20260709 12:25:40.167663  6380 rpcz_store.cc:276] 0709 12:25:38.413108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.421450 (+  8342us) service_pool.cc:224] Handling call
0709 12:25:38.474324 (+ 52874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.556909 (+ 82585us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:38.556948 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.586104 (+ 29156us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.789023 (+202919us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:38.789077 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.156192 (+1367115us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:40.156248 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.156298 (+    50us) raft_consensus.cc:1685] finished
0709 12:25:40.156314 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.160699 (+  4385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2166,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":8236,"spinlock_wait_cycles":135373568,"thread_start_us":176526,"threads_started":1,"wal-append.queue_time_us":213221,"wal-append.run_cpu_time_us":1941,"wal-append.run_wall_time_us":1018687}
W20260709 12:25:40.209100  8777 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.665s	user 0.004s	sys 0.000s
I20260709 12:25:40.164129  8638 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.191845  8569 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:40.210995  8638 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.211220  8569 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.213210  8638 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:40.215611  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1629) took 1046 ms. Trace:
I20260709 12:25:40.216037  6379 rpcz_store.cc:276] 0709 12:25:39.168977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.206517 (+1037540us) service_pool.cc:224] Handling call
0709 12:25:40.215122 (+  8605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.215299 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.215324 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.215363 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:40.215416 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.215464 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.215561 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.217471  6379 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:40.218055  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1632) took 1009 ms. Trace:
I20260709 12:25:40.218569  6379 rpcz_store.cc:276] 0709 12:25:39.208730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.216981 (+1008251us) service_pool.cc:224] Handling call
0709 12:25:40.217274 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.217849 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.218010 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.219105  8738 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.219952  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:40.220669  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:40.220952  6379 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:40.221552  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:40.222556  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-2483d7801887403e873adc62429fe046 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:25:40.223389  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-2483d7801887403e873adc62429fe046 on tablet 2483d7801887403e873adc62429fe046 with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:40.229491  8800 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:25:40.240521  8796 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.242127  8789 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.243646  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2193) took 1096 ms. Trace:
I20260709 12:25:40.244184  8735 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.244136  6454 rpcz_store.cc:276] 0709 12:25:39.147277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.147564 (+   287us) service_pool.cc:224] Handling call
0709 12:25:39.168760 (+ 21196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.314966 (+146206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.243356 (+928390us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b54001e23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:40.243572 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178004864}
I20260709 12:25:40.162403  8774 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:40.249809  6425 leader_election.cc:400] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:40.250190  6425 leader_election.cc:403] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:25:40.214093  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:40.212158  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1529) took 1318 ms. Trace:
I20260709 12:25:40.251132  6301 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:40.242045  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:40.251072  6307 rpcz_store.cc:276] 0709 12:25:38.893633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.893750 (+   117us) service_pool.cc:224] Handling call
0709 12:25:38.906821 (+ 13071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.310068 (+403247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.211941 (+901873us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000938d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:40.212104 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775689600}
I20260709 12:25:40.168567  8588 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.155100  8711 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.253551  8498 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.194141  6310 rpcz_store.cc:276] 0709 12:25:38.842209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.911355 (+ 69146us) service_pool.cc:224] Handling call
0709 12:25:38.911803 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.003097 (+ 91294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.003139 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.137595 (+134456us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.376181 (+238586us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:39.376225 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.376299 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.872354 (+496055us) spinlock_profiling.cc:243] Waited 4.32 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:40.038241 (+165887us) raft_consensus.cc:1685] finished
0709 12:25:40.038262 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.049705 (+ 11443us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90814,"raft.queue_time_us":168074,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":28088,"spinlock_wait_cycles":9942400,"thread_start_us":183652,"threads_started":1,"wal-append.queue_time_us":296880}
W20260709 12:25:40.285112  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.294034  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:40.294881  6307 raft_consensus.cc:2376] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5. Re-sending same reply.
I20260709 12:25:40.292290  6304 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:40.222283  6309 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.232474  8529 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.361678  6457 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:40.384268  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:40.386322  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:40.267215  8796 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:40.385293  6310 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:40.268736  8603 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.433316  8368 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:40.269438  8810 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:40.275377  8774 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:40.268200  8594 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.272362  8711 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.508342  6301 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:40.523020  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:40.383803  8569 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:40.274788  8791 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.542184  8791 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:40.538735  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 397762us
I20260709 12:25:40.543655  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:40.551252  8588 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.552232  6382 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:40.390391  8734 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:40.558447  8734 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.516063  8556 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:40.642531  8811 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:25:40.560132  8810 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:40.658006  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114422us
I20260709 12:25:40.662386  8657 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:40.663102  8657 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:40.665261  8657 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.653254  8617 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.535151  8774 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.638175  8569 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:40.564617  8368 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.674013  6309 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:40.668524  8711 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.665759  8657 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:40.701323  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:40.699596  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 255) took 1392 ms. Trace:
I20260709 12:25:40.703490  6385 rpcz_store.cc:276] 0709 12:25:39.306723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.675614 (+1368891us) service_pool.cc:224] Handling call
0709 12:25:40.698929 (+ 23315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.699155 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.699193 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.699248 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:40.699308 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.699378 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.699526 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.695235  8711 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:40.716447  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.716817  6302 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:40.676349  8569 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.707780  8648 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:40.728065  8648 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.721279  8711 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.736536  8368 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.717079  8810 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.691670  8668 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.726948  8791 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.760246  8591 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:40.726279  8617 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:40.708889  8657 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.727105  8594 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:40.770347  8814 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.775089  8368 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.780190  8814 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.780354  8368 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.781108  8657 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.787593  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:40.788347  6383 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:40.789379  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:40.790799  6383 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:40.792348  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:40.791487  8765 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.622s	user 0.012s	sys 0.000s
I20260709 12:25:40.794790  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 254) took 1488 ms. Trace:
W20260709 12:25:40.798650  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.795563  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1622) took 1869 ms. Trace:
I20260709 12:25:40.799415  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:40.800182  6383 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:40.816320  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:40.799870  6381 rpcz_store.cc:276] 0709 12:25:38.925776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.531287 (+605511us) service_pool.cc:224] Handling call
0709 12:25:39.531791 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.596026 (+ 64235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.596059 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.624902 (+ 28843us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:39.624988 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.795332 (+1170344us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:40.795382 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.795504 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33447,"raft.queue_time_us":22303,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":19540,"spinlock_wait_cycles":95968000}
I20260709 12:25:40.818604  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:40.819432  6381 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:40.798957  6379 rpcz_store.cc:276] 0709 12:25:39.306420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.222659 (+916239us) service_pool.cc:224] Handling call
0709 12:25:40.223510 (+   851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.224731 (+  1221us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:40.225943 (+  1212us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.674941 (+448998us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.692681 (+ 17740us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.692867 (+   186us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:40.692917 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.693096 (+   179us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.794561 (+101465us) raft_consensus.cc:1685] finished
0709 12:25:40.794579 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.794719 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":143293,"prepare.queue_time_us":703,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":144078,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":9019,"thread_start_us":305142,"threads_started":1,"wal-append.queue_time_us":482441}
I20260709 12:25:40.835371  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:40.838357  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:40.842499  8813 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:40.843194  8588 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:40.843086  8813 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.843847  8588 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.845342  8813 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.845878  8813 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.846292  8813 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.836958  8368 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:40.850855  8603 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:40.851437  8603 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:40.851908  8603 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:40.852419  6275 connection.cc:256] RPC call timeout handler was delayed by -1.0575863940002819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 12:25:40.853440  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:40.854386  8817 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:40.855229  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2056) took 1206 ms. Trace:
I20260709 12:25:40.855775  6381 rpcz_store.cc:276] 0709 12:25:39.648678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.820242 (+1171564us) service_pool.cc:224] Handling call
0709 12:25:40.820633 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.837152 (+ 16519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.837178 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.854837 (+ 17659us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:40.854897 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.854985 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.855145 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.856663  8529 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:40.857203  8529 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:40.857682  8529 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:40.857925  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 264) took 1020 ms. Trace:
I20260709 12:25:40.803535  8591 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.858466  6381 rpcz_store.cc:276] 0709 12:25:39.837753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.856952 (+1019199us) service_pool.cc:224] Handling call
0709 12:25:40.857301 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.857525 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.857562 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.857615 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:40.857658 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.857724 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.857832 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.865588  6379 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:40.867244  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:40.869518  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2059) took 1220 ms. Trace:
I20260709 12:25:40.869983  6379 rpcz_store.cc:276] 0709 12:25:39.649224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.821684 (+1172460us) service_pool.cc:224] Handling call
0709 12:25:40.865129 (+ 43445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.868991 (+  3862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.869436 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.870177  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:40.767726  8791 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.824909  8648 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:40.800155  8815 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.782349  8814 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.840824  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:40.846230  8588 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.839588  8789 raft_consensus.cc:2764] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:40.900396  8813 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.776484  8617 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.899180  6379 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:25:40.837402  8657 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:40.881421  8817 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.802438  8594 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.825372  8765 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.905715  8657 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:40.899399  8556 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:40.894654  8734 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:40.862095  8591 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.906912  8648 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.930207  6379 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:40.883801  8815 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.930162  6377 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:25:40.928089  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1693) took 2246 ms. Trace:
I20260709 12:25:40.913039  8657 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:40.913614  8556 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:40.938105  6304 rpcz_store.cc:276] 0709 12:25:38.681367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.682499 (+  1132us) service_pool.cc:224] Handling call
0709 12:25:39.735577 (+1053078us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182e9c 00007f6fa0182a5b 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:40.241904 (+506327us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182eee 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:40.927770 (+685866us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b54000b3954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:40.927957 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66934144}
I20260709 12:25:40.919685  8734 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:40.960970  7331 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:25:40.937019  8794 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.774s	user 0.012s	sys 0.000s
I20260709 12:25:40.966836  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:40.935714  8622 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:40.968588  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:40.965711  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:40.952270  8556 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:40.967006  8812 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.201s	user 0.003s	sys 0.002s
I20260709 12:25:40.953159  8363 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.959424  8617 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.933679  8711 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:40.959254  8591 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:40.980288  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:40.984411  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:41.017973  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:40.967873  8809 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:25:40.988008  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:40.938113  8648 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:40.970600  8814 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.987680  8789 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:25:41.010452  8815 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.011291  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:40.969686  8785 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.982255  8588 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:40.974707  8668 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:40.982594  8569 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:41.025554  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:41.025825  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.039848  8556 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.005486  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:41.032732  8588 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.051108  8588 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:40.970157  8794 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.053582  8794 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:41.050618  6380 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:41.047590  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2061) took 1136 ms. Trace:
I20260709 12:25:41.055954  6379 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:41.056237  6384 rpcz_store.cc:276] 0709 12:25:39.911089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.994076 (+ 82987us) service_pool.cc:224] Handling call
0709 12:25:41.047396 (+1053320us) spinlock_profiling.cc:243] Waited 59.2 ms on lock 0x7b5400064854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:41.047534 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":136216960}
I20260709 12:25:41.022801  8814 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.027887  8617 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:41.023803  7314 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.006465  8622 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.031009  8735 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:41.045744  8557 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:41.046701  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.10456e+06us
I20260709 12:25:41.062479  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:41.047441  8820 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:41.048094  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2063) took 1136 ms. Trace:
I20260709 12:25:41.049510  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1714) took 1140 ms. Trace:
I20260709 12:25:41.049576  6378 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:41.069188  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.022950  8764 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:41.059062  6452 raft_consensus.cc:2368] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5.
I20260709 12:25:41.050580  8711 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.058889  8824 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:25:41.026731  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.067118  6382 rpcz_store.cc:276] 0709 12:25:39.911616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.918887 (+  7271us) service_pool.cc:224] Handling call
0709 12:25:41.047923 (+1129036us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54001946d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:41.048043 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385245056}
I20260709 12:25:41.074234  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:41.068229  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.022295  8363 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.060595  8814 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.067284  6300 rpcz_store.cc:276] 0709 12:25:39.909530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.086926 (+177396us) service_pool.cc:224] Handling call
0709 12:25:40.110886 (+ 23960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.190547 (+ 79661us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.190578 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.264796 (+ 74218us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.777679 (+512883us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:40.777722 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.777795 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.049252 (+271457us) raft_consensus.cc:1685] finished
0709 12:25:41.049277 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.049440 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2701,"raft.queue_time_us":138724,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":4253,"thread_start_us":299601,"threads_started":2,"wal-append.queue_time_us":588382}
I20260709 12:25:41.085536  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:41.062657  8735 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:41.032534  8810 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:25:41.061038  8617 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:41.111460  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.117947  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:41.076228  8668 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:41.059319  8785 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:41.120669  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96356.1us
I20260709 12:25:41.106237  8735 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:41.126201  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:41.132716  6309 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.132530  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.133047  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:41.115244  8617 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:41.135506  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.118618  8711 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:41.115091  8594 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:41.091859  8764 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.060622  8734 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.143672  6384 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:25:41.136358  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.144591  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100908us
I20260709 12:25:41.149932  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:41.154201  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 259) took 1504 ms. Trace:
I20260709 12:25:41.156756  6309 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:25:41.138571  8711 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:41.162014  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.164594  6382 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:41.141793  8765 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.165867  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:41.134986  8591 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:25:41.147679  8835 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:41.169064  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51192.1us
I20260709 12:25:41.175443  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.154824  6383 rpcz_store.cc:276] 0709 12:25:39.649851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.824388 (+1174537us) service_pool.cc:224] Handling call
0709 12:25:40.860650 (+ 36262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.860894 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:40.974578 (+113684us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.152100 (+177522us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:41.152172 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:41.152244 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.152333 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.154116 (+  1783us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":119187,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":43988,"spinlock_wait_cycles":1822976,"thread_start_us":289156,"threads_started":3,"wal-append.queue_time_us":127070}
I20260709 12:25:41.176541  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:41.183911  8594 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.191812  8594 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.193826  8594 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.194944  8785 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.177949  8556 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:41.196878  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.197933  8711 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.182435  8833 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.199131  8711 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:41.200237  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:41.172142  8835 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:41.201416  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:41.204936  7331 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:41.183202  8829 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:25:41.209281  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:41.218513  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.219302  7335 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:41.223632  7314 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:41.223843  8764 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.225399  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:41.230821  7317 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:41.231318  7319 raft_consensus.cc:1217] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 12:25:41.232383  7319 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:25:41.236250  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.237110  7314 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:41.205988  8835 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:41.238394  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.241748  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.242115  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.242996  7318 raft_consensus.cc:2368] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6.
I20260709 12:25:41.194944  8363 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:41.202993  8648 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:41.217535  8806 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.949s	user 0.007s	sys 0.010s
I20260709 12:25:41.206116  7316 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:41.244184  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.306041  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.317898  7317 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:41.306980  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.312186  8603 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:41.316541  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 107363us
I20260709 12:25:41.353922  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.369235  7331 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:41.371137  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.540978  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.190397  8657 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:41.400719  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.373149  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:41.263590  8569 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.428694  6275 leader_election.cc:400] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6.
I20260709 12:25:41.429453  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.430817  8641 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:41.431852  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:41.433521  8557 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:41.405283  8363 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:41.405186  8833 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:41.438866  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 213563us
I20260709 12:25:41.441119  8839 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:25:41.485562  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.484387  8785 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:41.485939  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.500563  8603 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:41.501303  8529 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.643s	user 0.006s	sys 0.006s
I20260709 12:25:41.514653  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.698575  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:41.699590  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:41.700644  6381 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:41.514901  8617 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.516157  8735 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.440274  8556 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.439605  8433 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:41.439678  8765 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:41.632457  8622 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.541621  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.538445  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.562073  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.562376  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.562731  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.563483  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.564177  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.730664  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.731395  6458 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:41.566304  6275 leader_election.cc:403] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:41.577642  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 170712us
I20260709 12:25:41.577875  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.733525  6379 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:41.579573  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 206534us
I20260709 12:25:41.579852  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.566974  8814 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.591984  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1627) took 2480 ms. Trace:
I20260709 12:25:41.655050  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.739818  6451 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:41.592844  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 220) took 1680 ms. Trace:
I20260709 12:25:41.682161  8463 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.740254  6452 rpcz_store.cc:276] 0709 12:25:39.912242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.938268 (+ 26026us) service_pool.cc:224] Handling call
0709 12:25:41.592580 (+1654312us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:41.592735 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2450388736}
I20260709 12:25:41.631583  8791 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.614034  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 256) took 2307 ms. Trace:
I20260709 12:25:41.628290  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1630) took 2443 ms. Trace:
W20260709 12:25:41.677134  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2062) took 1765 ms (client timeout 1655 ms). Trace:
I20260709 12:25:41.742831  6454 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:25:41.677037  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 158637us
I20260709 12:25:41.743075  6380 rpcz_store.cc:276] 0709 12:25:39.184422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.205319 (+1020897us) service_pool.cc:224] Handling call
0709 12:25:41.050339 (+845020us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:41.058170 (+  7831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.628020 (+569850us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b5400064850. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:41.628207 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922078464}
I20260709 12:25:41.681067  6384 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:41.699494  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:41.438476  8588 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:41.644217  8827 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.470s	user 0.006s	sys 0.010s
I20260709 12:25:41.713739  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:41.718117  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.719266  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:41.440193  8817 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.004s
I20260709 12:25:41.720532  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.648252  8572 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:41.719909  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 262092us
I20260709 12:25:41.723105  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.637406  8843 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.609880  8648 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:41.734768  6382 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:41.736307  8668 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.654387  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.732797  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:41.737361  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 310601us
I20260709 12:25:41.737694  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:41.734144  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:41.580646  8569 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:41.716915  8711 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.000s
I20260709 12:25:41.726962  8433 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.739470  6378 rpcz_store.cc:276] 0709 12:25:39.112090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.163333 (+1051243us) service_pool.cc:224] Handling call
0709 12:25:41.049376 (+886043us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.069347 (+ 19971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.591711 (+522364us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b54001946d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:41.591899 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2037151616}
I20260709 12:25:41.661512  8738 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:41.657369  8641 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.757869  8603 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.773867  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:41.667647  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.741545  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.775516  6456 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:41.742529  6385 rpcz_store.cc:276] 0709 12:25:39.306987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.704954 (+1397967us) service_pool.cc:224] Handling call
0709 12:25:40.741961 (+ 37007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.918467 (+176506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.918529 (+    62us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.918574 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:40.918619 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.613608 (+694989us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:41.613657 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.613966 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405400320}
W20260709 12:25:41.743155  6377 rpcz_store.cc:269] 0709 12:25:39.911361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.922604 (+ 11243us) service_pool.cc:224] Handling call
0709 12:25:41.676941 (+1754337us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:41.677071 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142446336}
I20260709 12:25:41.715307  8529 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.747380  6305 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:41.710835  8557 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.745196  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.749557  6304 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:41.751791  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:41.715866  8764 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:41.710115  7314 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:41.745914  8734 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6.
I20260709 12:25:41.755012  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:41.804805  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:41.805112  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50184us
I20260709 12:25:41.805578  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:41.642910  6206 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.759505  8817 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:41.758853  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.809867  6457 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:41.761404  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.680531  8785 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.764370  6453 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:41.767246  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:41.812788  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:41.763280  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.769989  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:41.763662  8572 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.741504  8765 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:41.744843  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.776207  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.777057  6300 raft_consensus.cc:2368] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6.
I20260709 12:25:41.716406  8363 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.720410  8556 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.859717  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:41.859784  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124299us
I20260709 12:25:41.860493  6382 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:41.749112  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:41.781121  6309 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:41.757820  6307 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6.
I20260709 12:25:41.781085  8738 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:41.813454  6308 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:41.814618  6379 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:41.809170  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.817022  6206 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.817878  8569 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:41.808840  8764 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.775200  8774 raft_consensus.cc:2764] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260709 12:25:41.848248  8835 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.609s	user 0.006s	sys 0.005s
I20260709 12:25:41.829459  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:41.832046  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62140.3us
I20260709 12:25:41.847188  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:41.876762  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:41.877197  6381 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:41.856992  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 89842.2us
I20260709 12:25:41.750348  8463 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.880065  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.880913  6309 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:41.875998  6377 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:41.859891  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 275) took 1750 ms. Trace:
I20260709 12:25:41.882207  6310 rpcz_store.cc:276] 0709 12:25:40.109308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.514571 (+405263us) service_pool.cc:224] Handling call
0709 12:25:40.539752 (+ 25181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.638828 (+ 99076us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.638879 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.623711 (+984832us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:25:41.624250 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.742680 (+118430us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:41.742715 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.742778 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.832887 (+ 90109us) raft_consensus.cc:1685] finished
0709 12:25:41.832916 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.859828 (+ 26912us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6396,"raft.queue_time_us":5623,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":3865,"spinlock_wait_cycles":23452032,"thread_start_us":117986,"threads_started":1,"wal-append.queue_time_us":187897}
I20260709 12:25:41.866882  6303 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:41.770952  8843 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.829391  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1623) took 2903 ms. Trace:
I20260709 12:25:41.886444  6305 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:41.885133  6376 rpcz_store.cc:276] 0709 12:25:38.926029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.093282 (+1167253us) service_pool.cc:224] Handling call
0709 12:25:40.093605 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.103936 (+ 10331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.103960 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:41.615858 (+1511898us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:25:41.616430 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.724368 (+107938us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:41.724404 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.725050 (+   646us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.829094 (+104044us) raft_consensus.cc:1685] finished
0709 12:25:41.829117 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.829334 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4619,"raft.queue_time_us":74,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":9170,"spinlock_wait_cycles":8892800,"thread_start_us":107481,"threads_started":1,"wal-append.queue_time_us":210588}
I20260709 12:25:41.815223  8603 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:41.863163  6385 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:41.776521  8648 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.889732  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:41.897262  6378 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:41.891968  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:41.866734  8773 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:41.869131  6206 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 4 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 5 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.907924  6206 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.855650  8641 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.865180  8711 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:41.903995  8857 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:41.904102  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:41.904659  8852 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.809556  8557 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.915773  6453 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:41.917702  6378 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:41.920209  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:41.888226  8734 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:41.923049  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:41.930820  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:41.941546  6457 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:41.927711  6377 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:41.933213  8773 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:41.909952  8622 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:41.931602  8852 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:41.943492  6377 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:41.944222  6452 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:41.946436  8735 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:41.947105  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 214369us
I20260709 12:25:41.907683  8603 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:41.948084  8588 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.948709  8529 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.949258  8738 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:41.951154  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.014256  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:41.952684  8765 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.955232  8814 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.955351  8648 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:41.896469  8794 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:41.955994  8857 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:41.957240  8556 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.957643  8859 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:41.961131  7316 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.000s
I20260709 12:25:41.961457  8557 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:41.935810  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1547) took 1601 ms. Trace:
I20260709 12:25:41.921020  7332 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:41.974395  6305 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:41.994010  6201 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 295) took 12119 ms (client timeout 15000 ms). Trace:
I20260709 12:25:41.994309  6381 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:41.912248  7318 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:41.995877  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:41.947501  8572 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:41.980996  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:42.045133  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:42.045975  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65091.9us
W20260709 12:25:41.998250  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.047384  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:41.894706  8835 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:41.992271  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:41.951722  8843 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:42.050704  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54911.2us
I20260709 12:25:42.051012  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:42.028945  8735 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:41.946480  8363 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:42.056433  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64246.3us
I20260709 12:25:42.030418  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:42.089462  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 59146.4us
I20260709 12:25:42.090114  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:42.090983  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:41.997573  8622 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:42.009522  7285 leader_election.cc:400] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5.
W20260709 12:25:42.093231  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97244.6us
I20260709 12:25:42.020265  6308 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:42.052389  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:42.025727  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:42.102731  6301 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:42.056838  8529 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:42.056949  8833 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:42.041136  6302 rpcz_store.cc:276] 0709 12:25:40.334521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.725118 (+390597us) service_pool.cc:224] Handling call
0709 12:25:40.778948 (+ 53830us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.836902 (+ 57954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.836948 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.878152 (+ 41204us) log.cc:844] Serialized 26 byte log entry
0709 12:25:41.075354 (+197202us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:41.075420 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.075526 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.643451 (+567925us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:41.913045 (+269594us) raft_consensus.cc:1685] finished
0709 12:25:41.913069 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.935721 (+ 22652us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57490,"raft.queue_time_us":63617,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":143,"spinlock_wait_cycles":6893952,"thread_start_us":168101,"threads_started":1,"wal-append.queue_time_us":235393}
I20260709 12:25:42.052717  8557 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:42.051796  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:42.104679  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53013.2us
I20260709 12:25:42.108512  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:42.108726  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:42.109568  7332 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
W20260709 12:25:42.043596  6201 rpcz_store.cc:269] 0709 12:25:29.640636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.646415 (+  5779us) service_pool.cc:224] Handling call
0709 12:25:41.754579 (+12108164us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:38.344890 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:38.345784 (+   894us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:38.345862 (+    78us) write_op.cc:435] Acquired schema lock
0709 12:25:38.345906 (+    44us) tablet.cc:662] Decoding operations
0709 12:25:38.346773 (+   867us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:38.346895 (+   122us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:38.346933 (+    38us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:38.347662 (+   729us) tablet.cc:701] Row locks acquired
0709 12:25:38.347729 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:25:38.348012 (+   283us) write_op.cc:270] Start()
0709 12:25:38.359968 (+ 11956us) write_op.cc:276] Timestamp: P: 1783599938347897 usec, L: 0
0709 12:25:38.363833 (+  3865us) op_driver.cc:348] REPLICATION: starting
0709 12:25:38.527904 (+164071us) log.cc:844] Serialized 1890 byte log entry
0709 12:25:38.913338 (+385434us) op_driver.cc:464] REPLICATION: finished
0709 12:25:39.411923 (+498585us) write_op.cc:301] APPLY: starting
0709 12:25:39.424552 (+ 12629us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:39.504825 (+ 80273us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:39.549137 (+ 44312us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:39.550114 (+   977us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:39.610672 (+ 60558us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:39.610730 (+    58us) write_op.cc:312] APPLY: finished
0709 12:25:39.611119 (+   389us) log.cc:844] Serialized 41 byte log entry
0709 12:25:39.611335 (+   216us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.611440 (+   105us) write_op.cc:454] Released schema lock
0709 12:25:39.611511 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:39.611938 (+   427us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:39.611969 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"dns_us":271,"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":474950,"apply.run_cpu_time_us":4280,"apply.run_wall_time_us":212058,"num_ops":4,"prepare.queue_time_us":282065,"prepare.run_cpu_time_us":32972,"prepare.run_wall_time_us":430344,"raft.queue_time_us":63080,"raft.run_cpu_time_us":54392,"raft.run_wall_time_us":293205,"replication_time_us":549314,"spinlock_wait_cycles":2944,"thread_start_us":695184,"threads_started":4,"wal-append.queue_time_us":272739,"wal-append.run_cpu_time_us":14697,"wal-append.run_wall_time_us":1823380}]]}
I20260709 12:25:42.060653  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1723) took 1043 ms. Trace:
I20260709 12:25:42.110448  6303 rpcz_store.cc:276] 0709 12:25:41.016928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.568574 (+551646us) service_pool.cc:224] Handling call
0709 12:25:41.568817 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.884827 (+316010us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.060269 (+175442us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54001b3fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:42.060559 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684684672}
I20260709 12:25:42.058552  8835 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:42.066386  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1718) took 1089 ms. Trace:
I20260709 12:25:42.073135  8657 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:42.074992  8773 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.079288  8603 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.081691  8852 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:42.082813  8463 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:42.037379  8859 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.088186  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.056018  8556 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:42.042863  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:42.113523  6307 rpcz_store.cc:276] 0709 12:25:40.976761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.016513 (+ 39752us) service_pool.cc:224] Handling call
0709 12:25:42.066113 (+1049600us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b540010fc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:42.066299 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574882176}
I20260709 12:25:42.069093  8857 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.042212  7316 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:42.116676  6425 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.609s (SENT)
I20260709 12:25:42.021978  8734 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.091461  8648 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.120368  7317 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:42.039985  8794 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:42.120788  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 259430us
I20260709 12:25:42.120893  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:42.213505  6302 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.123118  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:42.112197  8833 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:42.139789  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.158131  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:42.215392  6309 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:42.101334  8622 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:42.115499  8789 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:42.224944  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101893us
W20260709 12:25:42.185096  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76695.6us
I20260709 12:25:42.239365  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:42.239565  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:42.185118  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:42.115069  8363 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:42.119874  8817 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.197333  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:42.209031  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 284750us
I20260709 12:25:42.122097  8813 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:42.132704  8814 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:42.223675  6380 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:42.381160  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 141687us
I20260709 12:25:42.326411  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:42.190412  8735 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.383072  6377 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.318812  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:42.262157  7329 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:25:42.312842  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 179392us
I20260709 12:25:42.132985  8859 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.331375  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.333585  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:42.324244  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:42.326813  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:42.388334  6302 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:42.395565  6309 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:42.196492  8648 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:42.329166  8871 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:25:42.190845  7320 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.466157  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:42.430816  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:42.240969  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:42.196732  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:42.430680  6383 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.196394  8835 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:42.328462  8869 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:25:42.327032  8870 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:25:42.431615  6381 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.374954  8843 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.325995  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:42.223125  8833 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.294761  8791 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:42.299885  8557 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.468987  6304 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:42.524394  6459 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:42.387202  8817 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.268448  8774 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.242s
I20260709 12:25:42.477680  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:42.388976  8617 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:42.319067  8837 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:42.238157  8789 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:42.385058  8814 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:42.383548  8813 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.330348  8785 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:42.381703  8765 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:42.464139  8648 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.336078  8572 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:42.262749  8657 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.440078  8363 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:42.238423  8852 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.240s
I20260709 12:25:42.238281  8764 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:42.530072  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.537438  6381 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:42.529878  8817 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.513170  8879 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.531277  8791 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:42.526839  8557 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:42.561422  6305 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
W20260709 12:25:42.537513  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 106790us
I20260709 12:25:42.465205  8735 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.535914  8789 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:42.527259  8878 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:42.545544  8785 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:42.538779  8814 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:42.533341  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:42.538647  8813 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.527170  8843 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:42.596866  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 480199us
I20260709 12:25:42.530212  8711 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:42.544158  8657 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.598330  8557 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.639575  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:42.640545  6301 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:42.606657  8572 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:42.644170  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.645215  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:42.539225  8837 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.616389  8657 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.545210  8765 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.652005  8836 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:25:42.658779  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 132986us
I20260709 12:25:42.665174  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.674707  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1647) took 1158 ms. Trace:
I20260709 12:25:42.608134  8797 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:25:42.678965  6380 rpcz_store.cc:276] 0709 12:25:41.516376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.856230 (+339854us) service_pool.cc:224] Handling call
0709 12:25:41.875322 (+ 19092us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.386985 (+511663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.674412 (+287427us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540023dcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:42.674643 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800711424}
I20260709 12:25:42.647504  8617 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:42.682881  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:42.685186  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70261.2us
I20260709 12:25:42.661911  8433 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:42.689980  8852 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.690555  8852 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:42.691764  8874 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:25:42.692867  8852 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:42.695274  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:42.696682  6425 leader_election.cc:336] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.637s (SENT)
I20260709 12:25:42.658052  8836 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:42.698819  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:42.685286  6302 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:42.647393  8572 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:42.703740  8881 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:42.704169  8881 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.704500  8881 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.706578  8852 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:42.706959  8875 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:25:42.707684  8852 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:42.708142  8852 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:42.709108  8852 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.587543  8879 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.700860  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:42.711755  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:42.713096  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1724) took 1690 ms (client timeout 2220 ms). Trace:
I20260709 12:25:42.713526  6459 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:25:42.713582  6300 rpcz_store.cc:269] 0709 12:25:41.022926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.166782 (+143856us) service_pool.cc:224] Handling call
0709 12:25:42.712889 (+1546107us) spinlock_profiling.cc:243] Waited 594 ms on lock 0x7b5400062a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:42.713025 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1366283136}
I20260709 12:25:42.638322  8711 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:42.741791  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 155847us
I20260709 12:25:42.662638  8794 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:42.702601  8866 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:25:42.746557  6377 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:42.753031  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:42.754619  6301 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:42.677960  8876 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 12:25:42.749320  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:42.705912  8622 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.765985  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:42.769444  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:42.710415  8852 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:42.702852  8734 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:42.773460  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.010s	sys 0.016s
I20260709 12:25:42.769141  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:42.794283  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.011s	sys 0.016s
I20260709 12:25:42.744678  8872 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:42.618129  8878 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:42.794834  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 556639us
W20260709 12:25:42.858044  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 88694.9us
I20260709 12:25:42.856951  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.749580  8858 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:42.872275  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:42.859190  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:42.755359  8886 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:42.883661  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:42.666833  8363 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.880221  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:42.874883  8668 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:25:42.887871  8794 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:25:42.766445  8433 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.124s
I20260709 12:25:42.747224  8791 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.876544  8884 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:42.712898  8617 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.871886  8861 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.897449  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.699543  8836 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.914626  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.919708  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:42.875428  8872 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:42.947045  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:42.949494  6380 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:42.952203  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:42.952939  6381 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:42.895865  8883 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:42.900152  8878 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:42.921849  8588 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:42.991564  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:42.983376  6383 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.001s
I20260709 12:25:42.992394  6383 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
W20260709 12:25:42.994304  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.917344  8886 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:42.996623  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.998420  8711 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:42.888530  8572 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:42.998929  8765 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.000726  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.001562  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.001694  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.002365  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.003237  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.020305  6308 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:43.020800  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:43.011210  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.006s	sys 0.004s
W20260709 12:25:42.989512  8859 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.526s	user 0.006s	sys 0.003s
W20260709 12:25:43.015964  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.011s	sys 0.015s
I20260709 12:25:42.889338  8858 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:42.964241  8836 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:43.022346  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.006s	sys 0.005s
I20260709 12:25:43.036296  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:42.920692  8861 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:43.036469  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.011s	sys 0.015s
I20260709 12:25:43.021804  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1645) took 1506 ms. Trace:
I20260709 12:25:43.039979  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.964337  8891 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:43.040130  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 651461us
W20260709 12:25:43.027100  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.040907  6302 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:43.040555  6378 rpcz_store.cc:276] 0709 12:25:41.515856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.773597 (+257741us) service_pool.cc:224] Handling call
0709 12:25:43.020363 (+1246766us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:43.021714 (+  1351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282412928}
I20260709 12:25:42.994381  8879 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.043401  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:42.964820  7320 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:43.056753  6301 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:43.044589  8859 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.059649  8859 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:43.060271  8859 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:43.060657  8859 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.061017  8859 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.062889  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:43.063347  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:43.064028  7335 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:43.046016  8863 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.809s	user 0.005s	sys 0.000s
I20260709 12:25:43.048225  8814 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.003221  8872 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.072083  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:43.075493  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.015110  8588 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:43.078383  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:43.086582  6450 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:25:43.087991  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51777.3us
I20260709 12:25:43.014909  8881 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.090355  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.091373  6301 raft_consensus.cc:2410] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:25:43.091704  8881 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:43.092284  8879 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.093885  6451 raft_consensus.cc:2368] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6.
I20260709 12:25:43.084383  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:43.083302  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:43.096413  6454 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:43.097306  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:43.098182  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:43.099047  6378 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:43.112537  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:43.119400  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.120352  6302 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:43.125133  6450 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:25:43.106990  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:43.131659  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.136379  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 216751us
I20260709 12:25:43.136399  6304 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:43.144304  6378 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:43.154456  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:43.089406  8884 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:43.036074  8833 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.160514  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:43.081123  8765 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.170240  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:43.207424  6450 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:43.080341  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 545) took 1101 ms. Trace:
I20260709 12:25:43.212502  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.084024  8572 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:43.187652  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:43.182294  8617 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.158042  8879 raft_consensus.cc:2764] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:43.184659  8594 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:43.204069  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43662.2us
I20260709 12:25:43.208084  6301 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:43.209200  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.210536  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.243839  6303 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6.
I20260709 12:25:43.173375  8888 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.137596  8896 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.214733  6450 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:43.142438  8843 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:43.232345  8648 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 12:25:43.257643  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:43.162473  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:43.218713  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:43.227363  8900 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.212895  7320 rpcz_store.cc:276] 0709 12:25:41.979260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.995342 (+ 16082us) service_pool.cc:224] Handling call
0709 12:25:43.079507 (+1084165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:43.235901  7314 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.205133  8892 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:25:43.258615  6452 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:43.238158  8858 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.172884  8641 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:25:43.172452  8881 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 12:25:43.244727  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:43.243219  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:43.177656  8657 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.265681  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:43.228377  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:43.226404  8904 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:43.225044  8837 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.304926  8858 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:43.309132  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64522.7us
I20260709 12:25:43.249298  8594 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.224155  8765 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:43.312281  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69121.9us
I20260709 12:25:43.324923  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:43.248622  8888 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:43.312367  7335 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:43.317358  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:43.328873  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:43.311034  8734 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:43.321388  8889 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.316972  8363 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:43.313133  8641 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:43.330214  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:43.310878  8904 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:43.367898  7288 leader_election.cc:336] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.538s (SENT)
I20260709 12:25:43.268805  8648 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.334663  8892 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:43.332734  8896 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:43.337136  8881 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.337466  8897 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:25:43.338383  8765 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:43.338922  8657 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.340358  8594 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:43.329790  8888 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.316447  8858 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:43.341282  8889 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.273226  8900 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.229130  8886 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.346854  8908 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:43.409456  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.352301  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 465) took 2021 ms. Trace:
I20260709 12:25:43.353420  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2082) took 1545 ms. Trace:
I20260709 12:25:43.411018  6382 rpcz_store.cc:276] 0709 12:25:41.807664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.951970 (+144306us) service_pool.cc:224] Handling call
0709 12:25:43.020536 (+1068566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.021392 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.353228 (+331836us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003f25d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:43.353360 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2456749824}
I20260709 12:25:43.355870  8735 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.359404  8833 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.313839  8791 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:25:43.331084  8898 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:43.416836  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81935us
I20260709 12:25:43.367839  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:43.241426  8588 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:25:43.354339  8817 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.770s	user 0.005s	sys 0.003s
I20260709 12:25:43.362090  8837 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:43.364392  8641 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:43.422883  8641 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:43.362681  8363 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:43.370332  8557 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.713s	user 0.008s	sys 0.002s
I20260709 12:25:43.365121  8734 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:43.404374  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 242008us
W20260709 12:25:43.393494  8789 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.000s
W20260709 12:25:43.404498  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 185873us
I20260709 12:25:43.371580  8814 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.370237  8904 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:43.392448  8711 raft_consensus.cc:2764] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:43.390026  8881 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.410939  7331 rpcz_store.cc:276] 0709 12:25:41.331123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.568485 (+237362us) service_pool.cc:224] Handling call
0709 12:25:41.568786 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:41.775402 (+206616us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:41.780831 (+  5429us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.945120 (+164289us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b180033b860. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:25:42.115492 (+170372us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:42.115807 (+   315us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:42.115998 (+   191us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.350799 (+1234801us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b180033b860. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419
0709 12:25:43.352014 (+  1215us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:43.352061 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.352201 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":256,"prepare.queue_time_us":5381,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3808,"raft.queue_time_us":2177,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":1731,"spinlock_wait_cycles":519440256,"thread_start_us":168459,"threads_started":1,"wal-append.queue_time_us":557440}
W20260709 12:25:43.426607  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58835.1us
I20260709 12:25:43.429261  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:43.334983  8884 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.429244  8363 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.448801  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:43.444766  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.432281  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:43.461328  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.428516  8837 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:43.431593  8735 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.453883  8900 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.465572  8889 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.474606  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:43.436271  8861 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:43.448524  8881 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:43.479696  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61598.2us
I20260709 12:25:43.480822  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:43.434162  8908 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:25:43.474534  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:43.433507  8789 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.433449  8557 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.432439  8817 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.478063  8711 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.492926  6307 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:43.477247  8872 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 12:25:43.473657  8735 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:43.495791  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63587us
I20260709 12:25:43.496538  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:43.490343  8814 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:43.488137  8861 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:43.526753  8861 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:43.527819  8861 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.495391  8734 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:43.529526  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 230) took 1511 ms (client timeout 1607 ms). Trace:
I20260709 12:25:43.498745  6377 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
W20260709 12:25:43.512965  7288 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.686s (SENT)
I20260709 12:25:43.530921  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1733) took 1798 ms. Trace:
I20260709 12:25:43.494266  8711 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:43.523092  8852 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 12:25:43.517982  8789 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:43.518173  8910 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:25:43.542090  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:43.543422  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:43.543565  8789 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:43.521169  8891 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:43.544425  8789 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:43.544684  8891 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:43.544878  8789 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:43.545382  8789 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.523155  8817 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:43.525307  8859 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.497490  8648 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:43.531353  6310 rpcz_store.cc:276] 0709 12:25:41.732947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.884636 (+151689us) service_pool.cc:224] Handling call
0709 12:25:41.906952 (+ 22316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:42.243551 (+336599us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.243595 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:42.243652 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:42.243695 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.530687 (+1286992us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54002e8454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:43.530737 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.530859 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773590912}
I20260709 12:25:43.508038  8791 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:43.502303  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 73139.4us
W20260709 12:25:43.530032  6451 rpcz_store.cc:269] 0709 12:25:42.018361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.745575 (+727214us) service_pool.cc:224] Handling call
0709 12:25:43.529318 (+783743us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:43.529456 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741188992}
I20260709 12:25:43.448954  8858 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.540670  8711 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.518474  8837 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.546692  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:43.556622  6382 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:43.497602  8872 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:43.495934  8900 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:43.498425  8735 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:43.558204  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:43.559327  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:43.550225  6454 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260709 12:25:43.551188  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:43.560418  6454 pending_rounds.cc:85] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 7
I20260709 12:25:43.560770  6381 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:43.553390  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:43.552321  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 222195us
I20260709 12:25:43.530616  8814 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:43.561851  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:43.565925  6382 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:43.571187  6381 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:43.560698  6454 pending_rounds.cc:107] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:25:43.574832  6454 raft_consensus.cc:2889] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:43.556972  8813 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.914s	user 0.010s	sys 0.002s
I20260709 12:25:43.557389  8872 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:25:43.600625  7288 leader_election.cc:336] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.774s (SENT)
I20260709 12:25:43.601079  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:43.601156  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:43.602035  6381 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:43.605444  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 43698.7us
I20260709 12:25:43.548126  8888 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:43.545722  8891 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.627621  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:43.628209  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:43.612089  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:43.634193  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:43.859105  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.775401  6274 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.655s (SENT)
W20260709 12:25:43.873014  7288 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.790s (SENT)
I20260709 12:25:43.606526  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:44.229468  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 454113us
W20260709 12:25:44.223453  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 670068us
W20260709 12:25:44.230921  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 357960us
I20260709 12:25:43.628418  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:44.223354  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2084) took 1905 ms (client timeout 1655 ms). Trace:
I20260709 12:25:43.569026  8836 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:44.231359  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:44.228015  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:44.234864  6382 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:44.232195  6379 rpcz_store.cc:269] 0709 12:25:42.318195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.318382 (+   187us) service_pool.cc:224] Handling call
0709 12:25:44.223004 (+1904622us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:44.223245 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569652096}
W20260709 12:25:44.233955  6381 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.622s	user 0.020s	sys 0.013s
I20260709 12:25:44.244294  6381 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:43.573411  8900 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:43.569268  8791 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:43.576411  8588 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:44.355547  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:43.894539  8926 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
W20260709 12:25:44.204694  8363 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
W20260709 12:25:44.204550  7314 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.897s	user 0.009s	sys 0.005s
W20260709 12:25:44.260557  8904 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.798s	user 0.014s	sys 0.000s
I20260709 12:25:43.786458  8925 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:44.076292  8572 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.848s	user 0.004s	sys 0.008s
I20260709 12:25:43.866917  8892 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:43.893003  8924 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:25:44.223631  8725 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:43.629001  8817 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.754s
I20260709 12:25:44.226012  8858 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:43.661811  8915 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:44.356488  6305 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:43.611291  8896 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:44.355680  8886 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.025s	sys 0.019s
I20260709 12:25:43.611794  8738 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:43.708034  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:44.393097  7314 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:44.424256  6305 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:44.406278  6382 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:43.600056  8813 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.625505  8617 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.770s
I20260709 12:25:43.600636  8641 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:44.410122  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 798087us
I20260709 12:25:44.390122  8859 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:44.436517  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 535) took 3188 ms. Trace:
I20260709 12:25:44.447589  7331 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:44.394059  8588 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.404923  8904 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.450757  6302 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:43.909425  7313 tablet_service.cc:1558] Processing DeleteTablet for tablet 2483d7801887403e873adc62429fe046 with delete_type TABLET_DATA_TOMBSTONED (TS 561a4e21d8b344a79af44568ca80a3c5 not found in new config with opid_index -1) from {username='slave'} at 127.0.0.1:45486
I20260709 12:25:43.735456  8814 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:43.735791  8872 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.411597  8725 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:25:44.422833  8886 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.026s	sys 0.020s
I20260709 12:25:44.447446  7329 rpcz_store.cc:276] 0709 12:25:41.240122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:41.240236 (+   114us) service_pool.cc:224] Handling call
0709 12:25:41.240590 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.427851 (+1187261us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.428541 (+2000690us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54006f7a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:44.428690 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2301721344}
I20260709 12:25:44.452713  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:44.395413  8363 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:44.454834  8896 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:44.457520  7319 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:44.441785  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 473) took 1347 ms. Trace:
I20260709 12:25:44.412755  8892 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.448956  8938 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.469100  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.471454  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.471727  6305 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:44.472198  6300 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:44.472158  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.472455  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.473053  6309 raft_consensus.cc:2376] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4. Re-sending same reply.
I20260709 12:25:44.473845  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.474133  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.474671  6309 raft_consensus.cc:2376] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6. Re-sending same reply.
I20260709 12:25:44.474995  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.475399  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.476040  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.476372  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.477092  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.477321  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.477751  6308 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:44.473076  6301 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:44.479705  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.462298  8836 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:44.484850  6301 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:44.485085  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.490921  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1560) took 1833 ms. Trace:
I20260709 12:25:44.469774  8896 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:44.491392  6304 rpcz_store.cc:276] 0709 12:25:42.657471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.657663 (+   192us) service_pool.cc:224] Handling call
0709 12:25:42.671677 (+ 14014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.168525 (+496848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.489145 (+1320620us) spinlock_profiling.cc:243] Waited 450 ms on lock 0x7b54001b35d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:44.490853 (+  1708us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1034136448}
I20260709 12:25:44.486868  8872 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:44.478572  8938 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.478821  8859 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.486807  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.479086  8886 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.510605  8935 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.511188  8935 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:44.512133  8789 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.966s	user 0.007s	sys 0.007s
I20260709 12:25:44.513129  6304 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:44.485572  8940 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.517089  6301 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:44.453428  8791 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.449951  8900 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.521059  6301 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:44.465287  7314 rpcz_store.cc:276] 0709 12:25:43.094684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.102567 (+  7883us) service_pool.cc:224] Handling call
0709 12:25:44.438182 (+1335615us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:44.514245  8813 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:44.520357  8641 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:44.512530  8789 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:44.524204  8735 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.950s	user 0.008s	sys 0.005s
I20260709 12:25:44.526901  6301 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.521927  8888 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:44.555928  6301 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:44.515112  8859 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:44.561686  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 1.40545e+06us
I20260709 12:25:44.496663  8725 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.517659  8886 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.564332  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:44.569000  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 164256us
I20260709 12:25:44.569391  6304 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
W20260709 12:25:44.565515  8927 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:25:44.582881  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:44.577476  6425 connection.cc:256] RPC call timeout handler was delayed by -1.3904453000000103s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
W20260709 12:25:44.578807  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 347594us
W20260709 12:25:44.585948  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.141s (SENT) [suppressed 27 similar messages]
W20260709 12:25:44.588500  6425 connection.cc:256] RPC call timeout handler was delayed by -1.0453088059998663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:25:44.588871  6425 connection.cc:256] RPC call timeout handler was delayed by -1.0453088059998663s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
I20260709 12:25:44.583781  6304 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:44.593638  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:44.594563  6459 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:44.595858  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:44.596812  6452 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:44.597744  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:44.598471  6458 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:44.524004  8940 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.559700  8934 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.592026  8837 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.008s
I20260709 12:25:44.605954  6309 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:44.606527  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.662786  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.663560  6304 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:44.554474  8735 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.574120  8725 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:44.584731  8896 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.604056  8794 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:44.589820  8864 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:44.654752  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:44.655324  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:44.659293  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:44.688959  6458 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:44.669246  6301 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:25:44.678136  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.600538  8915 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:44.658402  8603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.325s	user 0.028s	sys 0.037s
I20260709 12:25:44.684785  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:44.658080  8881 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.170s	user 0.021s	sys 0.001s
I20260709 12:25:44.689759  6301 pending_rounds.cc:85] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 5
I20260709 12:25:44.690802  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:44.657220  8929 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.664592  8837 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.006s	sys 0.008s
I20260709 12:25:44.658684  8836 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:44.696848  6301 pending_rounds.cc:107] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:25:44.774914  6301 raft_consensus.cc:2889] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:44.706607  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.693800  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 250630us
I20260709 12:25:44.777189  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:44.694414  8603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.362s	user 0.029s	sys 0.037s
I20260709 12:25:44.728307  8881 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.779544  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.727152  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.795748  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.665302  8908 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:25:44.800488  7332 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:44.786366  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:44.682566  8904 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:44.691825  8363 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:44.771708  8892 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:44.796128  7331 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:44.796808  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:44.785919  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 131603us
I20260709 12:25:44.803900  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.813088  6304 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:44.816744  6455 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:44.794749  8734 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.815845  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:44.822515  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:44.793664  8817 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:44.829036  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:44.801127  8837 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:44.664389  8934 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.844642  8955 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:44.844491  8734 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.817852  8938 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:44.848428  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 131128us
I20260709 12:25:44.811590  8622 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:44.814178  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:44.779536  8836 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.781430  8929 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.695233  8864 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.847780  8617 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:44.867718  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:44.876986  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78403.8us
I20260709 12:25:44.721194  8915 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.830606  8947 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.837689  8789 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:44.886732  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:44.799873  8948 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:44.838126  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:44.871233  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:44.873792  8711 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.310s	user 0.006s	sys 0.010s
I20260709 12:25:44.866748  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:44.855373  8859 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:44.880292  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:44.882185  6384 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:44.847957  8817 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:44.885510  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:44.898298  6380 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:44.846823  8556 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:25:44.891621  6379 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:44.868036  8886 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:44.899811  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87928.3us
I20260709 12:25:44.877324  8957 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.901290  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:44.905572  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:44.907723  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:44.909510  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:44.963204  6379 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:44.964624  6379 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:44.877173  8617 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:44.860759  8837 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.876634  8946 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:44.883457  8956 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.890383  8791 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:44.956988  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60285.4us
I20260709 12:25:44.878053  8931 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:25:44.881922  7314 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:44.960294  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59102.3us
I20260709 12:25:44.911828  8908 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:44.980237  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74949.4us
I20260709 12:25:44.961014  6380 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:44.966866  6385 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:44.951951  8944 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:44.978816  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:44.894784  8774 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:25:44.986673  6376 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:44.957965  8949 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:44.990442  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:44.896221  8711 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.997975  6380 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:44.973202  8617 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:44.984587  8363 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:44.979568  8908 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:45.005069  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:44.994150  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.002810  8940 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.000725  8949 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:45.015928  8814 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.023128  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:45.024209  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:44.890161  8947 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:44.995296  8883 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:45.004877  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:45.037868  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47570.6us
I20260709 12:25:45.021791  6425 leader_election.cc:400] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4.
I20260709 12:25:45.038774  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:45.039460  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:44.988641  8735 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.020016  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.010147  8363 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.988215  8791 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.042055  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.038172  6380 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:45.010830  8881 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.017119  8908 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.046797  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:45.014816  8774 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:25:45.025815  8959 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:45.035573  8872 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.538s	user 0.004s	sys 0.007s
I20260709 12:25:45.036057  8572 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.043884  8859 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.133819  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91875.3us
I20260709 12:25:44.993113  8946 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.063862  8363 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.063448  8940 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:44.989177  8956 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.123972  8938 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:45.037578  8883 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:44.977977  8957 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.145254  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.151194  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.152868  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.154699  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:45.156242  6376 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.156980  6384 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:45.158090  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:45.160751  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.161504  6384 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:45.141464  8940 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.164212  6304 raft_consensus.cc:2410] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:45.141647  8881 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:45.165694  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1748) took 1562 ms. Trace:
I20260709 12:25:45.168079  6310 rpcz_store.cc:276] 0709 12:25:43.602998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.610801 (+  7803us) service_pool.cc:224] Handling call
0709 12:25:43.634937 (+ 24136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.466899 (+831962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.466948 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:44.467001 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:44.467045 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.165459 (+698414us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:45.165502 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.165624 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912759552}
I20260709 12:25:45.139336  8836 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.141250  8888 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:45.146018  8572 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:45.145749  8734 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.172782  6304 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:45.176095  6380 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:25:45.177743  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1571) took 2177 ms. Trace:
I20260709 12:25:45.150405  8622 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.175315  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.153198  8938 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:45.211148  6385 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:45.162884  8872 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.139338  8859 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.159204  8864 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.164709  8905 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.180s
I20260709 12:25:45.158964  8949 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.172224  8940 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:45.403116  8952 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.426s	user 0.003s	sys 0.000s
W20260709 12:25:45.280575  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1751) took 1669 ms (client timeout 1685 ms). Trace:
I20260709 12:25:45.414651  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.343597  8791 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.416218  8886 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:45.166790  8959 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:45.269057  8966 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:45.415818  8814 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:45.380774  6453 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:45.301517  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.425330  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.455472  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.410784  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.276449  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.382596  8947 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.378728  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.459062  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.305909  7329 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:45.172470  8881 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:45.206322  8973 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:45.302695  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.216120  8934 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.346167  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:45.381878  8969 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 12:25:45.289614  8954 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:25:45.171921  8888 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:25:45.382794  8961 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
W20260709 12:25:45.458585  8858 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.022s	user 0.009s	sys 0.006s
W20260709 12:25:45.276829  8965 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 12:25:45.443938  6303 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:45.460654  8791 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:45.205983  8962 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:45.484985  8843 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:45.485421  8962 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:45.485656  8843 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:45.485989  8962 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.487244  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1771) took 2344 ms. Trace:
I20260709 12:25:45.487807  6454 rpcz_store.cc:276] 0709 12:25:43.143225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.159584 (+ 16359us) service_pool.cc:224] Handling call
0709 12:25:43.187974 (+ 28390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:43.575548 (+387574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.486837 (+1911289us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b5400343854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:45.487111 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":829391616}
I20260709 12:25:45.292604  8891 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:45.489805  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.491395  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 546) took 1051 ms. Trace:
I20260709 12:25:45.491976  7319 rpcz_store.cc:276] 0709 12:25:44.440476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.441267 (+   791us) service_pool.cc:224] Handling call
0709 12:25:44.452202 (+ 10935us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.461005 (+  8803us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.491035 (+1030030us) spinlock_profiling.cc:243] Waited 5.12 ms on lock 0x7b540016dd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:45.491309 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11766016}
I20260709 12:25:45.212248  8972 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.449s
I20260709 12:25:45.262251  8572 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.184s
I20260709 12:25:45.275147  8837 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.289788  8963 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.213166  8915 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.467351  8958 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:25:45.269100  8887 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 2.573s	user 0.019s	sys 0.024s
I20260709 12:25:45.214664  6307 rpcz_store.cc:276] 0709 12:25:43.000834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.001407 (+   573us) service_pool.cc:224] Handling call
0709 12:25:45.177546 (+2176139us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54001b21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:45.177678 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088913152}
I20260709 12:25:45.172245  8735 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.263098  7288 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.048s (SENT)
I20260709 12:25:45.380653  8557 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:45.448081  6302 rpcz_store.cc:269] 0709 12:25:43.611345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.611552 (+   207us) service_pool.cc:224] Handling call
0709 12:25:45.280319 (+1668767us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001b3ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:45.280497 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":512520064}
I20260709 12:25:45.380151  8957 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.308786  8794 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.438927  8725 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.756s	user 0.006s	sys 0.006s
I20260709 12:25:45.526700  8978 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.447163  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.253645  6376 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
W20260709 12:25:45.451860  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 405117us
W20260709 12:25:45.267916  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102057us
I20260709 12:25:45.307591  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.457435  6451 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:45.536985  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:45.538655  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.539526  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.463996  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.540043  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:45.541097  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.460799  8881 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.449050  8949 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.468565  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.541884  6456 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:25:45.464448  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.465941  6455 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:45.466153  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.451215  8886 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:45.217082  8622 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.451730  8931 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:45.303807  8938 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.458325  8956 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.506884  8617 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.509042  8814 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:45.510270  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.206110  8953 tablet_replica.cc:333] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:25:45.543641  8989 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 12:25:45.573627  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63461.5us
I20260709 12:25:45.576139  8814 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:45.576536  8814 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:45.577119  8922 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:45.577569  8922 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:45.577374  8814 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.472108  8966 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:45.537930  8794 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.579938  6458 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.205750  8883 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:45.515621  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 252556us
I20260709 12:25:45.524576  8859 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.516085  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.463852  8982 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.482360  8791 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:45.450227  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.513957  8963 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.528561  8929 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.532383  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1579) took 1055 ms. Trace:
I20260709 12:25:45.527956  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.538702  6457 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:25:45.464955  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 239709us
W20260709 12:25:45.442598  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.001s	sys 0.009s
W20260709 12:25:45.610201  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.002s	sys 0.009s
I20260709 12:25:45.537923  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.609942  6450 raft_consensus.cc:2368] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6.
I20260709 12:25:45.608758  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.560750  8557 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.609113  6309 rpcz_store.cc:276] 0709 12:25:44.476558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.478145 (+  1587us) service_pool.cc:224] Handling call
0709 12:25:45.532180 (+1054035us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540010f254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:45.532326 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279551744}
W20260709 12:25:45.563038  8879 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:25:45.568959  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.570853  8622 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.572085  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.566226  8935 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.582981  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.567277  8931 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.543318  8878 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:45.580267  8892 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.599002  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.627660  7329 raft_consensus.cc:2410] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:25:45.610785  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 795246us
I20260709 12:25:45.611924  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.595633  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.562288  8572 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:45.616155  6458 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:45.599936  8588 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.608762  8987 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:45.619062  8859 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:45.568431  8978 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.624462  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:45.611831  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:45.625676  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56816.7us
I20260709 12:25:45.524816  8735 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.608208  8617 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.595150  8794 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.634949  8861 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:45.615509  8814 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.544205  8888 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.629846  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.577986  8908 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.657011  6301 pending_rounds.cc:85] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:25:45.657406  6301 pending_rounds.cc:107] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:25:45.640249  8859 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.628872  7332 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
W20260709 12:25:45.632696  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2090) took 2858 ms (client timeout 1829 ms). Trace:
I20260709 12:25:45.614900  8668 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:45.609409  8791 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:45.627669  8935 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.680384  6310 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:45.681630  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.677279  8859 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:45.685420  7316 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:45.637949  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.622475  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.579312  8966 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.623214  8937 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.639745  8945 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.736s	user 0.002s	sys 0.001s
I20260709 12:25:45.623965  8883 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:45.642149  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:45.627858  8944 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.634s	user 0.007s	sys 0.005s
I20260709 12:25:45.546943  8959 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:45.544500  8843 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.628542  6351 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6.
I20260709 12:25:45.619884  8557 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:45.583572  7314 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.576s	user 0.011s	sys 0.000s
I20260709 12:25:45.655144  8588 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.657675  6301 raft_consensus.cc:2889] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:45.647640  8572 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:45.658095  8878 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.619671  8982 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.658558  8617 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.644536  8987 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:45.664990  8908 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.667563  8817 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:45.671904  6377 rpcz_store.cc:269] 0709 12:25:42.769068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.770291 (+  1223us) service_pool.cc:224] Handling call
0709 12:25:45.627406 (+2857115us) spinlock_profiling.cc:243] Waited 621 ms on lock 0x7b54003d40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:45.627508 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1427779328}
I20260709 12:25:45.672497  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:45.673349  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.696733  8888 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:45.674731  6304 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:45.677858  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:45.678872  6308 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:45.682298  6310 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:45.686789  6450 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:45.687043  6454 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:45.687548  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 120536us
I20260709 12:25:45.630507  8363 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.694530  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.635792  8877 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:25:45.677518  8861 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:45.696916  6351 leader_election.cc:403] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:45.697467  8879 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.726430  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 97998.8us
I20260709 12:25:45.630174  8886 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.674031  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:45.709511  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.710093  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.727758  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.700067  8944 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.698846  8937 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.725976  6459 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:45.655553  8794 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:25:45.712072  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.689463  8668 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.727730  6305 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:45.700208  7314 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:45.741472  6452 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:45.742266  6450 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 1.
I20260709 12:25:45.750108  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.750907  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.727775  8929 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6.
I20260709 12:25:45.705672  8878 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.709569  8617 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.705768  8588 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.578428  8922 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.744010  8886 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.758327  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.757391  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.763792  6450 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:45.729753  8879 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.765898  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.759586  8944 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:45.770066  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.767737  8668 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.753623  8963 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.777733  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.776335  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.783268  6454 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:45.784158  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.784835  6454 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:45.785989  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.786823  6454 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:25:45.768317  8794 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:45.790970  8936 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.768594  8898 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:45.794796  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.771086  8878 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:45.798542  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.796979  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.799041  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.799624  6450 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:45.800496  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.806979  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.807904  6302 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:45.806423  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:45.808410  6308 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.805756  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.809536  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.811194  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.749670  8962 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.812124  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.767370  8987 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.770308  8881 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.813946  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.814126  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.798512  8946 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:45.814189  8915 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:25:45.812722  6450 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:45.795226  8944 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:45.813764  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.812739  6454 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:45.817008  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.832468  6309 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:45.832774  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.827226  7319 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:45.828567  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.833606  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.768193  8861 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.786518  8957 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:25:45.834116  7332 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:45.834173  6309 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:45.808600  8963 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:45.830436  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.811394  8668 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:25:45.815955  8835 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:45.838353  8982 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.862432  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.859751  6309 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:45.820636  8858 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:45.862391  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.863054  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.831758  8915 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:45.858271  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 143945us
I20260709 12:25:45.873497  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.869197  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:45.822064  8557 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.874475  8883 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:45.873350  8982 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:45.874943  8883 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.875391  8883 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:45.869100  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.876456  6302 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:45.866124  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.885244  8883 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.887321  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.887394  8982 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:45.857403  8946 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:45.860052  8998 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:25:45.889287  6450 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.834187  8931 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.835762  8936 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:45.846259  8937 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.919196  8859 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.921690  8859 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.894591  8879 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.861994  8835 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.896171  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.897969  6300 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:45.898257  8881 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.931839  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 391) took 2723 ms. Trace:
I20260709 12:25:45.898903  8982 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:45.900013  8962 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.900723  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:45.932813  6377 raft_consensus.cc:606] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.933317  6377 raft_consensus.cc:614] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:45.871459  8363 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.892927  8946 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:45.903154  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:45.905087  8939 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:45.939677  8939 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:45.939548  8791 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:45.889366  8915 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:45.889192  8858 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.909143  8944 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:45.909832  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.913813  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.895474  8588 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:45.890684  8959 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:45.926270  6308 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
W20260709 12:25:45.927376  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 102645us
I20260709 12:25:45.930364  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:45.934667  6453 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:25:45.932384  7317 rpcz_store.cc:276] 0709 12:25:43.208851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.213274 (+  4423us) service_pool.cc:224] Handling call
0709 12:25:44.424577 (+1211303us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:25:44.424814 (+   237us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:44.449671 (+ 24857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:44.449706 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:44.455058 (+  5352us) log.cc:844] Serialized 52 byte log entry
0709 12:25:45.669768 (+1214710us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b1800387be0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:25:45.705619 (+ 35851us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:45.927955 (+222336us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.930629 (+  2674us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.930897 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.931003 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:45.931077 (+    74us) raft_consensus.cc:1685] finished
0709 12:25:45.931093 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:45.931776 (+   683us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14592,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":3477,"spinlock_wait_cycles":29474816,"thread_start_us":231548,"threads_started":2,"wal-append.queue_time_us":1322056}
I20260709 12:25:45.933599  8835 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.933840  8982 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:45.938673  8946 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:45.954423  7335 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:45.955375  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:45.956946  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:45.951665  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.958020  6380 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:45.958689  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:45.943372  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 248905us
I20260709 12:25:45.945102  6455 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:45.950453  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:45.950703  6376 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:25:45.951211  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.966457  8879 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:45.964747  8668 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:25:45.964661  8588 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.965094  8883 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:45.951135  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:45.971426  6382 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:45.961721  6385 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:45.959510  7285 leader_election.cc:336] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.607s (SENT)
I20260709 12:25:45.974828  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.962689  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:45.960983  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.958317  6450 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:45.960543  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.977028  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.942353  8940 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:45.981336  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:45.984340  6383 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:45.986419  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.986635  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.987030  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:45.990340  6450 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:45.964037  8861 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:45.992347  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.994452  6377 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:45.987741  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:45.967057  8879 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:45.979125  8943 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.000859  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:46.001816  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.002319  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.002954  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.003098  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.004529  6382 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:46.005457  6376 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:46.006568  8931 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:46.001367  8814 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.008488  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:45.994453  8982 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.009501  8982 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:46.003765  8879 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:46.010413  6384 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:46.010810  6450 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:45.996420  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.013074  6384 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.012590  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:46.026321  6450 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:45.989693  8873 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.027477  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.027756  6384 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:46.034384  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.035615  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.036999  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:45.995484  8889 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:46.003147  8883 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.007543  8940 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.047317  6307 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:46.050644  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:46.053663  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.054124  6377 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.054374  6376 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:46.055836  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.056321  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.056486  6384 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:46.073850  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.055716  8889 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:46.074700  8891 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:25:45.996152  8939 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.032766  8937 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.080338  6385 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
W20260709 12:25:46.077879  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.052033  9002 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:46.090785  8588 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.076035  8908 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:25:46.047062  8943 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.093464  8943 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.094403  8943 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.096117  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.097303  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.051350  8949 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:46.098831  8949 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.099344  8949 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:46.100171  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 268) took 2988 ms. Trace:
I20260709 12:25:46.100587  6383 rpcz_store.cc:276] 0709 12:25:43.111265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.111437 (+   172us) service_pool.cc:224] Handling call
0709 12:25:43.111801 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.986447 (+2874646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.099895 (+113448us) spinlock_profiling.cc:243] Waited 2.87 s on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:46.100077 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6605752064}
I20260709 12:25:46.084933  8858 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.103049  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 475) took 1323 ms. Trace:
I20260709 12:25:46.103602  6379 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:46.104511  7329 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.103446  7314 rpcz_store.cc:276] 0709 12:25:44.779815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.786970 (+  7155us) service_pool.cc:224] Handling call
0709 12:25:46.102867 (+1315897us) spinlock_profiling.cc:243] Waited 9.51 ms on lock 0x7b54004b8a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:46.102986 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":21870848}
I20260709 12:25:46.073201  8791 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.107329  8843 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.107709  9002 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:46.108134  9002 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:46.108963  9002 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.111075  6455 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:46.087471  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:46.112576  6456 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:46.085131  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:46.115278  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.115867  6380 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:46.116647  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.098351  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.117956  6452 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:46.119386  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.122624  6450 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:46.112118  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.124109  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:46.125667  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.088846  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 552) took 1071 ms. Trace:
I20260709 12:25:46.126442  7331 rpcz_store.cc:276] 0709 12:25:45.017457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.017680 (+   223us) service_pool.cc:224] Handling call
0709 12:25:46.088560 (+1070880us) spinlock_profiling.cc:243] Waited 544 ms on lock 0x7b5400a00ba8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa824bebc 00007f6fa828108c 00007f6fa826b93b 00007f6fa826b3d2 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c
0709 12:25:46.088781 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252014848}
I20260709 12:25:46.093405  8835 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.031278  9007 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:25:46.131876  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.128306  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.129034  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.052810  8814 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.129259  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.053166  8881 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.129714  6450 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:46.127846  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:46.131403  6377 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:46.134912  6456 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:46.072700  8861 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:25:46.179397  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.180050  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.180825  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.181020  6382 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.140226  6385 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:46.137064  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:46.178967  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.080453  8925 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.188616  6456 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:46.072767  8896 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:25:46.191365  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54406.7us
W20260709 12:25:46.215197  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.193565  8925 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.196491  9002 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.134615  8908 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:46.183014  8964 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.191110  8878 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.176455  8897 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:46.038610  8959 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.193161  8857 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:25:46.128985  8940 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:46.246245  8985 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:25:46.271464  9008 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.218s	user 0.001s	sys 0.000s
W20260709 12:25:46.259931  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64120.1us
I20260709 12:25:46.132175  8939 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:46.140554  8836 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:46.140287  8891 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.129860  8883 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.266395  6456 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:46.267390  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.300949  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:46.192586  8896 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.198060  8794 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:46.261786  8725 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:25:46.178980  8955 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:25:46.309734  8925 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.310724  8985 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:46.310554  8861 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.311697  8985 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.266551  8889 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.312327  8944 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:46.305327  8896 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:46.262707  8966 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:46.194532  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.314850  8966 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.315290  8966 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:46.315420  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 121021us
W20260709 12:25:46.290630  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.295421  8835 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.316452  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.297436  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.301174  6382 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:46.135229  8864 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:46.318359  8641 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:46.319092  8641 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:46.319782  8883 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.320475  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.321555  8864 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:46.321705  8725 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:46.303928  8836 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:46.322642  8864 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.323336  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.324589  8725 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.283200  8363 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.325708  6455 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:46.326066  8363 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.310159  8915 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.328022  6455 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:46.328655  8363 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.304916  8794 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:46.328963  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.320413  8883 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.329280  6385 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 1. Current term is 2. Ops: []
I20260709 12:25:46.130927  8943 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.304090  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:46.288123  8897 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.286818  8879 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.335155  6455 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:46.296665  8940 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:46.350430  6455 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:46.337648  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.344832  6456 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.276459  8964 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.283694  8878 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.358068  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.358469  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.359537  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:46.323462  8836 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.362905  6459 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:46.362933  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:46.363296  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.361081  8879 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:46.377264  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85153.3us
I20260709 12:25:46.341459  8794 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.371233  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:46.367980  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.368443  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.369288  8925 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:46.370141  6310 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:46.397444  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.370409  8949 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.360812  8941 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:46.347855  8791 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.372171  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:46.378276  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.350117  9012 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:46.342118  8725 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.381722  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:46.398906  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.400090  6305 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:46.370615  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:46.401067  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:46.359575  8711 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.367393  8944 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.411087  7331 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:46.411384  8944 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.402661  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:46.411715  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.402755  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:46.412744  6305 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:46.341956  8877 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:46.399519  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:46.413671  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:46.410400  6453 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.413854  6310 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.414530  6309 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.391698  8785 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:46.415200  6459 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.404620  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.416923  6308 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:46.417008  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:46.405304  8879 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:46.403534  8603 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:46.434201  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.405121  8957 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:46.458781  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:46.431903  8711 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 12:25:46.407042  8886 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:46.467657  8962 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.468219  8735 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.433143  8785 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.469347  8886 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:46.469782  8886 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.470167  8886 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.415302  8944 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:46.475438  8943 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.478585  8883 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.415311  8588 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.468490  8962 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:46.482681  8963 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.509960  8963 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.512207  8963 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:46.467717  9011 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:25:46.516232  8982 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.506s	user 0.004s	sys 0.006s
I20260709 12:25:46.500198  8964 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.501367  8897 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:46.517084  8966 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.430753  8925 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:46.517894  8925 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.518338  8925 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.521790  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:46.522605  6305 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.409037  8791 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.444986  8835 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:46.523972  6301 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:46.483209  6459 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 12:25:46.545902  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87248.1us
I20260709 12:25:46.547904  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:46.549666  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:46.594060  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.653823  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:46.430101  8814 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.093s
W20260709 12:25:46.656307  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 105089us
W20260709 12:25:46.674626  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 257679us
W20260709 12:25:46.685298  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.690121  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 271893us
I20260709 12:25:46.692322  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:46.695531  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.697010  6309 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:46.703011  8966 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.703552  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:46.703657  8813 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:46.704216  6302 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.704371  8813 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:46.655714  8963 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:46.711889  8791 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.712316  8915 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.713682  6310 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:46.705593  8813 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.715436  8813 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.717896  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.718806  6310 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:46.719475  8878 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.719535  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.722098  6310 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:46.723181  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:46.723831  6310 raft_consensus.cc:2376] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5. Re-sending same reply.
I20260709 12:25:46.724983  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:46.723505  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.726989  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:46.727878  6378 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:46.729068  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.683930  8946 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.697s	user 0.013s	sys 0.005s
I20260709 12:25:46.732000  6310 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:46.732422  6305 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:25:46.656076  8897 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.720561  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.657160  8936 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:46.655503  8962 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:46.744940  6310 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.745285  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.720441  8878 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:46.736202  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:46.747051  6459 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:46.730273  8963 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:25:46.746930  8937 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:46.735843  6456 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:46.748943  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.751330  8946 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:46.755311  9017 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:46.440645  8940 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.449587  8603 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.762887  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.766966  6305 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 2.
I20260709 12:25:46.730502  8904 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:46.779536  6452 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:46.791465  6305 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.777383  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.695029  8588 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:25:46.785753  6302 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:46.766263  8879 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:46.772701  8936 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:46.795717  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.801786  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.785267  8937 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.803194  9024 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:46.806228  8603 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.803279  8735 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:25:46.783324  9002 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:46.766701  8897 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.823341  8931 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:46.825973  8897 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.828088  8785 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:46.817334  6305 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:46.799875  8963 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:46.834081  6459 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.837167  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.798760  8864 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.811909  8861 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:46.846971  8861 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:46.819619  8794 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.842671  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:46.844579  6310 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:25:46.853271  6310 pending_rounds.cc:85] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 3
I20260709 12:25:46.853644  6310 pending_rounds.cc:107] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:25:46.854127  6310 raft_consensus.cc:2889] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:46.845036  8891 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:25:46.849534  8931 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:46.857057  8931 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.857359  8891 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:46.857640  8931 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:46.858058  8891 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:46.861605  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.863611  6459 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:46.832815  9002 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:46.851344  8963 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:46.838081  8872 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:25:46.883677  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:46.889853  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 230633us
I20260709 12:25:46.867960  9002 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:46.875010  8995 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:46.898187  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:46.902779  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:46.901661  8995 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:46.836375  8925 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.891906  8794 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:46.906658  8785 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:25:46.917223  8915 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:25:46.917758  9021 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:25:46.918221  8963 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.931097  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:46.859234  8864 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.932492  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:46.848037  8861 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.920503  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:46.926986  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.935626  6309 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:46.916143  8957 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:46.928258  9026 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:25:46.914928  8939 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.614s	user 0.007s	sys 0.006s
I20260709 12:25:46.930320  6302 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:46.899031  8962 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:25:46.948689  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.949512  6309 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:46.876511  8936 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:46.951457  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:46.928588  8995 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:46.952608  7329 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:46.933946  8915 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:46.953173  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2217) took 1547 ms. Trace:
I20260709 12:25:46.956821  6452 rpcz_store.cc:276] 0709 12:25:45.406438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.136311 (+729873us) service_pool.cc:224] Handling call
0709 12:25:46.136711 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.790710 (+653999us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.952887 (+162177us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54002c7c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:46.953104 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465225088}
I20260709 12:25:46.958609  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.968438  6452 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:46.969362  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 253) took 1060 ms. Trace:
I20260709 12:25:46.969802  6452 rpcz_store.cc:276] 0709 12:25:45.908775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.967770 (+1058995us) service_pool.cc:224] Handling call
0709 12:25:46.968191 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.969031 (+   840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.969284 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.972523  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 254) took 1063 ms. Trace:
I20260709 12:25:46.972996  6452 rpcz_store.cc:276] 0709 12:25:45.909043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.971185 (+1062142us) service_pool.cc:224] Handling call
0709 12:25:46.971842 (+   657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.972069 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.972111 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.972173 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:46.972223 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.972298 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.972441 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:46.975473  6384 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:46.977407  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:46.978401  6302 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:46.951177  8946 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:46.980525  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:46.956437  8915 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:46.976964  8896 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.000s
I20260709 12:25:46.993685  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:46.961228  8872 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:25:46.980126  8864 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:46.997186  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:46.992012  8908 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.002048  6383 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.003314  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:47.002934  8944 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.006s
I20260709 12:25:46.998778  8946 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:25:47.012305  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.013978  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2222) took 1605 ms (client timeout 1752 ms). Trace:
I20260709 12:25:47.014319  6455 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:25:47.015570  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.015574  6458 rpcz_store.cc:269] 0709 12:25:45.408539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.414333 (+  5794us) service_pool.cc:224] Handling call
0709 12:25:47.013795 (+1599462us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b5400344254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.013917 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303570176}
W20260709 12:25:47.016445  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.004521  8908 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.012988  8789 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:46.998951  8785 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:25:47.021291  8813 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.034343  8813 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.037353  8813 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.033686  8789 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:47.021827  8943 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:47.038831  8943 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:47.039786  8943 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.039394  8789 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.024209  8773 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:46.995335  8896 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.041515  8773 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:47.042506  8773 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.043247  8896 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.043511  8773 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:47.030112  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.022725  8960 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:47.040905  8617 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.046885  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:47.055064  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:47.056313  6380 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:47.057480  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:47.007961  8940 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.015159  8946 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.060648  8946 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.017057  8944 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:47.065177  9035 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:47.067085  6382 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:47.068311  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:47.069487  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:47.070197  6382 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:47.072808  8944 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:47.075289  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:47.076834  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:47.083436  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.065243  8789 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.078351  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:47.058667  8886 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.001s
I20260709 12:25:47.008765  8915 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:47.064095  9031 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:25:47.092342  8813 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:25:47.103917  8813 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:47.104434  8813 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:47.063581  8962 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.108009  8864 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.108577  8864 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.108963  8735 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:47.111346  8896 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:47.111555  8711 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:47.112097  8896 consensus_queue.cc:1243] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:47.112165  8711 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:47.114182  8957 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:47.114825  8957 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:47.115586  8925 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.116063  8978 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.064455  8872 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:47.117369  8872 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.117799  8987 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.117884  8872 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.122248  8886 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.125243  8886 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.126302  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1801) took 1225 ms. Trace:
I20260709 12:25:47.126791  6456 rpcz_store.cc:276] 0709 12:25:45.900544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.783504 (+882960us) service_pool.cc:224] Handling call
0709 12:25:46.783963 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.797361 (+ 13398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.797393 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.797882 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.942581 (+144699us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:46.942631 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.942713 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.125984 (+183271us) raft_consensus.cc:1685] finished
0709 12:25:47.126026 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.126220 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12911,"raft.queue_time_us":13865,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":12799,"thread_start_us":144357,"threads_started":1,"wal-append.queue_time_us":188471}
I20260709 12:25:47.142261  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 277) took 1015 ms. Trace:
I20260709 12:25:47.130329  8941 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:47.145597  6383 rpcz_store.cc:276] 0709 12:25:46.126881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.127587 (+   706us) service_pool.cc:224] Handling call
0709 12:25:46.185010 (+ 57423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.002621 (+817611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.134295 (+131674us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.142135 (+  7840us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878489856}
I20260709 12:25:47.152822  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:47.153462  6302 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:47.157371  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:47.147225  8941 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:47.151039  8915 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.165037  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:47.155799  8864 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.168219  6459 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.167546  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:47.169303  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:47.170220  6382 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:47.105399  8813 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:47.172029  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1595) took 1498 ms (client timeout 1912 ms). Trace:
W20260709 12:25:47.176425  6300 rpcz_store.cc:269] 0709 12:25:45.673629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.673756 (+   127us) service_pool.cc:224] Handling call
0709 12:25:47.171888 (+1498132us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b5400140d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.171985 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391595264}
I20260709 12:25:47.177042  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.177913  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:47.178113  6450 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.178735  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2218) took 1772 ms. Trace:
I20260709 12:25:47.179114  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:47.179061  6450 rpcz_store.cc:276] 0709 12:25:45.406889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.138232 (+731343us) service_pool.cc:224] Handling call
0709 12:25:46.138630 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.177972 (+1039342us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b54000cdb20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa321edc0 00007f6fa32a91c0 00007f6fa32aa231 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c
0709 12:25:47.178547 (+   575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.178691 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98467712}
I20260709 12:25:47.180368  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.182032  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.182832  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.183904  8891 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:47.187831  7316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 387) took 4722 ms (client timeout 2116 ms). Trace:
I20260709 12:25:47.180379  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:47.189438  6310 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:47.189927  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2241) took 1546 ms. Trace:
I20260709 12:25:47.184252  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:47.188380  7316 rpcz_store.cc:269] 0709 12:25:42.465332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.468514 (+  3182us) service_pool.cc:224] Handling call
0709 12:25:43.699482 (+1230968us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182eee 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:47.187527 (+3488045us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003bcc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.187700 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2841182336}
I20260709 12:25:47.190589  8937 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.182957  6450 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.191919  7317 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:47.192788  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 400) took 1364 ms. Trace:
I20260709 12:25:47.174119  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:47.191092  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 286) took 2720 ms. Trace:
I20260709 12:25:47.197824  6300 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:47.198300  6303 rpcz_store.cc:276] 0709 12:25:44.470595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.470738 (+   143us) service_pool.cc:224] Handling call
0709 12:25:44.471049 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.476055 (+1005006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.190831 (+1714776us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b54002d23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.191027 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236867456}
I20260709 12:25:47.200487  6303 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:47.202678  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:47.203346  6302 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.204622  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.206920  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:47.207818  6381 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:25:47.208469  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1683) took 1259 ms (client timeout 1657 ms). Trace:
W20260709 12:25:47.208956  6381 rpcz_store.cc:269] 0709 12:25:45.949470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.949682 (+   212us) service_pool.cc:224] Handling call
0709 12:25:47.206708 (+1257026us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:47.208395 (+  1687us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2417920}
I20260709 12:25:47.195180  7335 rpcz_store.cc:276] 0709 12:25:45.828480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.830074 (+  1594us) service_pool.cc:224] Handling call
0709 12:25:47.192606 (+1362532us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.192721 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222514944}
I20260709 12:25:47.190398  6455 rpcz_store.cc:276] 0709 12:25:45.643036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.351762 (+708726us) service_pool.cc:224] Handling call
0709 12:25:47.013956 (+662194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.015701 (+  1745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.189681 (+173980us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b5400344250. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.189847 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522145024}
I20260709 12:25:47.212750  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.214754  6381 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:47.216563  6381 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:47.218070  6381 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:47.226042  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.229089  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.175956  8861 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:47.222463  8966 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:47.226899  8859 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:47.235955  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:47.239347  6275 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:25:47.202521  8951 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.227072  8498 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.242398  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.240904  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.245497  7319 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:47.246610  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.249426  6302 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:47.250432  6302 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:47.257416  8498 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.193648  9002 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.260273  9002 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.227649  8886 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:25:47.261488  7316 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:25:47.262049  7316 pending_rounds.cc:85] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 6
I20260709 12:25:47.243048  8859 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:47.260360  6452 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:47.262995  8904 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.231009  8941 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.263935  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 255) took 1354 ms. Trace:
I20260709 12:25:47.236770  8861 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.263697  8859 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.264333  6452 rpcz_store.cc:276] 0709 12:25:45.909293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.974202 (+1064909us) service_pool.cc:224] Handling call
0709 12:25:46.974634 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.263153 (+288519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.263714 (+   561us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002c7754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.263877 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656688384}
I20260709 12:25:47.265691  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.267175  8931 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.262394  7316 pending_rounds.cc:107] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:25:47.268738  7316 raft_consensus.cc:2889] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.269229  8931 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.269965  6458 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.238657  8813 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.271027  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1804) took 1259 ms. Trace:
I20260709 12:25:47.271375  6458 rpcz_store.cc:276] 0709 12:25:46.011531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.019644 (+1008113us) service_pool.cc:224] Handling call
0709 12:25:47.020016 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.270321 (+250305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.270823 (+   502us) spinlock_profiling.cc:243] Waited 250 ms on lock 0x7b54000c12d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.270976 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":574373248}
I20260709 12:25:47.272507  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.272555  8946 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.273152  6458 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.274755  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.262387  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.276419  6451 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:47.275511  6452 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:47.276643  6383 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:47.264472  8861 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.281188  6450 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:25:47.283118  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:47.284827  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.296236  6450 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:47.298187  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:47.299927  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.301123  6450 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 3: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:47.306061  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.260605  8498 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:47.319231  6304 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.320502  8953 raft_consensus.cc:2243] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Raft consensus shutting down.
I20260709 12:25:47.321355  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2220) took 1913 ms. Trace:
I20260709 12:25:47.321691  6304 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:47.323549  6304 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.323823  8922 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.324138  6381 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:47.321748  6453 rpcz_store.cc:276] 0709 12:25:45.407667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.178168 (+770501us) service_pool.cc:224] Handling call
0709 12:25:46.178572 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.414359 (+235787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.321150 (+906791us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54000eb354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.321298 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532558720}
I20260709 12:25:47.328276  8960 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:47.332710  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.333436  6451 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.334009  8946 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.335501  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.318142  8951 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:47.346558  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.346954  8951 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:47.347186  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 1794) took 1654 ms (client timeout 2163 ms). Trace:
I20260709 12:25:47.347687  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:47.347906  6454 rpcz_store.cc:269] 0709 12:25:45.692750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.798782 (+106032us) service_pool.cc:224] Handling call
0709 12:25:47.346836 (+1548054us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b5400265f54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.347083 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29629824}
W20260709 12:25:47.349450  9038 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:25:47.350066  6454 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.240274  8966 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:47.352380  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.352723  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47111.7us
I20260709 12:25:47.354992  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2246) took 1299 ms. Trace:
I20260709 12:25:47.356117  6454 rpcz_store.cc:276] 0709 12:25:46.055190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.349393 (+1294203us) service_pool.cc:224] Handling call
0709 12:25:47.349813 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.354637 (+  4824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.354913 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.315625  8886 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:47.359797  6454 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.360668  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.364902  9023 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:47.365365  9023 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:47.369782  6451 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:25:47.370332  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.370519  6458 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:25:47.371168  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.371687  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1806) took 1294 ms. Trace:
I20260709 12:25:47.373029  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.373328  6454 rpcz_store.cc:276] 0709 12:25:46.077494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.358638 (+1281144us) service_pool.cc:224] Handling call
0709 12:25:47.359173 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.360259 (+  1086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.371624 (+ 11365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.378662  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.379367  8498 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:25:47.381554  8498 consensus_queue.cc:1243] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:25:47.381999  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.382658  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.383062  6454 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.367758  8908 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.370699  8953 raft_consensus.cc:2272] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEARNER]: Raft consensus is shut down!
I20260709 12:25:47.373733  6458 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.372979  8886 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:47.378973  8937 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.389398  8797 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:47.398725  6455 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:25:47.386929  8962 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.398636  8797 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:47.402113  8949 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:47.404278  8943 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.347849  8951 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.407572  6376 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.409819  6380 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:47.410333  6451 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:47.411988  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1776) took 1048 ms. Trace:
I20260709 12:25:47.366283  9023 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.422996  6301 rpcz_store.cc:276] 0709 12:25:46.363437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.364138 (+   701us) service_pool.cc:224] Handling call
0709 12:25:46.370017 (+  5879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.540204 (+170187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.411744 (+871540us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b540010f754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.411924 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353430272}
I20260709 12:25:47.424216  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2247) took 1132 ms. Trace:
I20260709 12:25:47.427817  6451 rpcz_store.cc:276] 0709 12:25:46.291438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.393208 (+1101770us) service_pool.cc:224] Handling call
0709 12:25:47.399742 (+  6534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.423908 (+ 24166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.424157 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.420151  6454 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:47.431577  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:47.432163  8797 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.435678  8966 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.435693  8797 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:47.436612  9042 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.086s	user 0.003s	sys 0.000s
I20260709 12:25:47.438980  8939 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:25:47.439474  8939 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:47.439657  8908 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.440442  8939 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.441766  8939 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:25:47.443051  9023 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.448029  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:47.448235  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:47.424191  6458 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:47.449294  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:47.449874  6304 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:47.450717  6305 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:47.460906  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.461184  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.461346  8953 ts_tablet_manager.cc:1916] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:25:47.461551  7319 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:47.462519  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.463057  7316 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.463402  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2483d7801887403e873adc62429fe046"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:25:47.464145  6300 raft_consensus.cc:606] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.465059  7335 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:47.468712  6455 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:47.469179  7316 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:47.470863  8872 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.472041  6453 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:47.474123  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.475274  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34bae875d7104f298673c223da6caf2e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:47.475854  6385 raft_consensus.cc:606] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.476164  6385 raft_consensus.cc:614] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:47.477722  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.477725  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.480199  6454 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:47.480400  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.483119  8603 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.483762  8603 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.486423  8603 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:47.488101  8861 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.491227  8861 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.492583  6300 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.499598  6451 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:47.458128  6301 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:25:47.500337  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 258) took 1165 ms. Trace:
I20260709 12:25:47.500535  6301 pending_rounds.cc:85] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 5
I20260709 12:25:47.500912  6301 pending_rounds.cc:107] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:25:47.501242  6301 raft_consensus.cc:2889] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:47.450734  8886 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.459786  8966 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.493607  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.495785  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:47.496411  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.506903  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.452340  6458 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:25:47.500715  6451 rpcz_store.cc:276] 0709 12:25:46.335367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.434072 (+1098705us) service_pool.cc:224] Handling call
0709 12:25:47.499346 (+ 65274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.500059 (+   713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.500278 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.509091  6451 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:47.501052  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:47.512421  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.513172  6455 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:25:47.514729  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1808) took 1178 ms. Trace:
I20260709 12:25:47.512291  8966 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.512327  8948 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:47.516228  8603 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:47.545593  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.515307  6458 rpcz_store.cc:276] 0709 12:25:46.336478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.450683 (+1114205us) service_pool.cc:224] Handling call
0709 12:25:47.451118 (+   435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.514343 (+ 63225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.514654 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:47.504204  8949 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.562145  6300 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:47.586048  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:47.615885  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.615132  6451 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:47.621654  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.516633  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 397) took 1692 ms. Trace:
I20260709 12:25:47.535759  8622 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:47.623365  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 176857us
I20260709 12:25:47.624696  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.611536  8588 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:47.623822  8948 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:25:47.634027  8588 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.634541  8588 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.549991  8925 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.626194  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:47.647476  6385 raft_consensus.cc:606] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.647811  6385 raft_consensus.cc:614] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:47.629833  6304 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:25:47.626693  7318 rpcz_store.cc:276] 0709 12:25:45.824661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.827518 (+  2857us) service_pool.cc:224] Handling call
0709 12:25:45.827763 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.176727 (+348964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.183205 (+  6478us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.193217 (+ 10012us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:46.196518 (+  3301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.502024 (+1305506us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400508554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.503727 (+  1703us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.514443 (+ 10716us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25264,"raft.queue_time_us":51299,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14931,"spinlock_wait_cycles":639607168}
I20260709 12:25:47.639220  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.639302  7317 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:47.649159  7317 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:47.613091  9026 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:47.690981  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:47.630775  8622 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.678879  8859 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.655100  7317 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.644500  8861 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.692971  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1697) took 1326 ms (client timeout 1512 ms). Trace:
I20260709 12:25:47.644713  9007 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.693830  9007 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.693161  7317 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.696074  9007 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.692543  8622 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:47.693164  8861 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.699124  8861 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.699347  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.700289  7320 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:25:47.701927  6377 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:47.702294  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 256) took 1792 ms. Trace:
I20260709 12:25:47.702728  6459 rpcz_store.cc:276] 0709 12:25:45.909554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.981807 (+1072253us) service_pool.cc:224] Handling call
0709 12:25:46.985695 (+  3888us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.168594 (+182899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.702051 (+533457us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54000c21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.702223 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419327744}
W20260709 12:25:47.705178  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 80550.3us
I20260709 12:25:47.706530  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:47.707432  7317 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:47.631454  8949 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:47.643858  8883 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.619473  8603 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.655104  8985 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.751428  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:47.752024  6385 raft_consensus.cc:606] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.755112  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:47.755613  6385 raft_consensus.cc:606] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.755914  6385 raft_consensus.cc:614] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:47.757249  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:25:47.765262  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 229693us
I20260709 12:25:47.744369  8951 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:47.715456  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:47.773494  6304 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:47.693569  6378 rpcz_store.cc:269] 0709 12:25:46.368037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.368188 (+   151us) service_pool.cc:224] Handling call
0709 12:25:47.692705 (+1324517us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54001955d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.692892 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745425280}
I20260709 12:25:47.691973  6456 raft_consensus.cc:2376] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
W20260709 12:25:47.726629  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 72581us
I20260709 12:25:47.776305  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:47.709496  6300 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:47.770802  6457 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:47.749934  9026 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:47.771823  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.787660  8966 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:47.808857  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.810174  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.811241  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:47.813230  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:47.814708  9047 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:25:47.818581  8603 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:47.819579  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.779292  8905 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.820451  8905 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.778880  8949 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.044s
I20260709 12:25:47.823119  8949 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:47.825107  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.771373  8617 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:25:47.825861  8617 consensus_queue.cc:1243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:47.827375  8617 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:47.828047  6385 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:47.829290  8622 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.765717  8985 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.831158  6450 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:47.831755  8960 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:47.832198  8960 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:47.832341  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1591) took 2160 ms (client timeout 1635 ms). Trace:
I20260709 12:25:47.832634  8985 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.791802  8569 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:47.833673  8569 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:47.834154  8569 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:47.834919  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:47.832805  6307 rpcz_store.cc:269] 0709 12:25:45.672348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.672467 (+   119us) service_pool.cc:224] Handling call
0709 12:25:47.832174 (+2159707us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b54002de1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.832278 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":632549760}
I20260709 12:25:47.839715  6385 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:47.842957  6304 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:47.848933  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:47.849792  6452 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
W20260709 12:25:47.850713  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.834100  6308 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:25:47.851857  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.855793  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 149356us
I20260709 12:25:47.856752  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:47.857390  6378 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:25:47.760052  8883 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.855858  6380 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:47.860752  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:47.828063  8617 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:47.785724  9026 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:47.865495  6383 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:25:47.869530  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.798532  8872 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.870397  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:47.856680  8939 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:47.791698  9007 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 12:25:47.855087  8773 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:47.880043  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.836299  8711 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.793291  8859 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:25:47.871809  9026 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:47.923321  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.872605  8978 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:47.870316  8789 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:47.878568  8603 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:47.918936  8943 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.870535  8972 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:25:47.940869  6301 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:47.872190  8889 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:25:47.886257  9033 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:47.943382  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.881027  8946 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:25:47.882262  8738 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:25:47.883836  8588 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.934964  8603 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:47.940457  8711 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:47.940677  8835 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:25:47.955245  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 168937us
I20260709 12:25:47.921124  8773 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.947945  8617 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.968794  8905 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:47.969627  8668 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.970274  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2114) took 1894 ms. Trace:
I20260709 12:25:47.971053  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 275) took 1845 ms. Trace:
I20260709 12:25:47.970274  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1619) took 1102 ms. Trace:
I20260709 12:25:48.012755  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:47.948342  8797 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:47.948637  9033 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:47.937033  8978 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:47.941020  8943 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:47.938131  8789 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:47.982018  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 170831us
W20260709 12:25:47.995950  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 116020us
W20260709 12:25:48.023429  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148920us
I20260709 12:25:48.024228  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:48.025476  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:48.027086  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:47.961640  8889 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:48.000605  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:48.027954  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:48.028405  7335 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:48.001670  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.012607  6376 rpcz_store.cc:276] 0709 12:25:46.075810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.110389 (+ 34579us) service_pool.cc:224] Handling call
0709 12:25:46.111157 (+   768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.418009 (+1306852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.970090 (+552081us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b540023ebd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.970216 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2981068032}
I20260709 12:25:48.013213  6308 rpcz_store.cc:276] 0709 12:25:46.868279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.868439 (+   160us) service_pool.cc:224] Handling call
0709 12:25:47.833794 (+965355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.851436 (+ 17642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.970090 (+118654us) spinlock_profiling.cc:243] Waited 964 ms on lock 0x7b54002de1d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:47.970214 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2216714496}
I20260709 12:25:48.013005  6384 rpcz_store.cc:276] 0709 12:25:46.126105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.126352 (+   247us) service_pool.cc:224] Handling call
0709 12:25:46.147053 (+ 20701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:46.976188 (+829135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.970839 (+994651us) spinlock_profiling.cc:243] Waited 828 ms on lock 0x7b54003e4ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:47.971001 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1904506112}
I20260709 12:25:48.013655  7332 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.035107  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:48.040669  6456 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:25:47.974416  8622 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:47.978132  8603 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.045652  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:48.046526  6456 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:48.049556  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:48.053743  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 263957us
I20260709 12:25:48.060814  7320 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.061379  7320 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.021840  8905 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:48.023175  9033 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.043536  8835 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:47.997574  8569 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.043382  8889 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.043251  8962 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:48.096813  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:48.097014  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69993.3us
I20260709 12:25:48.077270  7332 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:48.011677  8922 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:25:48.103767  6304 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:48.107537  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.108857  6300 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:48.018925  8617 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.022029  8668 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.132822  6300 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:25:48.133671  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.045089  8797 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:48.132840  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78099.5us
W20260709 12:25:48.136926  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101915us
I20260709 12:25:48.137450  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.140642  6310 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 3: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:48.130211  8966 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.138196  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91022.6us
I20260709 12:25:48.079509  8883 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:48.138875  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:48.095739  8946 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:48.079869  8937 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:25:48.127035  8813 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.099432  9070 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:48.184872  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86191.9us
I20260709 12:25:48.088706  8905 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.186451  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:25:48.187911  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46320.1us
I20260709 12:25:48.198094  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:48.203708  6384 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:48.204149  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.143621  8962 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.203722  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:48.210086  6384 pending_rounds.cc:85] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 1
I20260709 12:25:48.218743  6384 pending_rounds.cc:107] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:48.219005  6384 raft_consensus.cc:2889] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:48.212002  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.215552  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:48.206022  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.200484  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.221202  6300 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:48.220445  6305 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:48.218302  6303 raft_consensus.cc:2376] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3. Re-sending same reply.
I20260709 12:25:48.139362  9061 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:25:48.235309  6308 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:48.237869  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:25:48.239778  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95379.3us
I20260709 12:25:48.258471  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:48.261078  6304 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:48.262952  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:48.179976  8949 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:48.194123  9058 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:48.152937  8966 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.240885  9081 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:48.292050  9053 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:48.209810  8813 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.153822  7320 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.329921  9085 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:48.233394  8962 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.333523  9087 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:48.329190  9086 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:25:48.338697  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.228471  8946 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.080s
I20260709 12:25:48.298647  9061 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:25:48.330335  9052 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.590s	user 0.003s	sys 0.000s
W20260709 12:25:48.352291  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 114531us
I20260709 12:25:48.368717  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:48.177559  8889 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.374605  8889 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.376097  8966 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.304646  9058 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.377630  8872 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.378351  9058 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.382794  9085 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:48.350131  8943 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.383785  9085 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.329977  8937 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:48.342851  8588 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:48.395685  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 192106us
W20260709 12:25:48.397145  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 181701us
W20260709 12:25:48.398169  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.398680  6272 leader_election.cc:400] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:25:48.399010  6272 leader_election.cc:403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:25:48.404515  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 141987us
I20260709 12:25:48.348734  8773 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:48.350813  8978 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.378316  8872 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:48.389662  8789 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.413419  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:48.414054  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:25:48.401420  8951 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.000s
I20260709 12:25:48.392035  9061 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:48.409886  9058 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:48.431943  7317 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:48.409415  8943 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:25:48.407950  7320 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.409752  8966 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:25:48.420787  8569 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:48.425945  8951 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:48.417296  9082 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.069s
I20260709 12:25:48.460873  9084 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.447058  8937 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:48.451474  9058 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:48.456939  9087 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:48.461369  9093 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.489492  9083 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:25:48.487480  8978 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:25:48.495395  9026 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.001s
I20260709 12:25:48.506085  9087 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:48.514886  9087 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.500110  9061 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.498817  9084 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:48.498153  9088 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:48.530891  9081 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:48.533491  9086 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:48.533232  9084 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.533954  9086 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.534785  9084 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.513288  9093 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.534963  9086 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.537081  9086 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:48.537245  9092 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.208s	user 0.002s	sys 0.000s
I20260709 12:25:48.542527  9093 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:48.571939  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 171395us
W20260709 12:25:48.399956  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 213576us
I20260709 12:25:48.455464  8789 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.166s
I20260709 12:25:48.460180  8668 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.640136  8962 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:48.642863  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68668.5us
I20260709 12:25:48.647794  8937 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.648211  9093 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:48.449620  8797 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:48.647868  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 201046us
W20260709 12:25:48.530408  9051 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.918s	user 0.000s	sys 0.003s
I20260709 12:25:48.679121  8617 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.679368  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 305) took 1505 ms. Trace:
I20260709 12:25:48.679822  6302 rpcz_store.cc:276] 0709 12:25:47.173856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.251116 (+ 77260us) service_pool.cc:224] Handling call
0709 12:25:47.317005 (+ 65889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.317364 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.317393 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.317849 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.429220 (+111371us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:47.429272 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.429346 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.529754 (+1100408us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:48.678655 (+148901us) raft_consensus.cc:1685] finished
0709 12:25:48.678704 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.679127 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":758,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"spinlock_wait_cycles":4812160,"thread_start_us":70372,"threads_started":1,"wal-append.queue_time_us":75978}
I20260709 12:25:48.688385  8872 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.651012  9093 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:48.690292  9093 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.692334  8905 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:48.696543  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.517843  9026 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.700644  8789 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:48.701246  8789 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:48.701328  9026 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:48.704008  8668 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:48.703928  9058 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.705296  8668 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.512742  8978 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.706848  9098 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:48.707746  9098 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.708477  8668 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.708559  8978 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.675176  8813 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:48.675794  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 78926.4us
I20260709 12:25:48.641814  9087 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.706844  8966 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.018s
I20260709 12:25:48.734973  8936 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:48.735954  8936 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.740571  9061 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.741436  9061 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.741861  6454 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:25:48.720470  8797 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:25:48.744086  9061 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:48.736152  6457 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2219) took 3328 ms (client timeout 2138 ms). Trace:
W20260709 12:25:48.745615  9112 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
I20260709 12:25:48.717999  8617 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.744208  8797 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:48.747083  6457 rpcz_store.cc:269] 0709 12:25:45.407275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.616927 (+209652us) service_pool.cc:224] Handling call
0709 12:25:48.735512 (+3118585us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b5400344c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:48.735782 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4862514048}
I20260709 12:25:48.744232  9081 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.775668  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.774645  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:48.783598  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65294.1us
W20260709 12:25:48.785843  9074 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.453s	user 0.004s	sys 0.001s
I20260709 12:25:48.786516  9026 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:48.787207  9026 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.788179  9026 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.788357  8617 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.786919  9081 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.790355  8789 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.792253  8789 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:48.791808  9093 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.793097  9081 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.793602  9093 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:48.794699  9080 consensus_queue.cc:1175] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 10 status { last_received { term: 8 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.7-9.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:48.797169  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 315) took 1293 ms. Trace:
I20260709 12:25:48.798036  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:48.799320  6457 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
W20260709 12:25:48.804528  9076 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.474s	user 0.004s	sys 0.000s
I20260709 12:25:48.805297  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:48.805682  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:48.740607  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:48.762756  8797 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.807521  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:48.808528  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:48.808123  8797 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.809124  9087 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.809816  8797 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:48.814247  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:48.797654  6301 rpcz_store.cc:276] 0709 12:25:47.503868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.504230 (+   362us) service_pool.cc:224] Handling call
0709 12:25:47.544873 (+ 40643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.941839 (+396966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.796455 (+854616us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b5400140fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:48.796906 (+   451us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":909606656}
I20260709 12:25:48.825385  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:48.826346  6457 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:48.766181  8972 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:48.747526  8936 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.773880  9098 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.847259  8915 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:48.848611  8915 consensus_queue.cc:1243] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:48.853008  9071 raft_consensus.cc:993] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:25:48.853963  9063 raft_consensus.cc:1081] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
W20260709 12:25:48.856174  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.858951  8943 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.859328  8588 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:48.859819  8943 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.860397  8588 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.863058  8948 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.864109  8948 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:48.866577  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 320820us
I20260709 12:25:48.867563  8948 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:48.858139  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:48.829731  8972 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:48.872252  8936 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:25:48.875056  8891 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.877575  8915 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:48.878661  9073 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:48.879693  9073 consensus_queue.cc:1243] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:48.882130  8957 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:48.886718  8978 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.892069  8948 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:48.873207  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:48.899138  6457 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:48.883803  8957 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:48.869128  8588 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:48.902696  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 79999.6us
I20260709 12:25:48.870158  8943 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:48.893041  8941 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:48.893231  8948 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:48.910840  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:48.919222  9087 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:48.919764  9087 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:48.920162  9087 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:48.922101  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.777s	user 0.015s	sys 0.004s
W20260709 12:25:48.922590  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.778s	user 0.015s	sys 0.004s
W20260709 12:25:48.923014  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.77796e+06us
I20260709 12:25:48.923043  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:48.924099  7319 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:25:48.946547  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:48.875346  8972 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:48.959754  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:48.987926  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:48.947520  7316 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:48.951030  8960 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.118s	user 0.016s	sys 0.006s
I20260709 12:25:48.964648  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:48.955221  8943 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:48.973922  8617 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:49.006752  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.006963  9072 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.002033  9081 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:48.956768  8925 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.015962  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.018126  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:48.955843  8588 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:48.957154  8915 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:25:48.995750  8960 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:49.051622  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46837.2us
W20260709 12:25:49.034641  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.038856  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.118821  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:49.041857  8889 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.667s	user 0.000s	sys 0.014s
W20260709 12:25:49.046238  8835 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.923s	user 0.012s	sys 0.013s
I20260709 12:25:49.040797  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:49.054838  8603 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.966s	user 0.009s	sys 0.009s
I20260709 12:25:49.040205  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:49.046070  9033 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.947s	user 0.006s	sys 0.011s
I20260709 12:25:49.052305  8617 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:49.054657  7320 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:49.041404  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.142861  6302 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:49.048563  8972 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:49.043097  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.046365  9023 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.034430  8886 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.033020  9081 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.172076  6308 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:49.103677  8943 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.145756  8617 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.055243  9072 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.182889  6304 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:49.050040  8908 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.120s
I20260709 12:25:49.048648  8918 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.125s
I20260709 12:25:49.188521  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.189284  6452 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:49.191509  9072 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.195238  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.158109  8789 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.203373  8789 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.207033  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
W20260709 12:25:49.209982  7373 heartbeater.cc:646] Failed to heartbeat to 127.5.4.126:35813 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.4.126:35813 timed out after 15.000s (SENT)
I20260709 12:25:49.211320  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2249) took 2075 ms. Trace:
I20260709 12:25:49.211756  8738 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:49.212255  8738 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:49.196095  8773 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:49.213281  8738 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.198897  6456 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:49.145627  8905 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:25:49.198866  9091 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:49.200456  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 89453.6us
I20260709 12:25:49.200809  8936 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.201305  9084 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.627s	user 0.015s	sys 0.004s
I20260709 12:25:49.150285  8588 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.180929  8617 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:49.213795  8773 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:49.260299  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.158574  8835 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:49.118357  8948 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:49.223661  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73947.3us
I20260709 12:25:49.168723  9113 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:49.228562  9091 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:49.223459  8498 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:25:49.229641  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:49.211822  6453 rpcz_store.cc:276] 0709 12:25:47.135254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.347459 (+212205us) service_pool.cc:224] Handling call
0709 12:25:49.210806 (+1863347us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400265f54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:49.211052 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":283765632}
I20260709 12:25:49.232244  7314 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:49.232648  7332 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:25:49.233487  7317 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:25:49.234894  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:49.163409  9033 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.229517  8789 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:49.243530  9115 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.217s	user 0.002s	sys 0.000s
I20260709 12:25:49.244810  6456 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:49.142218  8925 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:49.248565  9122 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:25:49.251969  8711 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.244s	user 0.018s	sys 0.002s
I20260709 12:25:49.253672  9081 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.161435  8603 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.259621  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:49.224102  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 493) took 1195 ms. Trace:
I20260709 12:25:49.269719  7317 pending_rounds.cc:85] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 5
I20260709 12:25:49.195799  8937 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:49.287460  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:49.287734  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.288681  6451 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:49.288866  8603 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:49.283063  7320 rpcz_store.cc:276] 0709 12:25:47.998589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.007471 (+  8882us) service_pool.cc:224] Handling call
0709 12:25:48.051579 (+ 44108us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.158743 (+1107164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.193823 (+ 35080us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.290169  6458 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:49.291385  6450 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:49.291990  6309 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:49.283432  7317 pending_rounds.cc:107] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:25:49.267238  9113 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:49.292953  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 133107us
I20260709 12:25:49.294797  6301 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:49.292892  7317 raft_consensus.cc:2889] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.293192  8966 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.299188  8966 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.299724  8995 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:49.296353  9113 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.224138  8886 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.306190  8886 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:49.267120  9091 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.323705  9091 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.283362  8936 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.328938  8936 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:49.309077  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 79595.7us
I20260709 12:25:49.331275  8925 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.294514  9084 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:49.333807  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.334847  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:49.335505  6380 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:49.335929  9033 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.275365  8889 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.337069  8889 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.338162  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.338708  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.339143  6376 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:49.339335  8889 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.341151  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.334862  6384 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:49.341884  6379 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:49.344178  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 495) took 1344 ms. Trace:
I20260709 12:25:49.344781  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.345116  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.345906  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:25:49.346359  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.347121  6385 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:25:49.348392  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.309005  9087 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.311543  9094 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.344710  7318 rpcz_store.cc:276] 0709 12:25:47.999835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.007371 (+  7536us) service_pool.cc:224] Handling call
0709 12:25:48.078426 (+ 71055us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.322261 (+243835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.322533 (+   272us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.324911 (+  2378us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.417721 (+ 92810us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:48.422768 (+  5047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.431532 (+  8764us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.342345 (+910813us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b24002eee78. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f8fb0 00007f6fa2e6f488 00007f6fa2e6fbe8 00007f6fa2e47398 00007f6fa2e4276a 00007f6fa2e41bb6
0709 12:25:49.343544 (+  1199us) raft_consensus.cc:1685] finished
0709 12:25:49.343571 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.343961 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":197181,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":3370368,"thread_start_us":260224,"threads_started":2,"wal-append.queue_time_us":182179}
I20260709 12:25:49.351653  9094 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.310194  8773 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.355489  7329 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.312184  8960 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:49.357451  9087 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.354213  9094 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.359315  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2262) took 2220 ms. Trace:
I20260709 12:25:49.359778  6454 rpcz_store.cc:276] 0709 12:25:47.138897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.481928 (+343031us) service_pool.cc:224] Handling call
0709 12:25:48.740836 (+1258908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.742467 (+  1631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.359004 (+616537us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400344c50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:49.359203 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2887032576}
I20260709 12:25:49.280961  9026 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.315595  8943 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.369295  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:49.290333  8953 ts_tablet_manager.cc:1929] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.3
I20260709 12:25:49.371126  8953 log.cc:1199] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-3-root/wals/2483d7801887403e873adc62429fe046
I20260709 12:25:49.317687  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.332098  8603 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:49.314113  8588 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.368165  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.316937  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.374941  8738 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.375345  6458 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
W20260709 12:25:49.376758  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 92267.6us
I20260709 12:25:49.259018  9023 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.380924  9023 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.382850  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:49.386006  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:49.387292  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:49.388226  9094 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:49.391539  8735 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:49.391778  8966 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.391995  8735 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:49.393097  8735 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.375001  6376 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:49.398005  8995 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.398701  8995 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.394392  8735 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:49.401149  7332 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:49.383083  9026 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.402597  8995 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:49.403729  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.411443  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1626) took 2241 ms. Trace:
I20260709 12:25:49.411623  8953 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45486 (request call id 0) took 5711 ms. Trace:
I20260709 12:25:49.411893  6309 rpcz_store.cc:276] 0709 12:25:47.169720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.233140 (+ 63420us) service_pool.cc:224] Handling call
0709 12:25:47.243915 (+ 10775us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.292469 (+2048554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.408793 (+116324us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b54002d14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:49.411360 (+  2567us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4695355648}
I20260709 12:25:49.413801  6376 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:49.415001  8861 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:49.415788  8861 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.416604  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.417634  6382 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:49.417748  6456 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:49.417974  8908 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.417634  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.418764  8908 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.418033  8861 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:49.420499  6385 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:25:49.421120  6385 pending_rounds.cc:85] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 4
I20260709 12:25:49.421602  6385 pending_rounds.cc:107] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:25:49.421900  6385 raft_consensus.cc:2889] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.423358  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.424140  6382 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 2.
I20260709 12:25:49.423358  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.412181  8953 rpcz_store.cc:276] 0709 12:25:43.889710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.890044 (+   334us) service_pool.cc:224] Handling call
0709 12:25:44.965314 (+1075270us) ts_tablet_manager.cc:1151] Deleting tablet 2483d7801887403e873adc62429fe046
0709 12:25:44.995239 (+ 29925us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:25:47.974018 (+2978779us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:49.289963 (+1315945us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:49.411490 (+121527us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1767704,"tablet-delete.queue_time_us":457470,"thread_start_us":183613,"threads_started":1}
W20260709 12:25:49.434708  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.467061  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.443174  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.476128  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.476464  6454 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:49.421070  8908 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:48.837721  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:49.485020  6384 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:25:49.488617  6376 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:49.490252  6457 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:49.490465  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.491096  6456 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:25:49.491372  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:49.491695  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.492053  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.492334  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.493080  6455 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:49.494643  7316 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:25:49.496881  8957 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 12:25:49.498024  6383 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:25:49.498407  6383 pending_rounds.cc:85] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 3
I20260709 12:25:49.498576  8995 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.498795  8905 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.499046  8918 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:49.499349  8995 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.500375  8908 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:49.501708  7335 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:25:49.503352  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:49.498723  6383 pending_rounds.cc:107] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:25:49.488953  6450 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:25:49.499430  8905 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.502746  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.501922  8995 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:49.508273  6383 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.487934  9026 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.507262  8891 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.509191  6453 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:49.499493  8918 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.509944  9023 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:49.519111  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:49.510926  8957 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:49.511631  9080 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.502732  8738 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:49.514658  9082 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:49.520159  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.531949  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:49.532322  6456 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:49.517206  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.426048  8617 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.521744  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:49.523157  6307 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:49.524681  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.530771  6455 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:49.536516  8900 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.536703  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.537083  8900 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.537734  6302 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:49.523097  8918 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:49.538892  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.539021  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:49.540293  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:49.540521  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.541604  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.543915  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2145) took 1334 ms. Trace:
I20260709 12:25:49.545620  8905 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:49.516816  6457 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2483d7801887403e873adc62429fe046"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59632
I20260709 12:25:49.546921  6457 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:49.547313  6457 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:49.544215  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.549049  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.551719  6377 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:49.554474  9026 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.555825  8738 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.555109  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.537034  6309 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:49.560537  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.560825  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:49.562480  7332 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:49.537132  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.544353  6383 rpcz_store.cc:276] 0709 12:25:48.209547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.210755 (+  1208us) service_pool.cc:224] Handling call
0709 12:25:48.211152 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.518877 (+1307725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.543592 (+ 24715us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54003d0c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:49.543807 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2957955584}
I20260709 12:25:49.542377  6381 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:49.519385  8966 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.565573  6457 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:49.574402  6380 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:49.582782  6452 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:49.580054  6450 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:49.580449  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:49.587548  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:49.554009  9113 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.581671  6382 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:49.558463  9026 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:49.558862  9023 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:49.572767  8978 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:49.593449  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:49.531198  9080 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.597277  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:49.599835  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.567967  8957 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.601675  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.602474  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.602696  6303 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:49.625178  6457 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:49.603366  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.626417  6456 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 1.
I20260709 12:25:49.604346  6305 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:49.605005  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.628294  6307 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:49.607543  8617 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:49.608215  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:49.633057  7329 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:49.610746  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:49.544437  8891 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.587390  8738 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:49.619182  9107 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:49.619711  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:49.590374  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.638639  6310 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:49.621387  6377 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:49.624250  6454 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:49.635370  7331 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:49.629452  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:49.621876  8978 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.582422  8966 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.645651  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:49.642783  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45610.9us
I20260709 12:25:49.646675  6385 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:49.635218  8735 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:49.633821  8936 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.634090  8943 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:49.648078  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.645813  6304 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:49.651450  6301 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:49.651849  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:49.653823  6378 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:25:49.658749  6300 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:49.646525  8978 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.657830  6378 pending_rounds.cc:85] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 3
I20260709 12:25:49.669628  6378 pending_rounds.cc:107] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:25:49.669993  6378 raft_consensus.cc:2889] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.667215  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.648514  9026 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:49.674638  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45294.5us
I20260709 12:25:49.674818  6307 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:49.655670  9091 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:49.677608  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:49.679911  6382 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:49.659678  8936 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:49.611662  6193 catalog_manager.cc:5028] TS 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): tablet 2483d7801887403e873adc62429fe046 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:25:49.682722  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:49.774214  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70775.4us
I20260709 12:25:50.121794  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 420) took 1679 ms. Trace:
I20260709 12:25:49.686942  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.120833  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.183291  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.108151  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:49.752661  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:49.761008  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.109349  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.166918  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:49.686926  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.187322  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.904949  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 422) took 1279 ms. Trace:
W20260709 12:25:49.777187  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 98495.4us
I20260709 12:25:50.136631  7329 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:25:50.081961  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 579) took 1084 ms. Trace:
I20260709 12:25:49.866999  6456 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:49.826927  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.181602  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:49.734797  6304 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
W20260709 12:25:49.776046  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 282803us
I20260709 12:25:50.191970  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:50.204558  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:50.102632  9058 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.013s	sys 0.029s
I20260709 12:25:50.182718  7317 rpcz_store.cc:276] 0709 12:25:48.443359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.443596 (+   237us) service_pool.cc:224] Handling call
0709 12:25:48.443943 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.298529 (+854586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.121372 (+822843us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400566b54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:50.121710 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646470528}
I20260709 12:25:50.189814  7331 rpcz_store.cc:276] 0709 12:25:48.997893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.021858 (+ 23965us) service_pool.cc:224] Handling call
0709 12:25:49.049284 (+ 27426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.641803 (+592519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.063943 (+422140us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400438ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:50.081873 (+ 17930us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336470272}
I20260709 12:25:50.190689  7335 rpcz_store.cc:276] 0709 12:25:48.625750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.667829 (+ 42079us) service_pool.cc:224] Handling call
0709 12:25:49.357564 (+689735us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:25:49.501310 (+143746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.502176 (+   866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.899855 (+397679us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54003bcc50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:49.904858 (+  5003us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342070272}
I20260709 12:25:49.678990  9026 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:50.163964  9133 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
I20260709 12:25:50.123291  8957 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:49.753046  9085 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.146s	user 0.018s	sys 0.019s
I20260709 12:25:50.164239  8889 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.148600  8872 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:50.211561  6193 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 621239us
I20260709 12:25:50.208822  8948 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.122354  8891 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:49.930671  9137 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:25:50.032354  9080 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:50.213908  9086 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:50.222586  9085 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.778s	user 0.023s	sys 0.033s
I20260709 12:25:50.106802  8900 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:50.219888  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.222276  8773 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.211972  9058 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.014s	sys 0.038s
W20260709 12:25:50.223536  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 540909us
I20260709 12:25:49.733776  9007 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:50.216009  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:49.680387  8735 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:49.733588  8898 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.549s
I20260709 12:25:50.214527  8966 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:50.229985  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.228392  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:50.216732  9026 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:50.217669  6453 raft_consensus.cc:2410] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:50.219483  9084 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:49.855700  8918 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.221479  8738 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.275377  8891 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.261165  8966 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.278738  8891 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.278934  8966 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.281002  8891 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:50.281347  8966 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:50.281510  8898 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:50.281960  8898 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:50.283226  8898 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.284474  8898 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:50.286810  9058 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:50.286710  9007 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.247277  9086 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.289009  9007 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:50.289376  6206 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 140) took 16208 ms (client timeout 15000 ms). Trace:
I20260709 12:25:50.289901  9086 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:50.104568  8943 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.243551  6456 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:49.819070  9023 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.229122  8498 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.224223  8957 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.251192  6452 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:25:50.233637  9138 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:50.255188  9130 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:25:50.237947  9080 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:50.249917  9071 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:50.261477  9082 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.230927  8889 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.287781  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:50.294606  6454 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:25:50.346241  8918 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:50.352639  6454 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:50.206442  8908 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:25:49.718279  8617 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:50.300048  7329 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.332964  9143 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.281998  8966 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:50.237020  8948 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.257534  8773 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.302383  8905 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:50.299834  8978 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.003s
W20260709 12:25:50.314198  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45155.2us
I20260709 12:25:50.310995  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:50.314818  9139 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:25:50.289758  6206 rpcz_store.cc:269] 0709 12:25:34.096191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.098614 (+  2423us) service_pool.cc:224] Handling call
0709 12:25:50.289124 (+16190510us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:42.475248 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:42.646552 (+171304us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:42.649899 (+  3347us) write_op.cc:435] Acquired schema lock
0709 12:25:42.653444 (+  3545us) tablet.cc:662] Decoding operations
0709 12:25:42.709986 (+ 56542us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:42.710047 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:42.710076 (+    29us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:25:42.710468 (+   392us) tablet.cc:701] Row locks acquired
0709 12:25:42.710482 (+    14us) write_op.cc:260] PREPARE: finished
0709 12:25:42.710665 (+   183us) write_op.cc:270] Start()
0709 12:25:42.710786 (+   121us) write_op.cc:276] Timestamp: P: 1783599942710612 usec, L: 0
0709 12:25:42.710805 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:25:42.775331 (+ 64526us) log.cc:844] Serialized 1650 byte log entry
0709 12:25:43.417346 (+642015us) op_driver.cc:464] REPLICATION: finished
0709 12:25:44.448165 (+1030819us) write_op.cc:301] APPLY: starting
0709 12:25:44.451109 (+  2944us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:44.467920 (+ 16811us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:44.482547 (+ 14627us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:44.608509 (+125962us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:44.687968 (+ 79459us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:25:44.695354 (+  7386us) write_op.cc:312] APPLY: finished
0709 12:25:44.815980 (+120626us) log.cc:844] Serialized 41 byte log entry
0709 12:25:45.164134 (+348154us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:45.171767 (+  7633us) write_op.cc:454] Released schema lock
0709 12:25:45.176505 (+  4738us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:45.269782 (+ 93277us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:45.271433 (+  1651us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":56154,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":1001125,"apply.run_cpu_time_us":28521,"apply.run_wall_time_us":815488,"num_ops":4,"prepare.queue_time_us":482561,"prepare.run_cpu_time_us":24314,"prepare.run_wall_time_us":596524,"raft.queue_time_us":203939,"raft.run_cpu_time_us":6351,"raft.run_wall_time_us":190716,"replication_time_us":696811,"thread_start_us":896688,"threads_started":5,"wal-append.queue_time_us":668957,"wal-append.run_cpu_time_us":13162,"wal-append.run_wall_time_us":2339585}]]}
I20260709 12:25:50.334446  6456 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:50.335616  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.290231  9091 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:50.258806  8900 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:25:50.346545  9080 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:50.336625  8943 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:50.355908  7287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.502s (SENT) [suppressed 8 similar messages]
I20260709 12:25:50.348928  9071 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:50.354571  9082 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.376518  9085 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:50.353952  8936 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:50.361402  8617 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.360970  8966 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:50.392298  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81400.9us
I20260709 12:25:50.348376  8957 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.332942  8738 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.385762  9091 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.341614  8498 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.391481  9080 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.406194  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:50.383340  8972 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:50.390708  8905 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:50.398819  9071 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:50.393790  9086 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.402647  8891 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:25:50.430711  8948 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:50.430840  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:50.431735  8735 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:50.432906  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:50.434907  9107 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:50.434705  8937 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:50.435598  9107 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:50.435273  8905 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.436013  9107 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:50.436437  9107 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:50.437491  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:50.434310  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:50.450356  7329 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:50.454447  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:50.401400  8617 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:50.402881  8966 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.361845  9143 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.384730  8978 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.438259  9007 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:50.496639  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 315111us
I20260709 12:25:50.442575  9085 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.502513  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:50.459617  8936 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:50.508864  6378 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:50.459545  8900 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:50.516470  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:50.518405  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:50.442668  9086 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.524423  6376 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:50.491334  8891 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.055s
I20260709 12:25:50.513289  8900 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:50.526163  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 334301us
I20260709 12:25:50.491820  8937 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:25:50.528924  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:50.529564  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 325117us
I20260709 12:25:50.532034  6382 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:50.532928  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:50.537691  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:50.508145  9007 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:50.557636  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53151.4us
I20260709 12:25:50.556210  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:50.530506  8908 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:25:50.559477  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:50.563279  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 207400us
I20260709 12:25:50.544382  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:25:50.539541  9145 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:25:50.530968  8900 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:50.567974  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:50.569622  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:50.572877  6382 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:50.542960  8735 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:25:50.590297  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:50.590776  8738 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:50.592485  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:50.592835  8918 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:50.593246  8918 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:50.593732  9130 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:50.594089  9130 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:50.595080  9130 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.596308  9130 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:50.597064  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 267) took 1792 ms. Trace:
I20260709 12:25:50.597435  6459 rpcz_store.cc:276] 0709 12:25:48.804965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.805210 (+   245us) service_pool.cc:224] Handling call
0709 12:25:50.596866 (+1791656us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54001617d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:50.596997 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3751089408}
I20260709 12:25:50.591379  7316 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:50.608191  8498 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:50.526782  9153 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
W20260709 12:25:50.611043  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60727.5us
I20260709 12:25:50.611982  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:50.616343  6376 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:25:50.599107  9148 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:50.627002  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:50.630178  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85923.1us
I20260709 12:25:50.637401  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:50.631256  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:50.639213  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:50.601234  8735 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:50.649936  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 146493us
I20260709 12:25:50.608465  8773 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:50.679383  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 200349us
W20260709 12:25:50.697456  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.629475  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:50.709836  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:50.563954  9146 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:25:50.746948  6376 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:50.747839  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 135983us
I20260709 12:25:50.659209  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:50.639580  9023 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:50.686412  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 291815us
W20260709 12:25:50.755878  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 163463us
W20260709 12:25:50.762130  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 130983us
I20260709 12:25:50.636726  8957 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:50.763155  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:50.637677  8498 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:50.681330  9145 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:25:50.594136  8918 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.649282  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:50.601962  9026 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.779426  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.799400  6457 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
I20260709 12:25:50.679884  8735 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:50.789547  9084 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 12:25:50.810760  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:50.780201  8948 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:25:50.793614  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.780653  9058 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:50.801358  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:50.711267  9151 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:50.799768  9155 ts_tablet_manager.cc:933] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:25:50.786608  8908 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:50.838442  6459 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:50.838972  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:50.774267  7320 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:50.844566  9084 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:25:50.866005  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 285352us
W20260709 12:25:50.877969  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76678us
I20260709 12:25:50.648010  9148 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.817963  8948 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:50.813052  7318 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:50.823570  9151 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.780471  9150 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.107s
I20260709 12:25:50.825649  9058 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:50.790993  8498 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:50.838541  9082 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:50.856935  8773 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:50.824301  8936 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:50.908984  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 145882us
I20260709 12:25:50.897732  8948 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:50.885682  9161 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.900858  9084 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:50.895566  9159 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:25:50.907162  9142 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.313s	user 0.002s	sys 0.001s
I20260709 12:25:50.902141  9058 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:50.897037  9151 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:50.886662  9158 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
W20260709 12:25:50.947273  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 288206us
I20260709 12:25:50.946798  9176 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:50.952431  9176 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.952885  9176 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:50.951875  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.939165  9159 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.966677  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:50.939903  9084 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:50.978135  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 110355us
I20260709 12:25:50.976576  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:50.981458  8498 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:50.982172  9179 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.982836  9179 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:50.984957  9179 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:50.985486  9179 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:50.986660  9084 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:50.987494  9163 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:25:50.988612  8898 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
W20260709 12:25:50.990067  9091 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.581s	user 0.004s	sys 0.006s
I20260709 12:25:50.993222  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.012100  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:50.977927  8891 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:51.051623  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75197.3us
I20260709 12:25:50.937963  8773 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:51.055150  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 217986us
I20260709 12:25:51.059381  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.058210  6380 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:25:51.062763  6380 pending_rounds.cc:85] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:25:51.062991  6380 pending_rounds.cc:107] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:25:51.063216  6380 raft_consensus.cc:2889] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:50.980948  9150 consensus_queue.cc:1243] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:50.970641  8978 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.068825  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:51.068693  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:50.980118  9159 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:51.056728  8498 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:51.058939  8898 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:50.939513  9161 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.063285  8905 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:51.082705  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:51.063359  9143 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.096590  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.083103  9175 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.994459  9091 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.069919  9113 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.117136  9091 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.118837  9150 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:51.119438  9150 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.119908  9150 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.119582  8948 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.122833  7318 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:25:51.123208  6384 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:25:51.123652  6384 pending_rounds.cc:85] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:25:51.124197  6384 pending_rounds.cc:107] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:25:51.118851  9171 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:25:51.124429  6384 raft_consensus.cc:2889] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.125634  9171 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:51.091599  8898 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.079787  9178 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:25:51.072388  9023 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:51.050675  9177 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:25:51.084137  9026 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.111250  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 500) took 1843 ms. Trace:
I20260709 12:25:51.122879  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.087085  8498 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.101012  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:51.125728  8773 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.093036  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.063650  8936 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.068378  8891 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.125653  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.141785  6380 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:25:51.152544  6381 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 2: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:51.153275  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56808.3us
I20260709 12:25:51.122331  8948 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.126268  9171 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.155229  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 324) took 1477 ms. Trace:
I20260709 12:25:51.104508  9170 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.126298  9149 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:51.096146  9186 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.127889  9148 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.186556  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.140970  7319 rpcz_store.cc:276] 0709 12:25:49.268094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.318148 (+ 50054us) service_pool.cc:224] Handling call
0709 12:25:49.318546 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.394601 (+ 76055us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.394640 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.395279 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.721906 (+326627us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:49.721954 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.054924 (+332970us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b54006c16d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:50.054991 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.598832 (+543841us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:51.110902 (+512070us) raft_consensus.cc:1685] finished
0709 12:25:51.110936 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.111158 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11969,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":111963392,"thread_start_us":326114,"threads_started":1,"wal-append.queue_time_us":869291}
I20260709 12:25:51.198731  6381 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:51.198922  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.127115  9167 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.193701  8889 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:51.200644  9130 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.604s	user 0.013s	sys 0.000s
I20260709 12:25:51.203409  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.204228  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.107187  8905 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.163023  9026 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:51.130663  9047 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 12:25:51.152238  8498 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.218151  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.235261  6381 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:51.187433  6300 rpcz_store.cc:276] 0709 12:25:49.678350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.679148 (+   798us) service_pool.cc:224] Handling call
0709 12:25:50.178345 (+499197us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.497326 (+318981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.497368 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.497997 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.662844 (+164847us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:50.843836 (+180992us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.154901 (+311065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.154989 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.155030 (+    41us) raft_consensus.cc:1685] finished
0709 12:25:51.155044 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.155172 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5363,"prepare.queue_time_us":256833,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":208637,"raft.run_cpu_time_us":433,"raft.run_wall_time_us":59785,"thread_start_us":799502,"threads_started":4,"wal-append.queue_time_us":293168}
I20260709 12:25:51.230613  6384 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:51.187487  9149 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.230096  8948 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:25:51.251601  9149 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.252712  8948 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.253639  8948 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:51.291986  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 191021us
I20260709 12:25:51.147116  9155 tablet_copy_client.cc:323] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:25:51.201503  9158 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:25:51.207064  9148 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:51.250859  9184 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.252857  9178 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.255095  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:51.258841  9187 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:25:51.259842  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2285) took 1763 ms. Trace:
W20260709 12:25:51.266163  8966 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.755s	user 0.009s	sys 0.000s
I20260709 12:25:51.215754  9183 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:51.269428  9169 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.381s	user 0.006s	sys 0.000s
W20260709 12:25:51.270980  9154 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.589s	user 0.000s	sys 0.004s
I20260709 12:25:51.271651  9161 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:51.272902  8889 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:51.274523  8891 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.209880  9186 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.190178  9171 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:51.219702  9174 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:51.235772  8943 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.002s
I20260709 12:25:51.283347  9188 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:51.283706  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80417.1us
I20260709 12:25:51.284834  9086 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.148068  8898 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.288045  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.301057  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:51.286720  9107 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.850s	user 0.007s	sys 0.007s
I20260709 12:25:51.310935  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.423995  6458 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
W20260709 12:25:51.424116  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 123198us
I20260709 12:25:51.209053  9190 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:51.286772  8872 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:51.321794  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.322650  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.430819  6457 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 3.
W20260709 12:25:51.324397  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.314085  6450 rpcz_store.cc:276] 0709 12:25:49.495928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.212892 (+716964us) service_pool.cc:224] Handling call
0709 12:25:50.222723 (+  9831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.291274 (+ 68551us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:50.555427 (+264153us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.069415 (+513988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.069506 (+    91us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:51.069637 (+   131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.259525 (+189888us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b5400344254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:51.259591 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.259766 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80210,"prepare.queue_time_us":454893,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":14711,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":14550,"spinlock_wait_cycles":190630400,"thread_start_us":668492,"threads_started":2,"wal-append.queue_time_us":714144}
I20260709 12:25:51.336787  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.435657  6384 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:51.436833  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.437282  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.235543  9026 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.438299  7316 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:51.345731  8889 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.347946  9084 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.303182  9148 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.348639  9186 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:51.350152  8735 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.003s
I20260709 12:25:51.350924  9165 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:51.353333  8498 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.303455  9184 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.337606  7318 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:25:51.357985  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:51.325687  9080 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.883s	user 0.013s	sys 0.000s
W20260709 12:25:51.359805  8617 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.000s
I20260709 12:25:51.202533  9170 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.362368  9171 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.364079  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:51.366999  8900 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.739s	user 0.008s	sys 0.003s
I20260709 12:25:51.369863  9058 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.375612  8948 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.215673  9167 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.377055  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.378285  9159 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:51.442997  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.382748  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.300239  8905 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:51.396754  6381 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:25:51.385030  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 297177us
W20260709 12:25:51.405220  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.410745  8943 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.324311  8966 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.412181  9188 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.413693  9130 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:51.511528  6300 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:51.207417  9175 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.515697  7331 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.527429  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:51.519519  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:51.419961  9187 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:25:51.528463  6309 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:51.418262  9146 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:51.509704  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.529192  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.529274  9146 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:51.511247  8872 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:51.530031  6379 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:51.420943  9181 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.158s	user 0.000s	sys 0.002s
I20260709 12:25:51.530905  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.530748  8872 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.531378  9152 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.531672  8937 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.532043  9152 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:51.532680  9192 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:25:51.414322  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.533954  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:51.419266  9134 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.072s	user 0.006s	sys 0.003s
I20260709 12:25:51.534667  9091 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:51.535027  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.535301  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.535704  6450 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:51.536397  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.536721  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.422070  9007 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.537315  6450 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.447443  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:51.537586  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.460851  7329 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:51.326943  9177 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.131s
I20260709 12:25:51.424968  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:51.434762  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:51.534086  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 119872us
I20260709 12:25:51.505826  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.426736  9183 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:51.429212  9107 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.507143  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.538516  6378 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:25:51.532200  8937 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.450410  9184 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:51.512931  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:51.509088  9085 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.005s	user 0.012s	sys 0.000s
W20260709 12:25:51.509896  9071 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.079s	user 0.010s	sys 0.003s
I20260709 12:25:51.556663  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:51.518052  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.449761  6377 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:25:51.518720  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.438403  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.559499  6384 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:51.541778  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.560384  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:51.542968  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81182.3us
I20260709 12:25:51.548162  6304 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:51.538160  6452 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:51.551566  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:51.563064  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 138231us
I20260709 12:25:51.563771  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2277) took 2238 ms. Trace:
I20260709 12:25:51.537765  9007 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:51.456955  9058 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.523984  8889 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:51.561219  6454 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:25:51.432976  9179 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:51.454813  8900 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.475359  9176 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.448215  8735 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.450062  9080 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.451890  7320 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:51.440660  9026 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.450872  8498 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.452366  9170 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.514417  9159 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.428985  9151 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.567245  9007 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.384966  9023 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.553452  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.444598  9084 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.546775  9152 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.572202  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.564131  6451 rpcz_store.cc:276] 0709 12:25:49.314688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.314864 (+   176us) service_pool.cc:224] Handling call
0709 12:25:49.315244 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.315919 (+   675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.315947 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.316479 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.679931 (+363452us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:49.679983 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.680066 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.303843 (+1623777us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:51.537951 (+234108us) raft_consensus.cc:1685] finished
0709 12:25:51.537975 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.552922 (+ 14947us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":7837440,"thread_start_us":362948,"threads_started":1,"wal-append.queue_time_us":950786}
I20260709 12:25:51.595274  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.572705  6376 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:51.618896  6458 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:51.577857  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.621153  6456 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:25:51.571986  9086 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:51.567603  9071 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.637061  9071 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.581478  9183 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.591739  9007 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.644366  6450 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:51.586964  6383 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:51.573189  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:51.647398  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 74297.7us
I20260709 12:25:51.647557  8498 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:51.591205  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:51.512447  9167 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.648228  8498 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.648711  8498 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:51.649032  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57909.2us
I20260709 12:25:51.650506  9161 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:51.650678  8972 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.555778  9172 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.651289  8972 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.651657  9172 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.653942  9172 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.655475  9148 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:51.657657  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109607us
I20260709 12:25:51.557646  9190 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.562770  9174 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:51.658687  9190 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:51.658986  9174 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:51.659299  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:51.659880  9190 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:51.661068  9184 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:51.661721  9190 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.662585  8498 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.665804  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:51.668479  9185 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.422s	user 0.000s	sys 0.001s
I20260709 12:25:51.523491  8948 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:25:51.568518  9085 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:51.584079  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.670146  6452 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:51.672039  8905 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:51.672569  8905 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.672979  6385 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:51.450351  8617 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.572170  9198 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:51.606561  9170 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:51.627182  9150 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 12:25:51.675638  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.676352  6453 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:51.591428  9195 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:25:51.628985  8966 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.630970  9107 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.630780  9149 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.633522  8937 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.717640  9130 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:51.675858  8617 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.718717  9130 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:51.719116  9130 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:51.719589  9159 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.719897  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.720074  9159 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.721909  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.571923  8957 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:51.723719  8957 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.724664  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.725847  8957 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.726014  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.727757  8735 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.660082  9174 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.728242  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.728957  6376 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:51.729687  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.730276  6376 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:51.510630  9153 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:25:51.731525  9153 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:51.731870  9153 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:51.732765  8943 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.647825  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.576611  9177 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:51.733651  6451 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:51.733745  9177 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.734450  9177 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:51.675575  9170 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.680938  6456 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:51.682209  6452 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:51.666446  6450 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:25:51.682960  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 59017.9us
I20260709 12:25:51.734684  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.683799  6458 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:51.690641  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.583681  8900 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.634300  9086 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.709748  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:51.588009  9080 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.704802  8498 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.715170  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.712554  9186 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:25:51.716599  9149 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.727674  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.767096  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.767645  6381 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:51.679059  9172 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:25:51.715193  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.746433  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.747892  7335 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 4: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:51.448628  9165 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:51.653968  8972 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:51.752719  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.754849  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:51.758126  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.753434  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.758992  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.709000  9184 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.055s
I20260709 12:25:51.720304  9026 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:25:51.766157  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.766355  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.767843  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.732724  9153 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:51.675841  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1708) took 1179 ms. Trace:
W20260709 12:25:51.747924  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57363.6us
I20260709 12:25:51.762209  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.706067  9150 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:51.755131  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.754130  8905 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:51.750582  9086 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.752480  8735 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.874341  7332 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:51.745678  9170 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.823266  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.808022  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.875713  6456 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:51.806744  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:51.879508  6376 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:51.891521  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.892287  6376 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:51.806874  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.879760  7331 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:51.880887  6300 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:51.881340  6309 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:51.876230  6302 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:25:51.878456  6385 rpcz_store.cc:276] 0709 12:25:50.496715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.496903 (+   188us) service_pool.cc:224] Handling call
0709 12:25:51.675641 (+1178738us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003d3bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:51.675757 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2687264000}
I20260709 12:25:51.801263  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.719995  9130 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:51.903470  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 282780us
I20260709 12:25:51.886284  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.758816  9143 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:51.757675  9047 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 12:25:51.902626  6304 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:25:51.899937  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 153567us
I20260709 12:25:51.875572  7329 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:25:51.919256  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.744458  9159 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:51.906275  6455 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:51.915422  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:51.924979  6425 leader_election.cc:400] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:51.925362  6425 leader_election.cc:403] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:25:51.925261  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.915891  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:51.916064  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.926802  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.927498  6452 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:51.923621  7332 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:51.907438  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:51.929297  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 319635us
I20260709 12:25:51.884909  9184 consensus_queue.cc:1243] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:25:51.928439  6302 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:51.938086  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.926638  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:51.935597  6308 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:51.930110  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.939085  6379 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.941694  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.941953  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.943559  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:51.944458  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.944612  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.944799  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:51.945335  6300 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:51.945962  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.946537  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:51.817857  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 2483d7801887403e873adc62429fe046 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
W20260709 12:25:51.956621  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.959233  6376 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:25:51.960587  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.763809  9198 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.139s
I20260709 12:25:51.986903  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.978286  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:51.944196  7335 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:51.913508  8972 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:51.991211  6456 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:51.986979  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:52.007741  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.008824  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:52.009737  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.917342  9143 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:52.021534  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.022116  6376 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:52.019783  6381 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:52.013669  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:52.021974  6453 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:52.023432  6452 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:51.938632  8943 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:52.024865  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:52.028968  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.029119  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.029816  6376 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.030655  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.025429  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:52.025708  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 78425us
I20260709 12:25:52.032778  6452 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:52.020556  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:51.978916  9159 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:52.028226  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.035449  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.035737  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.036033  6455 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:51.986909  9178 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:52.037827  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.018978  9196 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:52.032975  6377 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:52.027155  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 131012us
I20260709 12:25:52.020596  8891 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.034646  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.033003  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:51.948915  8617 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.036057  6385 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:51.987090  9172 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.013854  9130 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.038456  6456 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.032871  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.039675  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:51.916317  8735 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.045203  6452 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:52.031682  9179 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.025456  9143 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.048398  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.051409  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.051656  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:52.052357  6309 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:52.052559  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.044581  7335 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:25:52.042636  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.049052  6376 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:52.049463  6381 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:52.053304  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:52.019805  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 2483d7801887403e873adc62429fe046 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-2483d7801887403e873adc62429fe046
I20260709 12:25:52.038137  9178 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.059785  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.061689  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:52.068151  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:52.069435  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 174969us
I20260709 12:25:52.065340  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.065606  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.068073  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.047281  8617 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.074357  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:52.075011  6303 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:52.075045  7335 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:52.047454  9172 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.077888  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.078617  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.083177  6381 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:25:52.086599  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.051803  9211 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.058470  9091 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:52.090386  8735 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:52.091131  9210 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.091246  9219 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.092681  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.055967  9196 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:52.099050  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.063246  9168 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:52.135023  6385 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:52.058686  8937 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:25:52.066301  9107 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:52.144881  9107 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:52.085964  9047 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:25:52.067634  9085 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:25:52.067378  8900 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:52.114619  6381 raft_consensus.cc:2376] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10. Re-sending same reply.
I20260709 12:25:52.114724  6456 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:52.164142  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.165300  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:52.077296  8617 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.099401  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.167699  6376 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.101079  8972 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:52.100785  9091 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:52.147218  9007 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
W20260709 12:25:52.148905  9209 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:52.170120  8900 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:52.187489  8900 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.107326  9210 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.188426  9210 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.188021  9107 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.151029  9196 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.190799  9220 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.191349  9220 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:52.191743  9220 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:52.178668  9214 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
W20260709 12:25:52.198589  8898 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.736s	user 0.012s	sys 0.000s
I20260709 12:25:52.152012  8872 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.152498  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.199079  8898 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.199234  8872 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:52.200110  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.072953  9178 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.201159  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.201820  8898 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.087008  9172 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.099886  9211 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.206851  6380 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:52.111639  9221 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:25:52.207442  8889 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.173820  6455 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:52.209951  6455 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:52.158308  6309 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:52.199273  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:52.175328  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 113694us
I20260709 12:25:52.133930  9149 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.114423  9219 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:52.207562  9204 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:25:52.169492  9159 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:52.206761  9211 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:52.143038  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100635us
I20260709 12:25:52.228045  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:52.184005  9091 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.176726  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:52.211978  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 143983us
I20260709 12:25:52.238116  6381 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:52.135205  9168 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.181485  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.226627  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.235201  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.218295  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.238646  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:52.245812  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 211230us
I20260709 12:25:52.221828  9165 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.249970  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.247323  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.245661  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:52.251883  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.228477  9219 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.233227  9150 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.244112  9168 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.255715  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.222461  9086 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.255275  6376 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:52.289041  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.262512  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:52.256235  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:52.275619  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.290828  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:52.223165  9007 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.293463  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.295584  7332 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:52.296927  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:52.290534  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:52.302474  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.235739  9149 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.236937  9177 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.304399  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:52.304483  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 123146us
I20260709 12:25:52.260979  8898 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:52.312222  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 91016.9us
I20260709 12:25:52.304878  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.307976  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.314138  6450 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:52.316494  6376 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.258459  9196 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:52.319070  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:52.257781  9223 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:52.325403  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:52.331373  6308 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:52.331627  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:52.332589  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.333164  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:52.322456  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:52.288362  8498 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.260021  8905 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:52.338836  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45492us
I20260709 12:25:52.356621  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.341539  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:52.345244  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48298.4us
I20260709 12:25:52.341053  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.304193  9189 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:52.357596  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:52.359417  6376 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:52.367974  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.308732  9231 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:52.368817  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49836.1us
I20260709 12:25:52.369153  9231 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.372251  9231 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.310804  9150 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:52.303929  9224 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.373385  9224 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:52.373893  9224 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.311359  9165 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:52.333442  6308 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:52.365090  6304 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:52.378867  9165 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:52.379750  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.379858  9165 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.381057  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:52.381721  6304 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:25:52.324889  9152 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:52.382464  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.379758  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.365785  6309 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:52.383116  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:52.384342  9150 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:52.384476  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.385624  6309 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:52.387549  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.388413  6454 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:25:52.391180  6301 raft_consensus.cc:2368] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6.
I20260709 12:25:52.318634  8898 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:52.388742  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.330108  8872 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:52.406947  9086 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.407424  9179 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:52.408011  9179 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.409344  9228 raft_consensus.cc:2764] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:52.410116  9225 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.358884  7320 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:25:52.410772  9225 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:52.411149  9213 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:25:52.411238  9225 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.412672  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2176) took 1265 ms. Trace:
I20260709 12:25:52.343482  9198 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:52.415248  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:52.413486  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.341579  9196 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:52.475467  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:52.383095  9152 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.355554  8905 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.356453  8937 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:52.394584  9229 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.220s	user 0.002s	sys 0.000s
I20260709 12:25:52.365641  9186 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:25:52.387964  9223 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.302619  8972 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.482324  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:52.407481  9234 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:52.412977  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:52.415431  9215 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:25:52.484328  9152 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.399377  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:52.413043  6378 rpcz_store.cc:276] 0709 12:25:51.147728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.555971 (+408243us) service_pool.cc:224] Handling call
0709 12:25:51.576372 (+ 20401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.577521 (+  1149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.577569 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.578141 (+   572us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.927575 (+349434us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:51.927613 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.927676 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.412437 (+484761us) raft_consensus.cc:1685] finished
0709 12:25:52.412458 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.412608 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":698,"raft.queue_time_us":13178,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":160,"thread_start_us":243814,"threads_started":1,"wal-append.queue_time_us":531241}
I20260709 12:25:52.414216  8891 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:52.496999  6303 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.317198  9159 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:52.516783  6379 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.014s
I20260709 12:25:52.520957  7329 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:25:52.546924  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 191113us
I20260709 12:25:52.528002  9242 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:52.388052  9189 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:52.532078  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 198988us
I20260709 12:25:52.552028  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:52.553376  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:52.483623  9086 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:52.499058  8943 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:52.540946  9210 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.541605  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1729) took 1378 ms. Trace:
I20260709 12:25:52.567545  6459 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:52.541908  6454 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 3.
I20260709 12:25:52.503113  8937 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.511652  8905 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.579002  6303 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:52.579419  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:52.476369  8872 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:52.547610  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:52.513964  9223 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.556907  9231 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.558099  9178 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.561121  8617 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.516635  9150 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:52.566151  6384 rpcz_store.cc:276] 0709 12:25:51.163408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.560378 (+396970us) service_pool.cc:224] Handling call
0709 12:25:51.578758 (+ 18380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.745207 (+166449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.745260 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:51.745331 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:51.745396 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.541354 (+795958us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:52.541400 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.541537 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":361458048}
I20260709 12:25:52.524832  9186 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:52.572412  7320 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:52.572718  9238 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:52.573176  9227 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:52.573412  9149 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.575338  9177 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:52.534994  8898 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.576535  9007 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:52.545706  8891 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:52.596185  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45054.2us
I20260709 12:25:52.549484  9189 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.595922  9233 raft_consensus.cc:2764] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:52.599519  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.563876  9198 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:25:52.570508  9143 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.506s	user 0.006s	sys 0.009s
I20260709 12:25:52.605602  6381 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:52.605125  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:52.643352  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.631493  9238 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.593420  9086 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:52.656103  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.630870  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:52.657235  6454 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:52.633279  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 234593us
I20260709 12:25:52.600329  9223 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.635293  6308 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:52.636884  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:52.637745  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1719) took 1760 ms. Trace:
I20260709 12:25:52.663483  6457 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:52.647554  6384 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:52.664340  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1733) took 1379 ms. Trace:
I20260709 12:25:52.664731  6384 rpcz_store.cc:276] 0709 12:25:51.284985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.630787 (+1345802us) service_pool.cc:224] Handling call
0709 12:25:52.642577 (+ 11790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.664068 (+ 21491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.664274 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:52.650312  7329 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:52.580446  8937 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.630474  9227 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.654132  9250 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:52.654013  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:52.626988  9231 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.659217  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.662653  6382 rpcz_store.cc:276] 0709 12:25:50.877236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.885595 (+  8359us) service_pool.cc:224] Handling call
0709 12:25:50.907592 (+ 21997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.537945 (+630353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.637409 (+1099464us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:52.637663 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1240971520}
I20260709 12:25:52.666729  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.700827  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.701715  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:52.599604  9196 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.667760  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.669102  9242 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:52.655048  9252 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:52.669653  9227 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.680686  9248 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:52.681080  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.642437  9245 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:25:52.686830  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:52.712287  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:52.687446  9226 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:52.686614  9146 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.696054  9086 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.656004  9238 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.697602  9210 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:52.698158  9203 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.041s	user 0.004s	sys 0.000s
I20260709 12:25:52.688442  9150 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.708442  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:52.708405  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67992.2us
W20260709 12:25:52.717020  8889 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.509s	user 0.004s	sys 0.007s
I20260709 12:25:52.716550  9186 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.717628  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.729341  6382 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
W20260709 12:25:52.718662  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 72273.2us
I20260709 12:25:52.719808  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.736852  6384 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:25:52.704433  8908 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:52.704325  8943 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:52.656103  9143 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:52.717598  9220 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.000s
I20260709 12:25:52.744904  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.710143  9233 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:52.705447  9174 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:25:52.739162  8889 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:52.744887  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.771792  6384 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:52.772497  9231 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.772789  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.773211  9255 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:52.773464  6384 raft_consensus.cc:2376] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4. Re-sending same reply.
I20260709 12:25:52.774115  9255 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:52.717535  9252 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.673092  8900 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.748821  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.768890  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:52.774464  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.800158  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1724) took 1720 ms. Trace:
I20260709 12:25:52.800997  6383 rpcz_store.cc:276] 0709 12:25:51.079490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.079769 (+   279us) service_pool.cc:224] Handling call
0709 12:25:51.080134 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.647118 (+566984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.799651 (+1152533us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b540023f5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:52.800056 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1165024512}
I20260709 12:25:52.811611  6384 raft_consensus.cc:2368] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:25:52.802707  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.821086  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.821910  6384 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.822197  9248 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:52.824342  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:25:52.824473  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 122879us
I20260709 12:25:52.825852  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.826730  6383 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:52.747220  9146 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:52.827684  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:52.752132  9242 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:52.775460  9246 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 12:25:52.796998  9161 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.907s	user 0.012s	sys 0.009s
W20260709 12:25:52.774639  9236 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:25:52.831331  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:52.752065  9159 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:52.680703  9250 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.834115  9242 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:52.714615  9224 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.777380  9143 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:52.868415  6457 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:52.866875  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.747619  8908 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.864893  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1848) took 1349 ms. Trace:
I20260709 12:25:52.870150  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.776773  9252 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.831802  9231 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.862210  9249 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:25:52.902420  6424 leader_election.cc:336] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.579s (SENT)
I20260709 12:25:52.831336  9233 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:52.774708  9255 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.879158  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:52.880041  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 171703us
I20260709 12:25:52.862563  9210 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:52.748930  9220 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.924974  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:52.925738  6457 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
W20260709 12:25:52.876899  9161 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.012s	sys 0.013s
I20260709 12:25:52.892184  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:52.888921  6453 rpcz_store.cc:276] 0709 12:25:51.515916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.041070 (+525154us) service_pool.cc:224] Handling call
0709 12:25:52.041427 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:52.041756 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.041781 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:52.053478 (+ 11697us) log.cc:844] Serialized 26 byte log entry
0709 12:25:52.246878 (+193400us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:52.246925 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.247002 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:52.864676 (+617674us) raft_consensus.cc:1685] finished
0709 12:25:52.864701 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.864836 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49130,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"thread_start_us":148622,"threads_started":1,"wal-append.queue_time_us":467441}
W20260709 12:25:52.896023  9170 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.981s	user 0.015s	sys 0.001s
I20260709 12:25:52.883075  9225 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.935907  7329 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:52.867781  8617 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:52.883860  9178 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.883456  9242 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.929003  6383 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:52.930297  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:52.933158  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54093.2us
I20260709 12:25:52.935215  9220 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.956959  6457 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:25:52.940482  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.951745  6384 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:53.023273  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:53.024788  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:52.867703  8943 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:52.921484  9210 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:53.028936  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2304) took 1393 ms. Trace:
I20260709 12:25:53.030345  6457 rpcz_store.cc:276] 0709 12:25:51.635042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.929960 (+1294918us) service_pool.cc:224] Handling call
0709 12:25:52.956704 (+ 26744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.028618 (+ 71914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.028875 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.032119  6457 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:53.033610  6457 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:53.034849  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:53.036832  9179 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.628s	user 0.014s	sys 0.030s
W20260709 12:25:52.940932  9211 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.000s
W20260709 12:25:52.954245  9190 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.292s	user 0.012s	sys 0.002s
W20260709 12:25:52.940258  9172 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.004s
W20260709 12:25:52.938531  9168 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.633s	user 0.005s	sys 0.006s
I20260709 12:25:52.935264  9176 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:52.944201  9153 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:52.911506  9231 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.945006  9225 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.940843  8891 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.045187  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2179) took 1752 ms. Trace:
I20260709 12:25:53.049315  9250 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:52.940222  9188 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.052265  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:53.052103  6382 rpcz_store.cc:276] 0709 12:25:51.291867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.037481 (+1745614us) service_pool.cc:224] Handling call
0709 12:25:53.043600 (+  6119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.043807 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.043836 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.043986 (+   150us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:53.044075 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.044135 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.044262 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.051272  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:52.937839  9170 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.041868  8900 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.029145  8905 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:52.881898  9248 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:53.055805  8498 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.700s	user 0.012s	sys 0.000s
I20260709 12:25:53.063719  9262 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:53.064301  9263 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.065100  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 340) took 1247 ms. Trace:
I20260709 12:25:53.066097  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:53.072266  6308 rpcz_store.cc:276] 0709 12:25:51.817734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.379236 (+561502us) service_pool.cc:224] Handling call
0709 12:25:53.064864 (+685628us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540025add4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.065030 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":578324608}
I20260709 12:25:53.075997  6382 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.046823  9179 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.078457  6453 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:25:53.079433  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1741) took 1662 ms. Trace:
I20260709 12:25:53.081080  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:53.081296  6382 rpcz_store.cc:276] 0709 12:25:51.416738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.060750 (+1644012us) service_pool.cc:224] Handling call
0709 12:25:53.075824 (+ 15074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.077488 (+  1664us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.079373 (+  1885us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:53.082384  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.051923  9172 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.083515  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.082993  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.065587  9188 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:53.087131  6382 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.089859  7335 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 6. Ops: []
I20260709 12:25:53.047603  9211 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.051796  9168 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.079942  9254 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:53.075346  8498 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:53.096002  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.096896  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:53.085687  9176 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:53.068192  8937 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.128952  6376 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:53.087113  8943 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:53.060303  9210 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.140796  7332 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:53.157600  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.172068  6308 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:53.172586  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1649) took 1065 ms. Trace:
I20260709 12:25:53.172888  6308 rpcz_store.cc:276] 0709 12:25:52.132168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.147337 (+1015169us) service_pool.cc:224] Handling call
0709 12:25:53.172520 (+ 25183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.176306  6383 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:25:53.177263  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:53.177960  6383 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:53.177273  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 189181us
W20260709 12:25:53.149969  9261 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:25:53.131260  9176 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.188028  6383 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:53.188640  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2193) took 1170 ms. Trace:
I20260709 12:25:53.188894  6383 rpcz_store.cc:276] 0709 12:25:52.018017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.184830 (+1166813us) service_pool.cc:224] Handling call
0709 12:25:53.187861 (+  3031us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.188451 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.188600 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.150539  8908 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:53.175568  9260 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:25:53.193810  9171 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:53.194272  9152 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.644s	user 0.007s	sys 0.004s
I20260709 12:25:53.214012  9152 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.196067  9084 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:25:53.196414  9233 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:53.197777  6451 raft_consensus.cc:1217] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[7.7-7.7]   Dedup: 7.7->[]
I20260709 12:25:53.270874  6451 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
W20260709 12:25:53.200332  9219 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.894s	user 0.009s	sys 0.001s
W20260709 12:25:53.200418  9091 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.948s	user 0.016s	sys 0.000s
I20260709 12:25:53.271600  9091 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.162161  9211 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.272215  6451 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:53.208209  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.274212  7331 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:53.181375  8943 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:53.229230  9231 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.229925  6378 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:53.187882  9176 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.210254  8891 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.215610  8918 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.243057  9255 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.255404  6385 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
W20260709 12:25:53.257001  9189 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.589s	user 0.007s	sys 0.001s
I20260709 12:25:53.303117  9189 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.304032  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1742) took 1238 ms. Trace:
I20260709 12:25:53.304538  6385 rpcz_store.cc:276] 0709 12:25:52.065351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.200941 (+135590us) service_pool.cc:224] Handling call
0709 12:25:53.303835 (+1102894us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003d36d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.303964 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2422821632}
I20260709 12:25:53.304626  8943 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.264992  7314 raft_consensus.cc:1217] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Deduplicated request from leader. Original: 5.6->[7.7-7.7]   Dedup: 7.7->[]
W20260709 12:25:53.267251  9269 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:25:53.210951  9210 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.130198  9188 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.267627  9179 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.271576  9219 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.202896  6383 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:53.160259  9254 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.274734  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1666) took 1005 ms. Trace:
I20260709 12:25:53.325654  6308 rpcz_store.cc:276] 0709 12:25:52.269211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.186598 (+917387us) service_pool.cc:224] Handling call
0709 12:25:53.274659 (+ 88061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.209261  6382 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
W20260709 12:25:53.259780  9227 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.005s
I20260709 12:25:53.305433  9189 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.307231  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1825) took 1794 ms. Trace:
I20260709 12:25:53.307431  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2195) took 1087 ms. Trace:
I20260709 12:25:53.264416  9152 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.333894  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.310725  9058 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:53.281528  9170 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.323535  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2194) took 1104 ms. Trace:
I20260709 12:25:53.340286  6383 rpcz_store.cc:276] 0709 12:25:52.219667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.202136 (+982469us) service_pool.cc:224] Handling call
0709 12:25:53.202719 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.323296 (+120577us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.323487 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.341140  6450 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:25:53.323225  7314 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:53.330680  6385 rpcz_store.cc:276] 0709 12:25:52.219997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.305751 (+1085754us) service_pool.cc:224] Handling call
0709 12:25:53.306118 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.306342 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.306370 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.306423 (+    53us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:53.306465 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.307266 (+   801us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.307379 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.306851  8900 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.307269  8905 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.334707  6308 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.268714  9084 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:53.329854  6310 rpcz_store.cc:276] 0709 12:25:51.513285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.519645 (+  6360us) service_pool.cc:224] Handling call
0709 12:25:51.519954 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.520603 (+   649us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:51.753107 (+232504us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.306627 (+1553520us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b18002ecf00. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a
0709 12:25:53.306851 (+   224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.306900 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:53.306954 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.307043 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.307164 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":414511,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":7382,"raft.queue_time_us":1461,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4520,"spinlock_wait_cycles":7602048,"thread_start_us":501041,"threads_started":2,"wal-append.queue_time_us":339534}
I20260709 12:25:53.347487  6383 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:53.332806  9172 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.354476  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.355350  6310 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.350175  8900 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.367599  6308 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:53.347649  9168 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.334479  9227 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.376187  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.377542  9263 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:53.378758  9176 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.381361  9176 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.377077  6308 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:53.388273  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1661) took 1184 ms. Trace:
I20260709 12:25:53.388772  6308 rpcz_store.cc:276] 0709 12:25:52.203549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.375290 (+1171741us) service_pool.cc:224] Handling call
0709 12:25:53.388202 (+ 12912us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.396723  9254 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.400871  6457 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 12:25:53.401360  6457 pending_rounds.cc:85] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 5
I20260709 12:25:53.401598  6457 pending_rounds.cc:107] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:25:53.401829  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1855) took 1884 ms. Trace:
I20260709 12:25:53.402796  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.403530  6305 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
W20260709 12:25:53.404103  6305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 333) took 1609 ms (client timeout 1577 ms). Trace:
W20260709 12:25:53.404476  6305 rpcz_store.cc:269] 0709 12:25:51.794459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.915042 (+120583us) service_pool.cc:224] Handling call
0709 12:25:53.402635 (+1487593us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:53.403962 (+  1327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22661376}
I20260709 12:25:53.406579  8908 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:53.407059  9143 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:53.358884  8937 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.401889  6457 raft_consensus.cc:2889] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:53.407182  8908 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.402350  6455 rpcz_store.cc:276] 0709 12:25:51.517702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.222727 (+705025us) service_pool.cc:224] Handling call
0709 12:25:52.239478 (+ 16751us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:52.240074 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.240122 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.250627 (+ 10505us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.451237 (+200610us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:52.494993 (+ 43756us) log.cc:844] Serialized 14 byte log entry
0709 12:25:52.495237 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:52.495332 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.398899 (+903567us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800343660. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:25:53.401488 (+  2589us) raft_consensus.cc:1685] finished
0709 12:25:53.401589 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.401739 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8179,"prepare.queue_time_us":43757,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":56,"raft.queue_time_us":22311,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":19246,"spinlock_wait_cycles":42384512,"thread_start_us":146910,"threads_started":1,"wal-append.queue_time_us":302860}
W20260709 12:25:53.406584  9180 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.135s	user 0.004s	sys 0.017s
I20260709 12:25:53.386859  9084 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.418211  9227 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.419261  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.420027  6308 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:53.420459  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1831) took 1498 ms. Trace:
I20260709 12:25:53.420874  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:53.420744  6308 rpcz_store.cc:276] 0709 12:25:51.921859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.412338 (+1490479us) service_pool.cc:224] Handling call
0709 12:25:53.420409 (+  8071us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.444873  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.445881  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:53.377045  9238 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.634s	user 0.004s	sys 0.009s
I20260709 12:25:53.410308  8891 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.418243  8908 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.449173  6305 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:53.452175  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1652) took 1318 ms. Trace:
I20260709 12:25:53.452606  6305 rpcz_store.cc:276] 0709 12:25:52.133433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.415594 (+1282161us) service_pool.cc:224] Handling call
0709 12:25:53.452107 (+ 36513us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:53.453680  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.459270  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.460119  6308 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:53.464552  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.465222  6305 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.450538  9158 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.455794  9084 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:25:53.497419  9272 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:25:53.462730  8908 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:53.456290  9238 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.513510  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2177) took 2325 ms. Trace:
W20260709 12:25:53.499174  9270 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:25:53.518442  6378 rpcz_store.cc:276] 0709 12:25:51.187916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.538585 (+1350669us) service_pool.cc:224] Handling call
0709 12:25:52.538942 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.280537 (+741595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.513268 (+232731us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b540023f0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:53.513448 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1555611520}
I20260709 12:25:53.458144  8891 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.524507  6305 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:53.525246  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1839) took 1282 ms. Trace:
I20260709 12:25:53.525701  6305 rpcz_store.cc:276] 0709 12:25:52.242868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.453395 (+1210527us) service_pool.cc:224] Handling call
0709 12:25:53.525165 (+ 71770us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.522071  6308 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:53.483206  9219 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.526783  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1833) took 1577 ms. Trace:
I20260709 12:25:53.531055  6308 rpcz_store.cc:276] 0709 12:25:51.949859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.444807 (+1494948us) service_pool.cc:224] Handling call
0709 12:25:53.526723 (+ 81916us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.502266  8937 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.547149  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1838) took 1312 ms. Trace:
I20260709 12:25:53.547097  8937 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.547636  6302 rpcz_store.cc:276] 0709 12:25:52.234919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.475166 (+240247us) service_pool.cc:224] Handling call
0709 12:25:53.546974 (+1071808us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400062a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.547084 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1324888832}
I20260709 12:25:53.549695  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2197) took 1329 ms. Trace:
I20260709 12:25:53.550128  6381 rpcz_store.cc:276] 0709 12:25:52.220662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.249717 (+ 29055us) service_pool.cc:224] Handling call
0709 12:25:53.549497 (+1299780us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.549632 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803065088}
W20260709 12:25:53.515663  9201 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.860s	user 0.000s	sys 0.003s
W20260709 12:25:53.533758  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1732) took 2249 ms (client timeout 2060 ms). Trace:
W20260709 12:25:53.534026  9146 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.004s
I20260709 12:25:53.535643  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1828) took 1625 ms. Trace:
I20260709 12:25:53.571866  6308 rpcz_store.cc:276] 0709 12:25:51.910549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.532298 (+1621749us) service_pool.cc:224] Handling call
0709 12:25:53.535141 (+  2843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.535331 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.535358 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.535404 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:53.535435 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.535492 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.535588 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.573415  6307 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:53.573873  6308 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:53.537032  6380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1740) took 2209 ms (client timeout 1579 ms). Trace:
W20260709 12:25:53.574946  6380 rpcz_store.cc:269] 0709 12:25:51.327776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.458546 (+130770us) service_pool.cc:224] Handling call
0709 12:25:53.536807 (+2078261us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b5400038c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.536937 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1489159936}
I20260709 12:25:53.503100  9242 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.575640  9091 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.577862  9242 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.514833  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 600) took 1767 ms. Trace:
I20260709 12:25:53.579692  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1656) took 1402 ms. Trace:
I20260709 12:25:53.580364  6308 rpcz_store.cc:276] 0709 12:25:52.172187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.573165 (+1400978us) service_pool.cc:224] Handling call
0709 12:25:53.573604 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.574341 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.574615 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.581044  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1851) took 2064 ms. Trace:
I20260709 12:25:53.539255  6378 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:53.508819  8900 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:53.540145  9223 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.001s
I20260709 12:25:53.581453  6452 rpcz_store.cc:276] 0709 12:25:51.516698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.054701 (+538003us) service_pool.cc:224] Handling call
0709 12:25:52.054954 (+   253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:52.245507 (+190553us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:52.245542 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:52.250920 (+  5378us) log.cc:844] Serialized 52 byte log entry
0709 12:25:52.600695 (+349775us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:53.580504 (+979809us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.580733 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.580805 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.580843 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:53.580855 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.580986 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":979403,"raft.queue_time_us":36561,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":460520,"threads_started":2,"wal-append.queue_time_us":343985}
I20260709 12:25:53.541163  8891 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.542234  9238 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:53.542448  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.583057  9223 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.582020  6308 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:25:53.538398  6310 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:53.585129  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1657) took 1409 ms. Trace:
W20260709 12:25:53.544423  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1658) took 1361 ms (client timeout 1767 ms). Trace:
I20260709 12:25:53.585953  9223 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.587059  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.586603  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.585497  6308 rpcz_store.cc:276] 0709 12:25:52.175534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.581374 (+1405840us) service_pool.cc:224] Handling call
0709 12:25:53.581779 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.584868 (+  3089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.585071 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.589375  6308 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:25:53.589952  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1660) took 1386 ms. Trace:
I20260709 12:25:53.590232  6308 rpcz_store.cc:276] 0709 12:25:52.203234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.588918 (+1385684us) service_pool.cc:224] Handling call
0709 12:25:53.589208 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.589719 (+   511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.589881 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.591481  6308 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 12:25:53.563786  9188 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.592031  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1662) took 1388 ms. Trace:
I20260709 12:25:53.592391  6308 rpcz_store.cc:276] 0709 12:25:52.203857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.590911 (+1387054us) service_pool.cc:224] Handling call
0709 12:25:53.591211 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.591806 (+   595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.591979 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.586129  6310 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:53.596089  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.596746  6456 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.600240  6308 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:25:53.551491  6379 rpcz_store.cc:269] 0709 12:25:51.284683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.530675 (+245992us) service_pool.cc:224] Handling call
0709 12:25:53.533565 (+2002890us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b5400195ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.533671 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937438592}
I20260709 12:25:53.549566  8937 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.505975  9224 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.569471  9158 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.578609  7316 rpcz_store.cc:276] 0709 12:25:51.747106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.747959 (+   853us) service_pool.cc:224] Handling call
0709 12:25:53.283979 (+1536020us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182eee 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:25:53.284167 (+   188us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.284589 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.284623 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:53.285272 (+   649us) log.cc:844] Serialized 52 byte log entry
0709 12:25:53.374192 (+ 88920us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:53.375792 (+  1600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.408361 (+ 32569us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.511989 (+103628us) spinlock_profiling.cc:243] Waited 1.56 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:53.514609 (+  2620us) raft_consensus.cc:1685] finished
0709 12:25:53.514635 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.514776 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1291,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"spinlock_wait_cycles":64111616,"thread_start_us":71209,"threads_started":1,"wal-append.queue_time_us":135137}
W20260709 12:25:53.555284  9165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.958s	user 0.016s	sys 0.023s
W20260709 12:25:53.635804  9165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.016s	sys 0.023s
I20260709 12:25:53.639104  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 592) took 2354 ms. Trace:
I20260709 12:25:53.639603  7319 rpcz_store.cc:276] 0709 12:25:51.284589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.284773 (+   184us) service_pool.cc:224] Handling call
0709 12:25:51.285222 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.285711 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.285746 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.286372 (+   626us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.544860 (+258488us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:51.544926 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.545030 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.515137 (+1970107us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:53.638721 (+123584us) raft_consensus.cc:1685] finished
0709 12:25:53.638756 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.639007 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":929,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":127,"spinlock_wait_cycles":117259648,"thread_start_us":257967,"threads_started":1,"wal-append.queue_time_us":289232}
I20260709 12:25:53.641882  6452 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:25:53.586694  6304 rpcz_store.cc:269] 0709 12:25:52.183094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.382210 (+199116us) service_pool.cc:224] Handling call
0709 12:25:53.544226 (+1162016us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.544361 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525780864}
I20260709 12:25:53.643939  6304 pending_rounds.cc:85] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 1
I20260709 12:25:53.644397  6304 pending_rounds.cc:107] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:53.644718  6304 raft_consensus.cc:2889] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:53.532598  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:53.646034  9202 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 2.019s	user 0.006s	sys 0.006s
I20260709 12:25:53.646437  6305 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:53.647141  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1832) took 1723 ms. Trace:
I20260709 12:25:53.647537  6305 rpcz_store.cc:276] 0709 12:25:51.923557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.532227 (+1608670us) service_pool.cc:224] Handling call
0709 12:25:53.647057 (+114830us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.648289  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:53.648842  6305 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:25:53.649329  9175 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:53.649480  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1837) took 1425 ms. Trace:
I20260709 12:25:53.649930  6305 rpcz_store.cc:276] 0709 12:25:52.224582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.648181 (+1423599us) service_pool.cc:224] Handling call
0709 12:25:53.648494 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.649217 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.649421 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:53.611548  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 165715us
I20260709 12:25:53.586710  7320 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:53.633260  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.654605  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:53.654767  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:53.658051  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:25:53.658689  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.660504  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.661664  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.662691  9219 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.663463  9219 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:53.667711  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.668406  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.672016  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.672932  6458 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.648408  6456 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:53.601838  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1659) took 1392 ms. Trace:
I20260709 12:25:53.674435  6310 rpcz_store.cc:276] 0709 12:25:52.202840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.585678 (+1382838us) service_pool.cc:224] Handling call
0709 12:25:53.585975 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.594745 (+  8770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.594988 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.650647  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 427) took 3424 ms. Trace:
I20260709 12:25:53.606505  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1663) took 1397 ms. Trace:
I20260709 12:25:53.677426  6308 rpcz_store.cc:276] 0709 12:25:52.204139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.593100 (+1388961us) service_pool.cc:224] Handling call
0709 12:25:53.593442 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.600828 (+  7386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.601034 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.651152  9158 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:25:53.651618  9191 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.376s	user 0.008s	sys 0.005s
W20260709 12:25:53.651669  6301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 331) took 1901 ms (client timeout 2120 ms). Trace:
W20260709 12:25:53.685487  6301 rpcz_store.cc:269] 0709 12:25:51.750249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.757810 (+  7561us) service_pool.cc:224] Handling call
0709 12:25:53.651476 (+1893666us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54000b3954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.651605 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064822656}
I20260709 12:25:53.686066  6308 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:53.652310  6454 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:53.652444  9224 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:53.653165  7319 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:53.684739  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:53.678979  6305 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:53.681294  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.689785  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1665) took 1464 ms. Trace:
I20260709 12:25:53.688256  9224 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.691108  6424 leader_election.cc:400] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:25:53.691349  9224 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:53.691962  9086 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:53.692564  9086 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:53.693416  9086 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.694994  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.691442  6424 leader_election.cc:403] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:53.675635  7317 rpcz_store.cc:276] 0709 12:25:50.226797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.227029 (+   232us) service_pool.cc:224] Handling call
0709 12:25:50.236403 (+  9374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.250669 (+1014266us) spinlock_profiling.cc:243] Waited 37.9 ms on lock 0x7b1800398380. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa329f060 00007f6fa329fbfb 00007f6fa32aa64b
0709 12:25:51.250863 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.250889 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.302009 (+ 51120us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.557753 (+255744us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:51.557811 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.531083 (+1973272us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54006595d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:53.531144 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.645431 (+114287us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:53.650425 (+  4994us) raft_consensus.cc:1685] finished
0709 12:25:53.650443 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.650592 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":187288,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":48878,"spinlock_wait_cycles":567532544,"thread_start_us":457014,"threads_started":2,"wal-append.queue_time_us":304803}
I20260709 12:25:53.690161  6305 rpcz_store.cc:276] 0709 12:25:52.225202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.653774 (+1428572us) service_pool.cc:224] Handling call
0709 12:25:53.678755 (+ 24981us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.689470 (+ 10715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.689726 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:53.708385  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:53.696630  8891 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:53.713061  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.714119  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:53.714982  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.715600  6454 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
W20260709 12:25:53.718091  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48903.1us
I20260709 12:25:53.719925  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1757) took 1074 ms. Trace:
I20260709 12:25:53.710856  9158 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 2. Current term is 3.
I20260709 12:25:53.703696  9228 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:25:53.711791  8891 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.705711  9223 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:25:53.726867  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.727854  6454 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.720651  6385 rpcz_store.cc:276] 0709 12:25:52.645923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.349560 (+703637us) service_pool.cc:224] Handling call
0709 12:25:53.357754 (+  8194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.357972 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:53.467152 (+109180us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.719197 (+252045us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.719440 (+   243us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:53.719508 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.719597 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.719850 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":146858,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":86,"thread_start_us":344916,"threads_started":2}
I20260709 12:25:53.734880  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:53.718034  9143 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:53.738881  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.744076  8918 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:53.726712  8891 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.745823  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:53.736667  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2319) took 1173 ms. Trace:
I20260709 12:25:53.738909  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:53.757015  6451 rpcz_store.cc:276] 0709 12:25:52.563403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.275530 (+712127us) service_pool.cc:224] Handling call
0709 12:25:53.275775 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.275970 (+   195us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:25:53.583484 (+307514us) log.cc:844] Serialized 14 byte log entry
0709 12:25:53.736226 (+152742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:53.736295 (+    69us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:53.736349 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.736436 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.736595 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312356,"prepare.queue_time_us":324808,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":7140,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":4665,"thread_start_us":152106,"threads_started":2}
I20260709 12:25:53.760967  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.769758  6458 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:53.771395  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 507) took 1442 ms. Trace:
I20260709 12:25:53.772010  7329 rpcz_store.cc:276] 0709 12:25:52.329168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.712715 (+383547us) service_pool.cc:224] Handling call
0709 12:25:52.717453 (+  4738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.936340 (+218887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.771065 (+834725us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b5400566154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:53.771320 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":501936384}
I20260709 12:25:53.772544  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.773319  6458 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:53.773675  9228 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:53.774739  9151 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:53.774657  6458 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:25:53.779076  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:53.782821  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:53.786320  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46437.9us
W20260709 12:25:53.787186  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.790073  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:53.793025  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.794275  6379 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:53.793625  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.796257  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.796586  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:53.797066  6381 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:53.799928  6385 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:53.800477  6383 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:53.801224  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:53.800529  6454 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:53.802779  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.802968  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.804536  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.762871  9091 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:53.805438  6455 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 5 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:25:53.808843  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.809149  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.809233  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.809937  6385 raft_consensus.cc:2376] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7. Re-sending same reply.
I20260709 12:25:53.767968  9143 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:53.809473  9289 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:53.807266  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.818723  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:53.806434  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.827364  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:53.767504  8937 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260709 12:25:53.805912  9238 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:53.851926  9238 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:53.817004  9151 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:25:53.853075  9238 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:53.854673  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1664) took 1650 ms. Trace:
I20260709 12:25:53.854950  6320 tablet_copy_source_session.cc:215] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:25:53.856161  6320 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:53970 (request call id 339) took 2039 ms. Trace:
I20260709 12:25:53.855145  6304 rpcz_store.cc:276] 0709 12:25:52.204422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.643042 (+1438620us) service_pool.cc:224] Handling call
0709 12:25:53.643574 (+   532us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:53.658893 (+ 15319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.658927 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:53.669494 (+ 10567us) log.cc:844] Serialized 52 byte log entry
0709 12:25:53.736863 (+ 67369us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:53.736904 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.736975 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:53.854359 (+117384us) raft_consensus.cc:1685] finished
0709 12:25:53.854398 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:53.854604 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5502,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":6385,"thread_start_us":31664,"threads_started":1,"wal-append.queue_time_us":143174}
I20260709 12:25:53.856549  6320 rpcz_store.cc:276] 0709 12:25:51.817304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.817541 (+   237us) service_pool.cc:224] Handling call
0709 12:25:53.854583 (+2037042us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b5400062a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32b13f7 00007f6fa8247036 00007f6fa8250a3e 00007f6f9fb1427b 00007f6fa82509ba 00007f6fa824bacc 00007f6fa8246c7b 00007f6fa823b8d1 00007f6fa72ef4b4 00007f6fa72ef442
0709 12:25:53.856073 (+  1490us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214576896}
I20260709 12:25:53.857280  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.859186  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:53.802171  9211 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:53.767212  9254 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:25:53.808053  9242 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:53.821926  9289 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:53.835737  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.836454  6385 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:53.839995  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:53.884253  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44333.9us
I20260709 12:25:53.841919  9179 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:53.842438  9248 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:53.845263  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.888063  6308 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:53.845888  9175 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:53.847234  6310 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:53.848235  6305 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:53.848699  9189 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:53.849630  9058 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:25:53.849853  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.807989  9091 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.864158  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.897440  6380 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:53.867108  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:25:53.881189  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:53.820272  9143 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:53.807528  9223 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:25:53.893208  6381 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:53.895430  6301 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:53.897964  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:53.919369  6304 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:25:53.854039  9224 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.868184  9211 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.863353  7318 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:53.887589  9263 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.925601  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:53.927947  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60935.7us
W20260709 12:25:53.931658  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112667us
I20260709 12:25:53.932168  6304 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:53.915627  9091 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.920972  9143 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:53.939558  6458 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:53.936051  8918 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:53.973322  8918 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:53.973845  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 514) took 1448 ms. Trace:
I20260709 12:25:53.974818  8918 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:53.977631  8898 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:53.944707  8891 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:53.993091  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:53.993600  9224 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:53.994783  8891 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:53.947135  9282 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:25:53.950989  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:53.993664  6452 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:53.953455  6301 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:53.974341  7319 rpcz_store.cc:276] 0709 12:25:52.525576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.641327 (+1115751us) service_pool.cc:224] Handling call
0709 12:25:53.641690 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.687958 (+ 46268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.973620 (+285662us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b540056f254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:53.973782 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25952512}
I20260709 12:25:53.953824  9219 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:53.955080  9155 ts_tablet_manager.cc:1916] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260709 12:25:53.921605  9211 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:53.979946  9281 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:53.982895  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:54.009408  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.009852  6456 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:54.010968  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:53.987313  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2302) took 2352 ms. Trace:
I20260709 12:25:54.011803  6450 rpcz_store.cc:276] 0709 12:25:51.634514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.321670 (+687156us) service_pool.cc:224] Handling call
0709 12:25:52.342205 (+ 20535us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.341569 (+999364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.986995 (+645426us) spinlock_profiling.cc:243] Waited 999 ms on lock 0x7b54000c12d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:53.987244 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2296824576}
I20260709 12:25:53.991178  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 1861) took 1933 ms. Trace:
I20260709 12:25:53.940523  9289 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.000250  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.014521  6304 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.011180  6456 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:54.014907  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:54.013000  6459 rpcz_store.cc:276] 0709 12:25:52.058071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.198983 (+140912us) service_pool.cc:224] Handling call
0709 12:25:53.990982 (+1791999us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:53.991110 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819340032}
I20260709 12:25:53.984375  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2316) took 1642 ms. Trace:
I20260709 12:25:54.016390  6456 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:25:54.016211  6457 rpcz_store.cc:276] 0709 12:25:52.341861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.034325 (+692464us) service_pool.cc:224] Handling call
0709 12:25:53.034625 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.410067 (+375442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.984046 (+573979us) spinlock_profiling.cc:243] Waited 366 ms on lock 0x7b54000eb354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:53.984301 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":841034624}
I20260709 12:25:54.019058  6379 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:25:54.020609  6385 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:54.020648  6459 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:53.994764  9292 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:25:54.009289  9281 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.027284  6457 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:54.027734  9281 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.029161  9281 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:54.029663  6459 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:54.030787  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:54.032188  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:54.033015  6300 raft_consensus.cc:2368] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6.
I20260709 12:25:54.033816  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:54.034869  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.035488  6301 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.025043  9289 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:54.042186  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.024050  6456 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:25:54.046424  6304 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:54.064697  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.049882  6452 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:54.047535  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.051782  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 437) took 1407 ms. Trace:
I20260709 12:25:54.072435  7319 rpcz_store.cc:276] 0709 12:25:52.644454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.020575 (+1376121us) service_pool.cc:224] Handling call
0709 12:25:54.046082 (+ 25507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.046297 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.046323 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:54.046362 (+    39us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:54.046402 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.046461 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.051723 (+  5262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:54.073251  6301 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:54.074689  6301 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:54.075730  6300 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:54.084499  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1728) took 2919 ms. Trace:
I20260709 12:25:54.068815  9080 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:54.097361  7319 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:54.055632  9143 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.056087  9211 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:54.113770  9268 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.850s	user 0.000s	sys 0.004s
I20260709 12:25:54.114784  9158 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:54.071336  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.128139  6377 rpcz_store.cc:276] 0709 12:25:51.154039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.558781 (+404742us) service_pool.cc:224] Handling call
0709 12:25:51.572410 (+ 13629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:52.041646 (+469236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.070452 (+2028806us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54003e7954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.070670 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1055080704}
W20260709 12:25:54.134033  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 161262us
I20260709 12:25:54.138096  9143 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:54.138458  9161 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.010s
I20260709 12:25:54.131556  9080 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:54.148783  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 602) took 2390 ms. Trace:
W20260709 12:25:54.151576  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.301314  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 438) took 1656 ms. Trace:
I20260709 12:25:54.301965  7319 rpcz_store.cc:276] 0709 12:25:52.644984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.086754 (+1441770us) service_pool.cc:224] Handling call
0709 12:25:54.087124 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.134122 (+ 46998us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.301243 (+167121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:54.260392  9242 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:54.254933  6452 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:54.304769  6452 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.305608  6452 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.306598  6452 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:54.138890  9091 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.305599  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:54.311753  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:54.312228  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1747) took 2246 ms (client timeout 2666 ms). Trace:
I20260709 12:25:54.312696  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:54.312779  6376 rpcz_store.cc:269] 0709 12:25:52.066218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.367676 (+301458us) service_pool.cc:224] Handling call
0709 12:25:54.312049 (+1944373us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b54001f54d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:54.312175 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1747694848}
I20260709 12:25:54.318168  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:54.319501  6376 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:54.320808  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:54.320940  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:54.322629  6385 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.328004  6378 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:54.309273  9189 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.308586  9158 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:25:54.341125  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.325162  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:54.310122  7314 rpcz_store.cc:276] 0709 12:25:51.758355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.761819 (+  3464us) service_pool.cc:224] Handling call
0709 12:25:51.902460 (+140641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.351961 (+1449501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.148461 (+796500us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400438ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.148715 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3123101440}
I20260709 12:25:54.327020  9080 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.332131  9143 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:54.323783  9238 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.334661  7318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 599) took 2593 ms (client timeout 1958 ms). Trace:
I20260709 12:25:54.335068  7319 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:54.335210  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:54.360474  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:54.335863  7316 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:54.336798  8905 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:54.337198  9184 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:25:54.338195  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:54.339351  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 325) took 2590 ms. Trace:
I20260709 12:25:54.342583  6451 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:25:54.318441  9281 raft_consensus.cc:2764] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 12:25:54.344484  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.380640  6459 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:54.344445  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:54.382848  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:54.385052  6452 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:25:54.386755  6452 pending_rounds.cc:85] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 8
I20260709 12:25:54.387184  6452 pending_rounds.cc:107] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:25:54.318204  8891 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:54.332672  9242 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.385695  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:54.362527  6378 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:54.364630  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:54.393267  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:54.365346  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.390436  6301 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:54.386639  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:54.324681  9219 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:25:54.365912  6304 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:54.371048  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:54.409094  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 503) took 2123 ms. Trace:
I20260709 12:25:54.379092  6308 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 2. Current term is 3. Ops: []
I20260709 12:25:54.373131  6307 rpcz_store.cc:276] 0709 12:25:51.748552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.757872 (+  9320us) service_pool.cc:224] Handling call
0709 12:25:51.802673 (+ 44801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.573848 (+1771175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.339081 (+765233us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54002d23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.339283 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4071535488}
I20260709 12:25:54.336056  9296 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:25:54.381788  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:54.386997  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:54.350951  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:54.387792  6452 raft_consensus.cc:2889] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:54.350421  6376 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:54.360841  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 440) took 1535 ms. Trace:
I20260709 12:25:54.424716  7319 rpcz_store.cc:276] 0709 12:25:52.825312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.325126 (+1499814us) service_pool.cc:224] Handling call
0709 12:25:54.334890 (+  9764us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.360606 (+ 25716us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.360787 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:54.369335  9293 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:25:54.335876  9292 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:25:54.389472  9258 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:54.389654  9195 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.402936  8937 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:25:54.364322  7318 rpcz_store.cc:269] 0709 12:25:51.740803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.744502 (+  3699us) service_pool.cc:224] Handling call
0709 12:25:53.051059 (+1306557us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:54.334385 (+1283326us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54005da2d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:54.334549 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":746640000}
I20260709 12:25:54.402293  9107 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:54.359238  9172 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:25:54.373411  9298 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:25:54.322057  9295 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:25:54.419786  9313 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:54.437712  9175 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:54.429554  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 78650.8us
I20260709 12:25:54.429433  6377 raft_consensus.cc:2368] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8.
W20260709 12:25:54.429250  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47521.6us
I20260709 12:25:54.409502  7335 rpcz_store.cc:276] 0709 12:25:52.285928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.286151 (+   223us) service_pool.cc:224] Handling call
0709 12:25:52.309200 (+ 23049us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.090506 (+781306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.408874 (+1318368us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.409034 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785435520}
I20260709 12:25:54.471964  9298 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:54.425503  9167 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:25:54.351266  9224 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:54.370466  8905 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.437566  6459 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:25:54.352245  9189 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:54.408120  6309 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1647) took 2312 ms (client timeout 1949 ms). Trace:
I20260709 12:25:54.472601  9298 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:54.324285  9297 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:25:54.429606  9254 raft_consensus.cc:2764] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:25:54.370172  9238 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.398689  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:54.500098  6310 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:54.436841  9058 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:54.441843  9195 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:54.463045  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 122373us
W20260709 12:25:54.468926  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.409751  8918 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 12:25:54.431396  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 93243.6us
I20260709 12:25:54.475670  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 509) took 2132 ms. Trace:
W20260709 12:25:54.480453  6309 rpcz_store.cc:269] 0709 12:25:52.095279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.386303 (+291024us) service_pool.cc:224] Handling call
0709 12:25:54.407907 (+2021604us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54001b2bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:54.408045 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476985728}
I20260709 12:25:54.442520  6307 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:54.509756  7331 rpcz_store.cc:276] 0709 12:25:52.343195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.132530 (+789335us) service_pool.cc:224] Handling call
0709 12:25:53.132914 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.281073 (+148159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.475340 (+1194267us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54005c1054. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.475591 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324457216}
W20260709 12:25:54.511802  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.481809  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94892.4us
W20260709 12:25:54.470428  9317 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:25:54.466711  9258 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:54.468143  9314 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:54.470398  9175 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:54.520820  9175 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:54.522979  9107 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:54.524471  9107 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.499842  9198 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:25:54.474341  9223 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.501533  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 81078.8us
I20260709 12:25:54.504501  7317 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:54.482848  8735 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:25:54.498148  9086 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:54.543092  8889 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:54.543670  8889 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:54.545940  8898 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
W20260709 12:25:54.506094  7320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 603) took 1884 ms (client timeout 1906 ms). Trace:
I20260709 12:25:54.502897  9058 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:54.510293  6307 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:54.482328  9149 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.010s	sys 0.015s
I20260709 12:25:54.537077  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:54.441831  9158 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.505877  9195 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:54.575594  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.548971  9175 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.542029  9242 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.591279  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 117360us
I20260709 12:25:54.533318  9219 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:54.533783  9311 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:54.595412  6384 raft_consensus.cc:2410] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:54.531121  8936 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.601711  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:54.596588  9058 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:54.588981  8918 consensus_queue.cc:1243] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:54.638059  8936 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:54.607944  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 97668.8us
I20260709 12:25:54.589460  9319 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 12:25:54.607733  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.549325  9223 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.612013  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:54.640379  6452 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:54.640443  8936 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:54.641633  9223 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:54.612459  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:54.614133  6385 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:54.603276  6377 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:54.643491  6424 leader_election.cc:400] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8.
I20260709 12:25:54.643850  6424 leader_election.cc:403] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
W20260709 12:25:54.646102  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:54.647743  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.643378  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:54.594475  9149 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.011s	sys 0.015s
I20260709 12:25:54.545994  8889 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:54.663251  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 192065us
W20260709 12:25:54.665000  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 69368.9us
I20260709 12:25:54.678401  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 506) took 2356 ms. Trace:
I20260709 12:25:54.595532  9263 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.607265  9320 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:54.662029  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:54.598914  9305 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.597858  6307 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.678825  7332 rpcz_store.cc:276] 0709 12:25:52.322424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.322602 (+   178us) service_pool.cc:224] Handling call
0709 12:25:52.322924 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.169192 (+846268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.665224 (+1496032us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b540070e4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.678333 (+ 13109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880319232}
I20260709 12:25:54.688833  6307 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.603664  9158 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:54.604804  9195 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:54.607244  9314 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:54.606709  9292 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:54.642814  9152 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.649715  9327 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:54.620335  9316 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:54.608218  9189 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:54.650509  9219 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.608404  9184 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:25:54.642584  9311 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.597990  9153 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.087s
I20260709 12:25:54.697947  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:54.677440  9313 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:25:54.660452  9242 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.602360  8937 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:54.693482  9263 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:54.704798  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50941.8us
I20260709 12:25:54.702903  9314 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:25:54.549121  7320 rpcz_store.cc:269] 0709 12:25:52.621451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.653608 (+ 32157us) service_pool.cc:224] Handling call
0709 12:25:54.505872 (+1852264us) spinlock_profiling.cc:243] Waited 886 ms on lock 0x7b5400567554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:54.506024 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2038793088}
I20260709 12:25:54.692667  9295 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8.
I20260709 12:25:54.696892  9107 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.695890  9195 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:54.720893  6377 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:54.698537  7332 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:25:54.694445  8936 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:25:54.702804  9292 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.620042  9238 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:54.705329  9327 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.708026  9316 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:54.710376  9152 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.697005  9305 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.027187  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:55.028007  6307 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:25:54.730212  8891 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:54.701890  9318 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.152s
I20260709 12:25:54.721613  9189 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:25:54.697347  9149 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:25:54.620663  9298 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.704485  9231 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:25:54.631093  9258 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:54.971076  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2305) took 3335 ms. Trace:
I20260709 12:25:55.048316  6456 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:55.016754  9297 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.020123  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:55.052417  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:55.021019  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:55.053444  6378 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:54.970324  8889 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.046661  9331 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.048485  6453 rpcz_store.cc:276] 0709 12:25:51.635287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.932763 (+1297476us) service_pool.cc:224] Handling call
0709 12:25:52.937122 (+  4359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.078891 (+141769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.734956 (+1656065us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000eae54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:54.970996 (+236040us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324477696}
W20260709 12:25:55.011092  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.018963  9321 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:55.052572  6377 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:25:55.057114  9238 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.059999  9231 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:55.039317  9023 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:25:55.037945  9155 ts_tablet_manager.cc:1929] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 4.3
I20260709 12:25:55.040007  8936 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:25:55.071694  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.124023  8898 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.124399  9291 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 12:25:55.083973  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.088003  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:55.126421  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:55.128044  7288 leader_election.cc:400] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6.
I20260709 12:25:55.128451  7288 leader_election.cc:403] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.129635  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:55.029541  9219 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.083495  8918 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.071051  9331 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.103339  9085 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.106832  9327 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.107457  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:55.138677  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.084846  9292 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.085050  9316 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.111591  9149 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:55.112947  9242 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.071478  8937 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:55.146502  7331 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.042s
I20260709 12:25:55.066458  9297 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.065385  9195 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.081341  9231 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:55.080993  9321 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.085430  9238 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:54.999846  9107 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:55.161957  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 188765us
I20260709 12:25:55.162036  6376 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:25:55.132707  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 401395us
I20260709 12:25:55.133234  6453 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:25:55.133313  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 457135us
I20260709 12:25:55.109469  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:55.165572  6383 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:55.124624  8898 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.125080  8936 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:55.142432  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.134315  9086 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:55.134040  9223 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.068009  8889 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.141933  9221 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.064927  9314 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.137262  9303 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:55.158236  7331 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.008s	sys 0.046s
I20260709 12:25:55.144497  9219 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.140645  9085 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.172363  6453 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
W20260709 12:25:55.161901  9058 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.522s	user 0.017s	sys 0.002s
I20260709 12:25:55.157850  9327 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.177759  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:55.159379  9242 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:55.172269  6459 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:55.157214  9292 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.137588  8918 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.171803  9149 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:25:55.260432  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 153044us
I20260709 12:25:55.182817  9263 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:55.182046  9155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2483d7801887403e873adc62429fe046. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:25:55.429453  6351 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.653s (SENT) [suppressed 41 similar messages]
I20260709 12:25:55.384819  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 518) took 1689 ms. Trace:
I20260709 12:25:55.436386  7331 rpcz_store.cc:276] 0709 12:25:53.695114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.641379 (+946265us) service_pool.cc:224] Handling call
0709 12:25:54.641685 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.641883 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:55.186820 (+544937us) log.cc:844] Serialized 14 byte log entry
0709 12:25:55.384235 (+197415us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.384322 (+    87us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:55.384399 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.384502 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.384726 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":553515,"raft.queue_time_us":14634,"thread_start_us":741436,"threads_started":2}
I20260709 12:25:55.416975  9242 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:55.488662  9085 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.488400  9242 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.472075  8936 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.490800  8936 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.319860  9231 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.491648  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.491976  9231 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.492300  6459 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
I20260709 12:25:55.486738  9349 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
W20260709 12:25:55.496984  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.393476  9086 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:55.497876  9086 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.498308  9086 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.193755  9221 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.395002  9149 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:55.518359  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 353875us
I20260709 12:25:55.515003  9342 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:55.532886  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.518085  9195 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.472702  9189 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:55.538437  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.553891  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.549078  6459 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:55.552482  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.393586  9058 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:55.560225  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 430676us
W20260709 12:25:55.565549  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.566108  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.562325  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.571848  6453 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:25:55.527230  9153 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:25:55.579686  6302 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:25:55.545030  9263 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.531369  9342 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:55.584250  6459 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:55.584694  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 516) took 2987 ms. Trace:
W20260709 12:25:55.589522  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1829) took 3664 ms (client timeout 2199 ms). Trace:
W20260709 12:25:55.593470  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.597797  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.598716  9168 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:55.599328  9168 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:55.591709  6303 rpcz_store.cc:269] 0709 12:25:51.914906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.414982 (+500076us) service_pool.cc:224] Handling call
0709 12:25:55.579224 (+3164242us) spinlock_profiling.cc:243] Waited 73.4 ms on lock 0x7b54002debd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:55.579343 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168742656}
I20260709 12:25:55.600337  9168 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:55.601083  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.598882  8898 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.591589  7317 rpcz_store.cc:276] 0709 12:25:52.597091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.782935 (+1185844us) service_pool.cc:224] Handling call
0709 12:25:53.805941 (+ 23006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.546393 (+740452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.584448 (+1038055us) spinlock_profiling.cc:243] Waited 698 ms on lock 0x7b54006c16d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:55.584628 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606047104}
I20260709 12:25:55.597302  6459 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:25:55.622370  9168 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:55.623617  9340 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.624204  9340 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.626689  9331 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:55.626354  9340 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:55.629436  9312 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:55.629788  9357 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.630133  9312 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:55.630524  9357 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.631173  9312 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.633351  9357 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.640437  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1887) took 1298 ms. Trace:
I20260709 12:25:55.640954  6456 rpcz_store.cc:276] 0709 12:25:54.342032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.343095 (+  1063us) service_pool.cc:224] Handling call
0709 12:25:54.343409 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.049048 (+705639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.640098 (+591050us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:55.640352 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620576512}
I20260709 12:25:55.648706  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.649643  6459 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:25:55.651221  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.551265  9338 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:25:55.651896  6456 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 3.
W20260709 12:25:55.651160  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86900us
I20260709 12:25:55.654824  9354 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.602433  9253 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.663734  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.667263  9312 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.667932  9325 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:55.700930  9325 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:55.669196  9189 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.629526  9297 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:55.675299  6456 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:25:55.679466  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:55.680759  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.589143  9153 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.602308  9263 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.719173  6456 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:25:55.701830  9325 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:55.724637  9276 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 2.149s	user 0.014s	sys 0.011s
I20260709 12:25:55.724784  9325 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:55.717497  9297 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.591223  9351 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.129s
I20260709 12:25:55.596637  9149 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.709846  9107 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:55.790557  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2196) took 3570 ms. Trace:
I20260709 12:25:55.796064  6382 rpcz_store.cc:276] 0709 12:25:52.220328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.327307 (+1106979us) service_pool.cc:224] Handling call
0709 12:25:53.327724 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.328208 (+   484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.328250 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.328802 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:25:53.443730 (+114928us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:53.443785 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:53.443885 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:55.720243 (+2276358us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:55.727523 (+  7280us) raft_consensus.cc:1685] finished
0709 12:25:55.727542 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.790446 (+ 62904us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":5996,"spinlock_wait_cycles":43496192,"thread_start_us":91486,"threads_started":1,"wal-append.queue_time_us":224629}
I20260709 12:25:55.815860  9153 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:55.816813  9352 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:25:55.815897  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 510) took 3453 ms. Trace:
I20260709 12:25:55.819094  7316 rpcz_store.cc:276] 0709 12:25:52.362225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.635151 (+1272926us) service_pool.cc:224] Handling call
0709 12:25:54.335611 (+700460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.377257 (+ 41646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.815646 (+1438389us) spinlock_profiling.cc:243] Waited 700 ms on lock 0x7b54005da2d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:55.815825 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1609994112}
I20260709 12:25:55.821126  9175 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.693424  9354 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.721526  8889 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.827673  8735 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:55.727262  9297 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.848942  9354 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.797408  9334 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.729475  9342 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.872673  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:55.803908  9321 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:55.861330  9219 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.000s
I20260709 12:25:55.703501  9253 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.884104  9086 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:55.887041  6384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1756) took 3241 ms (client timeout 2314 ms). Trace:
W20260709 12:25:55.889436  6384 rpcz_store.cc:269] 0709 12:25:52.645465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.024522 (+379057us) service_pool.cc:224] Handling call
0709 12:25:55.886755 (+2862233us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:55.886947 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3609496960}
I20260709 12:25:55.854992  8735 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:55.854899  9058 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:55.866523  9149 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:55.897454  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.865728  9338 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:55.898398  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 339559us
I20260709 12:25:55.865512  9221 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:55.904304  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:55.905001  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:55.888506  9342 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.913957  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:55.915122  6301 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:55.917047  6305 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:25:55.871917  9360 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:25:55.875962  9334 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:55.927712  8735 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:55.929342  8735 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:55.900759  9198 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:55.932827  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.930192  9334 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:55.907822  9351 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:55.961122  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:55.964664  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:55.969319  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.907575  9338 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:55.988873  6301 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:55.952965  9058 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260709 12:25:55.992681  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 388270us
W20260709 12:25:56.026903  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.027729  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.935776  9322 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:25:55.978412  9153 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.965539  9356 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260709 12:25:55.956014  9198 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:56.062723  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.042572  9210 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.047549  9175 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:56.067677  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 411187us
I20260709 12:25:56.049116  9263 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:56.075934  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.966688  9351 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.084053  9091 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:56.085640  9195 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.003s
I20260709 12:25:56.085804  9322 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.086108  9253 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.086308  9322 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:56.076059  9170 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.143s	user 0.024s	sys 0.041s
I20260709 12:25:56.102298  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:56.152637  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:56.100934  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:56.153808  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 52992.4us
I20260709 12:25:56.154552  7331 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:25:56.155224  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.155846  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:56.156769  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:56.157317  7332 raft_consensus.cc:2376] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
I20260709 12:25:55.983493  9324 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
W20260709 12:25:56.102564  6452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.005s	sys 0.007s
W20260709 12:25:56.158268  6452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.659s	user 0.007s	sys 0.011s
I20260709 12:25:56.123493  6301 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:25:56.097641  8936 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.004s
I20260709 12:25:56.060431  9153 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:56.126204  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.085650  9286 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 2.192s	user 0.009s	sys 0.007s
I20260709 12:25:56.129583  6309 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:56.130910  6305 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:25:56.168663  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 204083us
I20260709 12:25:56.086308  9195 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:56.088654  9297 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:56.124764  9332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.259s	user 0.043s	sys 0.062s
I20260709 12:25:56.135833  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:56.135082  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:56.074983  9210 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.125105  9351 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.128384  9086 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:56.074266  9362 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:25:56.143885  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:56.144941  9359 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:25:56.146394  9263 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:56.149075  9155 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.654s	user 0.003s	sys 0.008s
I20260709 12:25:56.174697  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:56.201951  6456 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.203460  9311 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:56.205562  9223 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.021s	user 0.011s	sys 0.000s
I20260709 12:25:56.206008  9227 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:56.206437  9227 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:25:56.207396  9227 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.208096  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.208693  9227 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.208999  6457 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:56.209918  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:56.213413  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.161696  9058 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.211647  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.214529  9058 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.215356  6457 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:56.215986  9208 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.216511  9208 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.216897  9058 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.218827  9314 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:56.219447  9312 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.510s	user 0.010s	sys 0.000s
I20260709 12:25:56.172891  9175 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:25:56.162230  9026 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:56.055343  9363 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:25:56.160545  7332 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:56.161922  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:56.166383  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.227505  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.167770  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:56.171275  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:56.172199  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:56.229230  6382 raft_consensus.cc:2376] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
W20260709 12:25:56.159158  9170 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.236s	user 0.027s	sys 0.043s
I20260709 12:25:56.231606  9170 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:25:56.232031  9170 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:56.232196  9371 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:25:56.233281  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1894) took 1088 ms. Trace:
I20260709 12:25:56.233726  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2215) took 1638 ms. Trace:
I20260709 12:25:56.233248  9170 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.181911  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:56.178557  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:56.234984  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56538.6us
I20260709 12:25:56.233973  6452 rpcz_store.cc:276] 0709 12:25:55.144679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.144777 (+    98us) service_pool.cc:224] Handling call
0709 12:25:55.145042 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.163784 (+ 18742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.163835 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:55.184206 (+ 20371us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.158670 (+974464us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:56.158712 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.158774 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.233086 (+ 74312us) raft_consensus.cc:1685] finished
0709 12:25:56.233106 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.233227 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14892,"raft.queue_time_us":19561,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":15847,"thread_start_us":659737,"threads_started":1,"wal-append.queue_time_us":678409}
W20260709 12:25:56.235881  9379 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.068s	user 0.002s	sys 0.000s
W20260709 12:25:56.187785  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.236081  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:56.236101  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.189771  6380 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:56.237660  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:56.238312  6452 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:56.234171  6376 rpcz_store.cc:276] 0709 12:25:54.595310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.595874 (+   564us) service_pool.cc:224] Handling call
0709 12:25:54.596375 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.162979 (+566604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.233460 (+1070481us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400064d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:56.233666 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1300235520}
I20260709 12:25:56.239456  9198 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:56.240046  9198 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:56.189908  6378 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:56.190196  9351 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:25:56.243163  9304 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.936s	user 0.012s	sys 0.010s
I20260709 12:25:56.242863  9351 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.190428  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:56.244534  6385 raft_consensus.cc:2376] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11. Re-sending same reply.
I20260709 12:25:56.245038  9322 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.245503  9026 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:56.245671  9322 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:56.246340  9026 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:56.172667  9374 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:56.190686  6377 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:25:56.176048  9332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.336s	user 0.043s	sys 0.063s
I20260709 12:25:56.175835  9356 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.016s
I20260709 12:25:56.191779  6383 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:56.192268  9263 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.149870  9107 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.178835  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:56.196365  8905 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.196916  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.359665  6453 raft_consensus.cc:2368] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6.
I20260709 12:25:56.199868  9238 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.201339  6303 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:56.308915  6454 raft_consensus.cc:1217] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 3.3->[4.4-4.4]   Dedup: 4.4->[]
I20260709 12:25:56.309034  7331 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:56.237737  9324 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:56.206019  9223 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6.
I20260709 12:25:56.317220  6380 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:25:56.202302  9231 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.001s
I20260709 12:25:56.318787  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1895) took 1172 ms. Trace:
I20260709 12:25:56.245090  9351 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:56.277974  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.309439  9210 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:56.320894  9170 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:56.366309  6455 rpcz_store.cc:276] 0709 12:25:55.145903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.146011 (+   108us) service_pool.cc:224] Handling call
0709 12:25:55.160456 (+ 14445us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:55.169050 (+  8594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.169088 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:55.169878 (+   790us) log.cc:844] Serialized 52 byte log entry
0709 12:25:55.786657 (+616779us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:56.045162 (+258505us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.045489 (+   327us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.045562 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.248893 (+203331us) raft_consensus.cc:1685] finished
0709 12:25:56.248911 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.318715 (+ 69804us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8060,"prepare.queue_time_us":356903,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":11873,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"spinlock_wait_cycles":25090944,"thread_start_us":623216,"threads_started":2,"wal-append.queue_time_us":726713}
I20260709 12:25:56.323215  9360 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:56.351284  9342 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.373972  9342 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.376092  9342 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.376675  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:56.377161  6380 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:56.377460  7316 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:56.352036  9378 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:56.361338  6454 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
W20260709 12:25:56.335532  9325 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.000s
I20260709 12:25:56.358510  7318 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:56.364699  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.347386  9363 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.369196  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.388612  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.348161  9161 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:25:56.371682  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:56.379908  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:56.380088  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:56.393370  6381 raft_consensus.cc:2376] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
I20260709 12:25:56.393985  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.394261  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.339372  9086 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:56.381508  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.350515  9107 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.396196  6380 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:25:56.387094  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.378237  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.394930  6381 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:56.397024  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.397500  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:56.398236  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.398993  6379 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:25:56.399999  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.390412  6459 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:25:56.400830  6459 pending_rounds.cc:85] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 4
I20260709 12:25:56.401063  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:56.401512  6459 pending_rounds.cc:107] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:56.401767  6459 raft_consensus.cc:2889] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:56.402308  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.403106  6381 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:56.405376  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.383630  9324 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:56.383280  9378 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:56.406850  6456 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:56.408108  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:56.412153  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:56.413504  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:56.415529  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:56.405527  8735 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 12:25:56.418794  9324 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.422684  9382 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:56.422819  9356 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:56.423448  9356 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:56.423628  9149 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:56.423902  9356 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.424201  9149 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.424718  9149 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.423380  9080 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:56.413056  9058 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:25:56.410919  9362 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:25:56.407213  9175 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:56.428323  9175 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:56.429180  9175 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:56.432359  9375 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:56.432319  9322 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:56.403228  9342 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:25:56.433853  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:56.435312  9332 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:56.435356  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:56.435916  9332 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.436321  9332 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.439064  9322 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:56.440544  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:56.402334  9189 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:56.443267  6459 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:56.446326  6272 leader_election.cc:400] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6.
I20260709 12:25:56.446949  6272 leader_election.cc:403] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:56.453179  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.448038  9322 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:56.455176  9322 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:56.455597  9322 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:56.395917  9363 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.441048  9263 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.458791  9363 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.460397  9107 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.423400  9382 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:56.463827  9307 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.156s	user 0.010s	sys 0.004s
I20260709 12:25:56.463431  9382 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.465380  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 300) took 2693 ms. Trace:
I20260709 12:25:56.467990  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:56.485379  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.462198  9058 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:25:56.465770  6450 rpcz_store.cc:276] 0709 12:25:53.772468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.012815 (+240347us) service_pool.cc:224] Handling call
0709 12:25:54.022441 (+  9626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:54.028189 (+  5748us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.028218 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:54.028686 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.159986 (+131300us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:54.160046 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.160146 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:56.463178 (+2303032us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:25:56.465102 (+  1924us) raft_consensus.cc:1685] finished
0709 12:25:56.465124 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.465319 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1633,"raft.queue_time_us":4247,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":20922752,"thread_start_us":130875,"threads_started":1,"wal-append.queue_time_us":279186}
I20260709 12:25:56.496703  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:56.503046  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:56.412217  9198 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 12:25:56.506825  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:56.454599  9189 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:56.515808  6379 raft_consensus.cc:2410] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:25:56.518198  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:56.519455  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:56.495888  9363 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:56.494349  9325 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:25:56.531719  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.504679  8735 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:56.540277  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 184608us
I20260709 12:25:56.534093  9324 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:25:56.515102  9210 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:56.522187  8918 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:56.532428  9172 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:56.552845  9393 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.560921  9393 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:56.565059  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61108us
I20260709 12:25:56.563258  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:56.567689  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:56.570658  8936 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:56.571440  8936 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:56.571852  8936 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:56.573089  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:56.573796  6380 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:56.559067  9170 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
W20260709 12:25:56.580325  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.538796  9325 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.560226  9172 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.570618  8957 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:56.562796  9210 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:25:56.586059  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.591179  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:56.592307  6301 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:56.539666  9189 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.603231  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:56.583238  9231 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:25:56.618770  6301 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:56.618870  9393 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:56.620373  9393 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:56.586767  8957 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.588725  9172 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:56.620812  9393 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.581771  9253 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:25:56.632894  6305 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:25:56.582281  8937 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:25:56.624799  8957 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:56.624261  9313 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.632762  9393 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:56.664327  9149 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.602231  9356 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:56.674520  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 80455.6us
W20260709 12:25:56.679996  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.633401  9161 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:25:56.648185  9080 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
W20260709 12:25:56.688056  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90248.8us
W20260709 12:25:56.680222  9384 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:56.667155  9313 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.703521  7329 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:56.704674  7331 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:25:56.687611  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:56.705888  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:56.661126  9210 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.724041  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:56.725471  7331 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:56.727018  9399 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:56.727767  9399 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:25:56.727998  9387 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:25:56.701164  9149 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.729007  9084 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:56.740288  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:56.737084  9399 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.756256  9084 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.767782  9208 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.768687  9084 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.729348  9325 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:56.777310  7329 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:56.778774  9179 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:56.779219  9179 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:56.780052  9179 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.781265  9179 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:56.783644  9356 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:56.784062  9309 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.479s	user 0.000s	sys 0.012s
I20260709 12:25:56.788056  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:56.788653  9058 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 5. Current term is 6.
I20260709 12:25:56.765785  8936 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.791386  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:56.793753  6378 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:56.739053  9324 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
W20260709 12:25:56.843904  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.844239  8957 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:56.847754  8936 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:56.847837  9252 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:25:56.881458  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.882159  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.882589  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.853108  9252 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:56.948119  9252 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:56.951256  9281 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:25:56.951768  9281 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:56.952593  9281 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:57.050697  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.190449  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.007422  9405 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
W20260709 12:25:57.031317  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.223906  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.181304  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.226805  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.229952  6451 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:25:56.970405  6378 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:25:57.006850  9406 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:25:57.263578  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.276284  6450 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:25:57.085943  9281 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:25:57.256065  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.972791  8957 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.082204  9313 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.217250  9341 raft_consensus.cc:476] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Not starting pre-election -- already a leader
W20260709 12:25:57.137476  9408 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 12:25:57.174731  9252 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.310577  9399 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.315568  9399 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.314128  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.320451  6450 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:57.085016  9172 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:57.314121  9416 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:25:57.306305  9143 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.318378  9399 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.322180  9324 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.324411  9343 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.349097  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2358) took 1015 ms. Trace:
I20260709 12:25:57.356211  6456 rpcz_store.cc:276] 0709 12:25:56.333798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.414593 (+ 80795us) service_pool.cc:224] Handling call
0709 12:25:56.414950 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.415330 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.415358 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.415862 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.702180 (+286318us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:56.702226 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.702342 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.348838 (+646496us) raft_consensus.cc:1685] finished
0709 12:25:57.348867 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.349019 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23367,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":105,"spinlock_wait_cycles":18688,"thread_start_us":263287,"threads_started":1,"wal-append.queue_time_us":300960}
I20260709 12:25:57.358467  6451 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:57.367323  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.373590  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:57.359158  9324 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:57.391619  9316 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.396000  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2336) took 1849 ms. Trace:
I20260709 12:25:57.396472  6454 rpcz_store.cc:276] 0709 12:25:55.547678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.547889 (+   211us) service_pool.cc:224] Handling call
0709 12:25:56.308783 (+760894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.382121 (+ 73338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.395747 (+1013626us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b5400344c50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:57.395926 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713751296}
W20260709 12:25:57.407234  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.361362  9343 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:57.416421  9322 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.960s	user 0.014s	sys 0.002s
W20260709 12:25:57.449612  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 391052us
I20260709 12:25:57.350433  9143 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:57.460372  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.452637  9332 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.016s	user 0.014s	sys 0.001s
I20260709 12:25:57.448285  9400 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.473378  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2239) took 1028 ms. Trace:
I20260709 12:25:57.475436  6380 rpcz_store.cc:276] 0709 12:25:56.444497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.574516 (+130019us) service_pool.cc:224] Handling call
0709 12:25:56.574841 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:56.575204 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.575227 (+    23us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:56.576083 (+   856us) log.cc:844] Serialized 52 byte log entry
0709 12:25:57.141056 (+564973us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:57.141121 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.141252 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.473060 (+331808us) raft_consensus.cc:1685] finished
0709 12:25:57.473098 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.473305 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":248,"thread_start_us":250759,"threads_started":1,"wal-append.queue_time_us":778751}
W20260709 12:25:57.453927  9179 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.003s
I20260709 12:25:57.474444  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:57.483103  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.483583  6456 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:57.493309  6380 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:57.493434  9172 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.495754  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2241) took 1050 ms. Trace:
I20260709 12:25:57.496353  9143 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.496201  6377 rpcz_store.cc:276] 0709 12:25:56.445038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.789921 (+344883us) service_pool.cc:224] Handling call
0709 12:25:56.790336 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.794757 (+  4421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.794805 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.826506 (+ 31701us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.335760 (+509254us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:57.335817 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.335918 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.495497 (+159579us) raft_consensus.cc:1685] finished
0709 12:25:57.495528 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.495683 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":223,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":2815,"thread_start_us":392238,"threads_started":1,"wal-append.queue_time_us":574697}
I20260709 12:25:57.442662  9322 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.505124  9322 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.473716  9289 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.506909  9289 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.489804  9400 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:57.507582  9343 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:57.479482  9332 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:57.509110  9289 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.510396  9332 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:57.473672  9324 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:57.512300  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.513142  6454 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:25:57.516584  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:57.517648  6380 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:57.521899  6377 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:57.522563  6377 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:57.510180  9400 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:57.530097  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:57.523763  6377 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.552273  6377 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:57.513165  9393 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.829s	user 0.007s	sys 0.004s
I20260709 12:25:57.559118  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:57.562855  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:57.563644  6378 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:25:57.564124  6455 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:57.565343  6383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.063s	sys 0.053s
W20260709 12:25:57.566769  6383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.063s	sys 0.053s
W20260709 12:25:57.571906  9412 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:25:57.579975  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:57.596606  6378 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:57.604888  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 280263us
I20260709 12:25:57.559084  9393 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.613200  6378 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:57.597404  9281 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.635406  9281 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.625380  6456 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:57.627053  6380 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:57.596534  9084 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:57.636732  9338 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:57.655069  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.675014  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.704535  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.634518  9400 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:25:57.630100  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:57.655689  9143 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:25:57.693526  6351 leader_election.cc:336] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.908s (SENT)
I20260709 12:25:57.696509  6377 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:57.699071  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 370) took 1087 ms. Trace:
I20260709 12:25:57.700063  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2243) took 1122 ms. Trace:
W20260709 12:25:57.710933  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.710336  6301 rpcz_store.cc:276] 0709 12:25:56.612052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.619647 (+  7595us) service_pool.cc:224] Handling call
0709 12:25:56.619932 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.620241 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.620268 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.620680 (+   412us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.828355 (+207675us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:56.828400 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.829071 (+   671us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.698816 (+869745us) raft_consensus.cc:1685] finished
0709 12:25:57.698840 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.699001 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2165,"raft.queue_time_us":41799,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":13516,"thread_start_us":206396,"threads_started":1,"wal-append.queue_time_us":739557}
W20260709 12:25:57.705644  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.713617  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:57.710453  6379 rpcz_store.cc:276] 0709 12:25:56.577715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.847151 (+269436us) service_pool.cc:224] Handling call
0709 12:25:56.847559 (+   408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.857649 (+ 10090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.857698 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.973403 (+115705us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.428152 (+454749us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:57.428212 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.428307 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.699777 (+271470us) raft_consensus.cc:1685] finished
0709 12:25:57.699808 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.699982 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9572,"raft.queue_time_us":9702,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":41095,"thread_start_us":152352,"threads_started":1,"wal-append.queue_time_us":388429}
W20260709 12:25:57.713855  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.712678  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 82650.1us
W20260709 12:25:57.717005  6351 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.928s (SENT)
I20260709 12:25:57.717519  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:57.718750  6351 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.937s (SENT)
I20260709 12:25:57.719082  6453 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:57.719381  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:57.707710  9418 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.111s	user 0.000s	sys 0.003s
I20260709 12:25:57.692488  9289 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.692279  9224 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.714098  9084 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.732295  9393 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.724932  9107 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:25:57.735596  6351 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.968s (SENT)
I20260709 12:25:57.718534  9400 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:57.725298  9341 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.738323  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:57.727483  9172 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:57.745581  6351 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.057s (SENT)
I20260709 12:25:57.737649  9289 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.746656  9170 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:57.732955  9210 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:25:57.750530  9289 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.720335  9143 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.752012  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:57.753774  9143 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.754709  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2240) took 1309 ms. Trace:
I20260709 12:25:57.755287  9322 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:57.756057  9322 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:57.755234  6383 rpcz_store.cc:276] 0709 12:25:56.444799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.608334 (+163535us) service_pool.cc:224] Handling call
0709 12:25:56.608638 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.850261 (+241623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.850307 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.926057 (+ 75750us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.588386 (+662329us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:57.588445 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.588557 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.754404 (+165847us) raft_consensus.cc:1685] finished
0709 12:25:57.754429 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.754612 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108180,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":875259,"threads_started":2,"wal-append.queue_time_us":783538}
I20260709 12:25:57.765834  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.767793  6454 raft_consensus.cc:2410] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:25:57.768466  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.769268  6456 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:57.757447  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1655) took 5595 ms. Trace:
I20260709 12:25:57.770642  6302 rpcz_store.cc:276] 0709 12:25:52.162116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.548566 (+1386450us) service_pool.cc:224] Handling call
0709 12:25:55.579445 (+2030879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:55.580113 (+   668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.757217 (+2177104us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b54002debd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:57.757353 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4669793664}
I20260709 12:25:57.772634  6459 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:57.774775  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.775542  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:57.783447  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:57.784497  6452 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
W20260709 12:25:57.786152  6351 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.199s (SENT)
I20260709 12:25:57.786912  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:57.742669  9400 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.788717  9281 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:57.737169  9378 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:57.789588  9400 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:57.789839  9252 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:57.747731  9341 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:57.790664  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.790581  9432 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.791317  9369 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.624s	user 0.011s	sys 0.004s
I20260709 12:25:57.791658  9432 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:57.794670  9393 consensus_queue.cc:1175] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 6 status { last_received { term: 6 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 6 index: 4 } }
I20260709 12:25:57.763199  9305 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:57.788082  9303 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.796808  9303 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.737149  9224 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:57.800467  9224 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:57.804464  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:57.811200  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.818329  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:57.819005  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:57.819949  9349 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:57.820482  9349 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.757342  9322 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:57.820633  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:57.824221  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:58.153676  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.118682  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.186228  6456 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:58.187036  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 362897us
W20260709 12:25:58.112780  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 80526.8us
W20260709 12:25:58.191570  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.227758  6424 leader_election.cc:336] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.314s (SENT)
W20260709 12:25:58.192255  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.202535  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.240060  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:58.240990  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.764483  8905 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:25:57.891477  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.246606  6380 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
W20260709 12:25:58.215046  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 428904us
I20260709 12:25:58.248060  6379 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:58.246621  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.251128  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.219874  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.254923  7319 raft_consensus.cc:2376] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
W20260709 12:25:58.255717  7319 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 604) took 3910 ms (client timeout 2088 ms). Trace:
W20260709 12:25:58.256585  7319 rpcz_store.cc:269] 0709 12:25:54.345164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.425758 (+ 80594us) service_pool.cc:224] Handling call
0709 12:25:58.219714 (+3793956us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:58.255646 (+ 35932us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43446016}
I20260709 12:25:58.259004  6456 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:58.259356  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.260681  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.261502  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:58.261864  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.262019  6378 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:25:58.262356  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.262815  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.263137  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.263873  6301 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:25:58.261263  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 461904us
I20260709 12:25:58.267181  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:58.266996  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.268512  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:58.269006  6456 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:58.269708  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:57.957041  9433 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:25:57.854848  9252 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.875185  9393 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.272564  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:58.270551  6379 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:25:58.123657  9311 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:58.218770  9378 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.286053  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.241364  9305 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.291999  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.254109  9428 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.264302  9440 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.278163  6383 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:25:58.296975  6308 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:25:58.297422  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.279527  6456 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:25:58.144601  9374 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:57.820964  9349 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.281649  6380 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 9. Ops: []
I20260709 12:25:58.281540  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.110733  9303 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:57.856221  9211 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:58.282181  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.241616  9426 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:58.283564  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.284134  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.221177  9343 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.221025  9363 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:25:58.267940  6424 connection.cc:256] RPC call timeout handler was delayed by -1.0066308959999333s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2666s for a response
I20260709 12:25:58.254531  9328 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.238615  9432 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.274793  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.295336  9253 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:57.876282  9153 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:58.346741  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.319325  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.255000  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:25:58.275169  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.275676  9295 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:58.282230  9326 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
W20260709 12:25:58.294852  9435 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 12:25:58.295840  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.297889  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.292299  9393 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.299314  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.359484  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.264123  9439 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:58.271515  9281 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.302241  7335 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.304347  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.367110  7329 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.305748  6378 raft_consensus.cc:2410] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:58.307997  6379 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:58.313294  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.299862  9305 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.292878  9155 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43752 (request call id 585) took 8123 ms. Trace:
I20260709 12:25:58.345453  7331 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:25:58.345717  9026 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:58.349509  6384 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:58.351856  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.310192  9426 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.306964  9362 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.285857  9252 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.357316  6308 raft_consensus.cc:2368] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9.
I20260709 12:25:58.266969  9224 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.357429  6301 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:58.360666  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.311699  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.369019  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.394188  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.302691  9440 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.347555  9253 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:58.373833  6383 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:58.301916  9428 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.305882  9374 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.386914  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.387969  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:58.388684  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.402807  6384 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:58.391963  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.393481  6380 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:58.394441  7331 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:58.403400  7329 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:58.351667  9363 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.315162  6305 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:58.405110  6300 raft_consensus.cc:2368] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8.
I20260709 12:25:58.405041  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.394659  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.406857  6379 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:58.407924  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.408609  6379 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:58.346045  9211 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:25:58.341153  9343 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.393146  7314 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.395043  6378 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:25:58.351302  9328 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:58.409988  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.410851  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:58.406330  6305 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:58.413697  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.389112  9426 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:58.415216  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.440145  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:58.440575  6453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2355) took 2112 ms (client timeout 1744 ms). Trace:
W20260709 12:25:58.441155  6453 rpcz_store.cc:269] 0709 12:25:56.327955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.364349 (+ 36394us) service_pool.cc:224] Handling call
0709 12:25:58.440338 (+2075989us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001e2dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:58.440484 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3069443328}
I20260709 12:25:58.356817  9295 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.445628  9295 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:58.446566  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:58.447114  6380 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:58.447458  9399 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:58.448000  9399 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:58.448333  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.448757  9341 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:58.423144  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:58.448840  9399 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.449482  9341 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:58.425391  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.432680  6456 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:25:58.433245  9303 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:25:58.451267  7318 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:58.437224  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.437615  9393 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.438153  9227 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:58.375402  9026 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:58.449043  7329 raft_consensus.cc:2410] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:25:58.449622  6383 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:58.374637  9281 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.454102  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.421579  7335 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:58.423167  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.446323  9295 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:58.458017  6378 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:58.370584  9439 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.459524  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2360) took 1233 ms. Trace:
I20260709 12:25:58.462868  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.463307  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.404265  7316 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.400889  9253 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.465068  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.472008  6453 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:58.498370  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:58.499212  6452 raft_consensus.cc:2376] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10. Re-sending same reply.
I20260709 12:25:58.466234  9084 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:58.466423  7314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 606) took 4120 ms (client timeout 2005 ms). Trace:
I20260709 12:25:58.467468  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.454876  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:25:58.502982  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 48209.4us
I20260709 12:25:58.470008  6301 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:58.504127  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.504799  7331 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.397760  9224 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.493485  6450 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:58.514065  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.514878  7335 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
W20260709 12:25:58.523218  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 254787us
I20260709 12:25:58.525151  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:58.473978  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.494158  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:25:58.526322  6451 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:58.526850  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.424269  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.391589  9362 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.501814  6300 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:58.527231  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.493039  6455 rpcz_store.cc:276] 0709 12:25:57.226258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.226428 (+   170us) service_pool.cc:224] Handling call
0709 12:25:58.459325 (+1232897us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b540009b354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:58.459468 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773356800}
I20260709 12:25:58.530944  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:58.531935  6383 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:58.532480  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.533357  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:58.534415  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:58.537889  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:58.528033  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 265197us
I20260709 12:25:58.529240  6379 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:25:58.471477  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 160188us
I20260709 12:25:58.535758  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.539729  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.539768  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:58.540069  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.540508  6379 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:58.540920  6305 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.541237  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:58.541602  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.542789  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:25:58.544003  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.544037  9084 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.544625  9084 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:58.545063  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.545825  6351 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.641s (SENT)
I20260709 12:25:58.546171  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:58.549263  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:58.550064  6455 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:58.551007  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.551860  6455 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:25:58.552906  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.553548  6455 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:25:58.556371  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.557471  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.562742  7319 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:58.566886  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.558321  6455 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:58.568909  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.569617  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:25:58.569957  6351 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.653s (SENT)
I20260709 12:25:58.570504  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:58.571681  7331 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:58.574193  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.574919  6450 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:25:58.575838  6308 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7. Re-sending same reply.
I20260709 12:25:58.529018  9295 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:58.579237  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.581931  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:58.584336  6455 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:58.587420  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.587740  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:58.505959  7314 rpcz_store.cc:269] 0709 12:25:54.345724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.387484 (+ 41760us) service_pool.cc:224] Handling call
0709 12:25:56.242093 (+1854609us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182eee 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:58.239405 (+1997312us) spinlock_profiling.cc:243] Waited 6.93 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182f8c 00007f6fa0182ef6 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:58.466328 (+226923us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24617088}
I20260709 12:25:58.547685  9026 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.596279  9305 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.596447  9026 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:58.375142  9155 rpcz_store.cc:276] 0709 12:25:50.163889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.313844 (+149955us) service_pool.cc:224] Handling call
0709 12:25:50.844909 (+531065us) ts_tablet_manager.cc:934] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
0709 12:25:53.865533 (+3020624us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:54.507799 (+642266us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:55.026378 (+518579us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:55.138067 (+111689us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 12:25:56.193090 (+1055023us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:56.193538 (+   448us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:56.419919 (+226381us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa01831a2 00007f6fa90c4889 00007f6fa81f5d88 00007f6fa82bc973 00007f6fa82b870f 00007f6fa82ba2d3 00007f6fa82ebf4e 00007f6fa82d0fdd 00007f6fa82d0f89
0709 12:25:58.277788 (+1857869us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:58.287454 (+  9666us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":51629,"spinlock_wait_cycles":70827904,"tablet-copy.queue_time_us":366289,"thread_start_us":303995,"threads_started":2}
I20260709 12:25:58.597637  9155 tablet_copy_client.cc:806] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:25:58.549930  9349 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.598165  9155 tablet_copy_client.cc:670] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:25:58.599385  6304 raft_consensus.cc:2368] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:25:58.582556  9326 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:58.537770  9426 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.608105  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.608930  7329 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.582908  9295 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:58.613857  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.614625  6300 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:25:58.615679  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.576650  9084 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:58.617093  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 197929us
I20260709 12:25:58.623979  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.626274  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:58.623237  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:58.635159  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 202921us
I20260709 12:25:58.610723  9357 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:58.636615  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 225892us
I20260709 12:25:58.640146  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:58.642632  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.643411  6300 raft_consensus.cc:2376] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11. Re-sending same reply.
I20260709 12:25:58.644160  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:25:58.631639  9407 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.912s	user 0.014s	sys 0.036s
I20260709 12:25:58.626223  9084 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.647804  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:58.648713  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 107572us
I20260709 12:25:58.653472  7331 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.654191  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:58.633056  9252 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.660933  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.661816  7331 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.662931  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.659552  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.664125  6300 raft_consensus.cc:2368] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9.
I20260709 12:25:58.665041  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.642647  9328 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:58.662818  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.667316  6308 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:58.666132  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.666193  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:58.668351  7316 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:58.662878  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.669788  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.636922  9428 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.668362  7331 raft_consensus.cc:2376] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
I20260709 12:25:58.658175  7329 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:58.680243  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.677297  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.651296  9305 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:58.652197  9407 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.933s	user 0.014s	sys 0.036s
I20260709 12:25:58.685249  7316 raft_consensus.cc:2376] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
I20260709 12:25:58.653864  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.699139  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:58.654795  9363 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.669548  9343 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.701960  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:58.710518  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84562.5us
I20260709 12:25:58.714798  7331 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.704751  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.716483  7316 raft_consensus.cc:2376] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
I20260709 12:25:58.683394  9328 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:58.718919  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.683991  9402 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.722441  7316 raft_consensus.cc:2376] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7. Re-sending same reply.
I20260709 12:25:58.684584  7329 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:58.639627  9446 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
W20260709 12:25:58.704007  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 164309us
I20260709 12:25:58.725533  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.726257  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:58.675011  9453 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:25:58.724658  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:58.690910  9428 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:58.698804  9227 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:58.731200  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:58.699239  9281 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:58.720903  9195 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.546s	user 0.015s	sys 0.024s
W20260709 12:25:58.700490  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 151293us
I20260709 12:25:58.701689  9349 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:58.776774  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50594us
I20260709 12:25:58.781226  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:58.786334  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55200.2us
W20260709 12:25:58.773500  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 119396us
I20260709 12:25:58.775321  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.791515  7329 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.794620  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:58.796808  7331 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:58.722224  9374 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:58.777256  9195 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.602s	user 0.018s	sys 0.027s
I20260709 12:25:58.722389  9224 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:58.772697  9227 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.737609  9439 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:58.775079  9328 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:25:58.799196  7314 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:58.794823  9281 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:58.775182  9453 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:58.793816  9428 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:58.776572  9454 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:58.803066  9195 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:58.785092  9362 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:58.823310  7329 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:25:58.744704  9401 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.026s	user 0.022s	sys 0.036s
I20260709 12:25:58.700263  9295 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:58.736814  9455 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 12:25:58.858952  7329 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:58.813115  9227 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.843894  9454 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.838299  9426 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:25:58.862911  9401 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.144s	user 0.028s	sys 0.051s
I20260709 12:25:58.821815  9328 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.822521  9253 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:58.874905  9448 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:25:58.875226  9453 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:58.874962  9447 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:25:58.864208  9428 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:58.864707  9468 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.869771  9295 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:58.879055  9446 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:58.881916  9446 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.882295  9446 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:58.883015  9468 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:58.878258  9328 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:58.887038  9457 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:58.888674  9468 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:58.889948  9378 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.593s	user 0.011s	sys 0.000s
I20260709 12:25:58.890729  9431 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:58.891405  9450 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:25:58.891314  9431 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.871430  9454 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:58.877545  9428 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:58.885807  9400 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.096s	user 0.010s	sys 0.004s
I20260709 12:25:59.009233  9400 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:58.942281  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 160919us
W20260709 12:25:58.943665  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 165836us
I20260709 12:25:58.945394  6305 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:58.947137  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 365) took 2362 ms. Trace:
I20260709 12:25:59.012445  6302 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 6. Ops: []
I20260709 12:25:59.019686  9195 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:58.813215  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:58.965914  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:59.011591  6310 rpcz_store.cc:276] 0709 12:25:56.584419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.584952 (+   533us) service_pool.cc:224] Handling call
0709 12:25:56.585298 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:56.585635 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.585666 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:58.420129 (+1834463us) spinlock_profiling.cc:243] Waited 3.31 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:25:58.436848 (+ 16719us) log.cc:844] Serialized 52 byte log entry
0709 12:25:58.607222 (+170374us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:58.607261 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.607335 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.945993 (+338658us) raft_consensus.cc:1685] finished
0709 12:25:58.946061 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.946780 (+   719us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1347,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3110,"spinlock_wait_cycles":7613184,"thread_start_us":169927,"threads_started":1,"wal-append.queue_time_us":265188}
W20260709 12:25:59.029227  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63770.1us
I20260709 12:25:59.030026  9252 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.031163  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.030952  9252 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:58.968431  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1910) took 1195 ms. Trace:
I20260709 12:25:58.969767  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1909) took 1197 ms. Trace:
I20260709 12:25:58.970534  7318 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:58.825567  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1911) took 1052 ms. Trace:
I20260709 12:25:58.972127  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:58.975211  7329 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:58.976078  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:58.982401  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:59.035192  9407 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.035861  9407 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:59.033437  6450 rpcz_store.cc:276] 0709 12:25:57.773093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.595144 (+822051us) service_pool.cc:224] Handling call
0709 12:25:58.613056 (+ 17912us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.613284 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:58.652530 (+ 39246us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.824747 (+172217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.824832 (+    85us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:58.825274 (+   442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.825350 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.825492 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3001,"prepare.queue_time_us":224359,"prepare.run_wall_time_us":49,"raft.queue_time_us":132645,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3517,"spinlock_wait_cycles":7819776,"thread_start_us":199721,"threads_started":3,"wal-append.queue_time_us":163750}
I20260709 12:25:58.985683  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:58.983392  8735 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.036381  9407 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.036792  8735 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.037540  8735 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:58.985735  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 289209us
I20260709 12:25:59.039150  6376 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:25:59.039600  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.041409  7332 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:58.988485  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.042866  9468 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:58.860438  9152 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:59.043630  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.044756  9474 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.045455  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.032217  6457 rpcz_store.cc:276] 0709 12:25:57.772851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.816776 (+ 43925us) service_pool.cc:224] Handling call
0709 12:25:57.817217 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.823596 (+  6379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.823808 (+   212us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.167667 (+343859us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.472062 (+304395us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:58.472096 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.472162 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.967608 (+495446us) raft_consensus.cc:1685] finished
0709 12:25:58.967651 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.968042 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3308,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"thread_start_us":173106,"threads_started":1,"wal-append.queue_time_us":370777}
I20260709 12:25:59.045509  9474 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:58.989918  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1679) took 3085 ms (client timeout 2161 ms). Trace:
W20260709 12:25:58.992978  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.042148  7332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 617) took 2869 ms (client timeout 2018 ms). Trace:
I20260709 12:25:58.884950  9305 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:59.047278  7332 rpcz_store.cc:269] 0709 12:25:56.172692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.225626 (+ 52934us) service_pool.cc:224] Handling call
0709 12:25:58.981887 (+2756261us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:25:59.042005 (+ 60118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53960448}
I20260709 12:25:59.048280  9474 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:25:59.000059  9311 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.000s
I20260709 12:25:59.049469  6381 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:59.050091  9221 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.050406  7319 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:59.051054  9221 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.051987  6307 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:25:59.053195  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.053340  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:59.054136  6307 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:25:59.054785  9152 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.055240  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.032598  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:59.042115  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:58.870195  9224 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.132s
I20260709 12:25:59.046370  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:59.033839  9252 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.031499  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:25:59.056157  6303 rpcz_store.cc:269] 0709 12:25:55.903862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.904042 (+   180us) service_pool.cc:224] Handling call
0709 12:25:58.989336 (+3085294us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540010f754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:58.989664 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679937280}
I20260709 12:25:59.054471  6376 raft_consensus.cc:606] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:59.065977  6351 leader_election.cc:400] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9.
I20260709 12:25:59.072595  6351 leader_election.cc:403] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:59.074900  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 275267us
I20260709 12:25:59.075610  9253 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:59.076840  9253 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:59.077453  9253 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:59.062788  6376 raft_consensus.cc:614] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:59.077771  9401 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9.
W20260709 12:25:59.078208  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.066717  9152 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:59.091022  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.091555  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.092015  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.092419  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.092646  9152 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.033087  6459 rpcz_store.cc:276] 0709 12:25:57.772607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.779312 (+  6705us) service_pool.cc:224] Handling call
0709 12:25:57.779683 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:57.789270 (+  9587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:57.789312 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.242389 (+453077us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.527756 (+285367us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:58.527794 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.527862 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.969154 (+441292us) raft_consensus.cc:1685] finished
0709 12:25:58.969193 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.969565 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1593,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":235892,"threads_started":1,"wal-append.queue_time_us":350739}
I20260709 12:25:59.094372  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.095281  6459 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:59.096359  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.097029  6459 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.066264  9426 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:25:59.099164  9426 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.100169  9311 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.067852  6457 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:59.100975  9311 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.101475  9153 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:59.101934  9153 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:59.070072  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.102357  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.103567  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.104154  9311 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:59.074015  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:59.105711  6310 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:25:59.085206  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:59.106313  6303 raft_consensus.cc:2376] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6. Re-sending same reply.
I20260709 12:25:59.106838  6376 raft_consensus.cc:606] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:25:59.107396  6376 raft_consensus.cc:614] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:59.099642  9453 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.108258  9311 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.108717  9453 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.109234  9311 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.111603  9453 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.111977  9311 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.111860  9253 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:59.118786  9395 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 2.433s	user 0.017s	sys 0.022s
I20260709 12:25:59.119378  9341 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:59.120299  9341 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.112926  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.113147  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:59.067178  9444 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.115155  6310 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:59.116206  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.085075  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:25:59.079007  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60176.9us
I20260709 12:25:59.109880  6303 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [5.4-6.5]
I20260709 12:25:59.106405  9407 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.146966  9439 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:25:59.147624  9439 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:59.148805  9407 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.149828  9407 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:59.150712  6376 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.151280  6376 raft_consensus.cc:614] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:59.151201  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 109290us
I20260709 12:25:59.152629  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:59.153141  9328 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.153689  9440 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:25:59.154918  6457 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:59.155275  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:25:59.156018  6303 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:59.156746  9252 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:59.157306  9252 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.157703  9252 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:59.158685  9328 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.158913  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:59.159824  6376 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.160233  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.161163  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:59.161893  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.163087  9305 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.163605  9305 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.163985  9305 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.164636  9468 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:59.164913  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.165944  6452 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:59.166779  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:25:59.166222  9468 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:59.131011  9341 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.131942  6456 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:25:59.132067  6307 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:59.168975  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.106842  9445 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:25:59.105721  9426 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:25:59.133431  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.161384  9444 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.163782  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:59.130393  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:59.162649  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.170225  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:25:59.098332  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.162338  6451 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:59.092350  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.172232  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:59.169369  9468 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:59.167552  6376 raft_consensus.cc:606] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.185859  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:59.185709  9153 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.187608  9153 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.187741  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.191844  6376 raft_consensus.cc:614] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:59.185981  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 391523us
I20260709 12:25:59.183568  9468 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:25:59.199676  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.108299  9231 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:59.201203  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:59.180210  9474 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:25:59.208950  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.213290  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.215737  8735 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:25:59.218055  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.219053  6304 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:25:59.205956  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.210718  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.223493  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.203284  9231 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:25:59.235419  6377 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:59.258457  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:25:59.255877  6382 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
W20260709 12:25:59.266623  9303 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.003s	sys 0.013s
W20260709 12:25:59.266865  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 106811us
W20260709 12:25:59.267159  9303 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.003s	sys 0.013s
I20260709 12:25:59.268177  6303 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:59.271927  6451 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:59.273394  6384 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:25:59.256436  9426 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:25:59.280447  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 304) took 1052 ms. Trace:
W20260709 12:25:59.282943  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95360.3us
I20260709 12:25:59.278267  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:59.280953  6451 rpcz_store.cc:276] 0709 12:25:58.220270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.196461 (+976191us) service_pool.cc:224] Handling call
0709 12:25:59.248740 (+ 52279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.272401 (+ 23661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.272563 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.287752  9434 raft_consensus.cc:2764] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:59.279141  9468 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.288916  9434 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:59.288993  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2382) took 1471 ms. Trace:
I20260709 12:25:59.289256  9468 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.289527  6459 rpcz_store.cc:276] 0709 12:25:57.817745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.181509 (+1363764us) service_pool.cc:224] Handling call
0709 12:25:59.235627 (+ 54118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.288544 (+ 52917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.288584 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.288634 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:59.288695 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.288770 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.288909 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.292515  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2383) took 1041 ms. Trace:
I20260709 12:25:59.293022  6459 rpcz_store.cc:276] 0709 12:25:58.250739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.290812 (+1040073us) service_pool.cc:224] Handling call
0709 12:25:59.291509 (+   697us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.292015 (+   506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.292056 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.292112 (+    56us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:59.292180 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.292276 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.292404 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:59.299139  9026 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.007s
I20260709 12:25:59.300720  6303 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:25:59.287664  9357 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.301369  6303 pending_rounds.cc:85] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 3
I20260709 12:25:59.301774  6303 pending_rounds.cc:107] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:25:59.300681  6300 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:59.301779  9481 raft_consensus.cc:2764] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:59.304880  9227 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.287998  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.302134  6303 raft_consensus.cc:2889] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:59.306665  9364 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:59.307309  9364 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:59.301935  9357 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.308719  9364 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.310246  6303 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:25:59.286147  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:25:59.319856  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1849) took 1123 ms. Trace:
I20260709 12:25:59.281143  9474 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.324043  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.320327  6302 rpcz_store.cc:276] 0709 12:25:58.190097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.190258 (+   161us) service_pool.cc:224] Handling call
0709 12:25:58.190648 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.013096 (+822448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.285943 (+272847us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54000aa854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:59.313597 (+ 27654us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1885452544}
W20260709 12:25:59.348845  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.354238  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.353333  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:25:59.352795  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85058.9us
I20260709 12:25:59.346315  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:59.358711  9393 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.647s	user 0.002s	sys 0.021s
W20260709 12:25:59.359406  9393 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.002s	sys 0.022s
I20260709 12:25:59.359460  9349 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:25:59.360467  9349 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.361728  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:59.358621  9431 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.348949  8735 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.365231  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:25:59.384770  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.386286  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.392105  6303 pending_rounds.cc:85] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 1
I20260709 12:25:59.392434  6303 pending_rounds.cc:107] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:25:59.392728  6303 raft_consensus.cc:2889] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:59.394101  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.395287  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 307) took 1143 ms. Trace:
I20260709 12:25:59.395536  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.395744  6459 rpcz_store.cc:276] 0709 12:25:58.251476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.294325 (+1042849us) service_pool.cc:224] Handling call
0709 12:25:59.345159 (+ 50834us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.394819 (+ 49660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.394851 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.394934 (+    83us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:59.394982 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.395071 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.395216 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.398214  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.391700  6451 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:59.366357  9357 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.405491  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 305) took 1185 ms. Trace:
I20260709 12:25:59.405880  6451 rpcz_store.cc:276] 0709 12:25:58.220729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.284437 (+1063708us) service_pool.cc:224] Handling call
0709 12:25:59.350036 (+ 65599us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.405244 (+ 55208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.405439 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:59.408226  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:25:59.408617  6377 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:25:59.415688  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:59.387501  9364 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:59.390290  9400 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.409344  6379 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:25:59.421430  6384 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:25:59.421566  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 258053us
I20260709 12:25:59.428102  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:25:59.430753  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.444319  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2381) took 1617 ms. Trace:
I20260709 12:25:59.444803  7329 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:25:59.445932  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2380) took 1620 ms. Trace:
I20260709 12:25:59.451920  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:25:59.452694  6378 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:25:59.453819  9349 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.399358  9253 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:59.454926  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:59.444881  6452 rpcz_store.cc:276] 0709 12:25:57.817494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.175916 (+1358422us) service_pool.cc:224] Handling call
0709 12:25:59.218798 (+ 42882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.299015 (+ 80217us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:59.338388 (+ 39373us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.429516 (+ 91128us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.429581 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:59.434588 (+  5007us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.434682 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.434832 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":52579,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":64,"raft.queue_time_us":13735,"thread_start_us":121830,"threads_started":2}
W20260709 12:25:59.455673  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 52433.1us
I20260709 12:25:59.461928  6380 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:25:59.463945  6310 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:25:59.470149  9434 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:59.439410  9444 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.439304  9227 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.449906  6456 rpcz_store.cc:276] 0709 12:25:57.817242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.168362 (+1351120us) service_pool.cc:224] Handling call
0709 12:25:59.168842 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.169205 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:25:59.259696 (+ 90491us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.436827 (+177131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.437035 (+   208us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:59.437185 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.437412 (+   227us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.437754 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16964,"prepare.queue_time_us":161393,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":6715,"raft.queue_time_us":23928,"thread_start_us":170942,"threads_started":2}
I20260709 12:25:59.477033  9434 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.487258  9434 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.487345  9313 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:59.490140  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.490665  9426 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:59.491364  9426 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:59.491537  9471 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:59.492471  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 228006us
I20260709 12:25:59.493449  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:25:59.496565  6378 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:25:59.456602  9155 tablet_copy_client.cc:538] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:25:59.444548  9305 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.509367  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.505039  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:59.498063  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:59.518342  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:59.520187  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:25:59.530247  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.540273  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.492726  9426 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:25:59.545769  6456 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:25:59.551586  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71699.4us
W20260709 12:25:59.554755  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 171019us
I20260709 12:25:59.526127  9311 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:25:59.570927  9311 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:25:59.572208  9311 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.568583  9434 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.546705  9252 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.576520  9252 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.579550  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:59.580585  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:59.576506  9434 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.559441  9486 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:59.560642  9490 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:25:59.562594  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:59.563010  8905 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.297s	user 0.007s	sys 0.014s
I20260709 12:25:59.562646  9341 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:25:59.563390  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1913) took 1252 ms. Trace:
W20260709 12:25:59.564622  9489 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:25:59.569036  9328 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.587172  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:59.600520  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47185.3us
I20260709 12:25:59.600613  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:59.601258  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:25:59.605762  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 106032us
I20260709 12:25:59.606122  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:59.587662  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1912) took 1814 ms. Trace:
I20260709 12:25:59.595547  6458 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:25:59.594889  9488 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.232s	user 0.003s	sys 0.000s
I20260709 12:25:59.638906  6450 rpcz_store.cc:276] 0709 12:25:57.773313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.038042 (+1264729us) service_pool.cc:224] Handling call
0709 12:25:59.039010 (+   968us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.040462 (+  1452us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:59.200830 (+160368us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.281679 (+ 80849us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.345861 (+ 64182us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.347691 (+  1830us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:59.347742 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.347840 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.587424 (+239584us) raft_consensus.cc:1685] finished
0709 12:25:59.587443 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.587577 (+   134us) inbound_call.cc:177] Queueing success response
0709 12:25:59.605389 (+ 17812us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b18001ab300. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
Metrics: {"prepare.queue_time_us":229522,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":27203,"raft.queue_time_us":1571,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":33792,"spinlock_wait_cycles":32056704,"thread_start_us":181498,"threads_started":2,"wal-append.queue_time_us":143409}
W20260709 12:25:59.614828  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94793.4us
I20260709 12:25:59.583734  9211 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.604383  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2378) took 1787 ms. Trace:
I20260709 12:25:59.647861  6457 rpcz_store.cc:276] 0709 12:25:57.816639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.103705 (+1287066us) service_pool.cc:224] Handling call
0709 12:25:59.104314 (+   609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.155397 (+ 51083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.604100 (+448703us) spinlock_profiling.cc:243] Waited 50.2 ms on lock 0x7b5400266e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:59.604319 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115453696}
I20260709 12:25:59.598520  9313 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:59.582448  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:25:59.654887  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 72503.1us
I20260709 12:25:59.647532  9211 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:59.657614  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103293us
I20260709 12:25:59.658773  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:25:59.588837  9305 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:25:59.584713  9486 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.598249  9153 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.628345  9328 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:59.591902  9454 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.697s	user 0.008s	sys 0.010s
W20260709 12:25:59.665889  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.696851  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.591570  9221 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.697284  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 627) took 1056 ms. Trace:
I20260709 12:25:59.698418  9155 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:25:59.697985  9454 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.700430  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.701086  6454 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.701515  9454 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.656165  8908 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.702759  8908 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:25:59.697307  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.705050  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.660480  9252 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:59.705950  9252 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.706533  9252 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.603821  9231 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.669305  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 338) took 1322 ms. Trace:
I20260709 12:25:59.669759  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.661326  9211 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.673207  9439 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.674686  9224 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.656193  9444 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:59.675269  9486 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.675631  9153 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.583343  9401 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:59.605579  9446 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.723s	user 0.015s	sys 0.002s
I20260709 12:25:59.677104  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.682282  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:25:59.683954  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:25:59.684930  9458 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:59.686563  9328 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:59.688374  9500 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:59.689121  9497 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:59.691273  6452 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:59.691785  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:25:59.624082  6453 rpcz_store.cc:276] 0709 12:25:58.251756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.307431 (+1055675us) service_pool.cc:224] Handling call
0709 12:25:59.307831 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.308275 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:59.370735 (+ 62460us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.504080 (+133345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:59.504152 (+    72us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:59.504216 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.504292 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.504425 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6564,"prepare.queue_time_us":45822,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":2015,"raft.queue_time_us":24697,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":12812,"thread_start_us":121146,"threads_started":2,"wal-append.queue_time_us":214532}
W20260709 12:25:59.693720  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87779.2us
I20260709 12:25:59.695767  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1884) took 1120 ms. Trace:
I20260709 12:25:59.695688  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.731894  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.676838  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.697850  7319 rpcz_store.cc:276] 0709 12:25:58.640369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.640502 (+   133us) service_pool.cc:224] Handling call
0709 12:25:58.667954 (+ 27452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.051030 (+383076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.697013 (+645983us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54003bcc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:59.697211 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878445952}
I20260709 12:25:59.714816  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:25:59.712357  6381 rpcz_store.cc:276] 0709 12:25:58.346908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.419815 (+ 72907us) service_pool.cc:224] Handling call
0709 12:25:58.420112 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.049978 (+629866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.668656 (+618678us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:59.669145 (+   489us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446015872}
I20260709 12:25:59.715701  9224 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:25:59.730393  6308 rpcz_store.cc:276] 0709 12:25:58.575034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.673490 (+ 98456us) service_pool.cc:224] Handling call
0709 12:25:59.695378 (+1021888us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b54002df0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:59.695580 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1608212352}
I20260709 12:25:59.721004  9401 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:59.740449  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 75792.6us
I20260709 12:25:59.719254  9153 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.743752  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.744383  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.723433  9253 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:59.750213  6455 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:59.739286  9224 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:59.756384  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.738849  9444 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:25:59.761235  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.758735  6457 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.755760  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:59.745745  9401 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.764369  7332 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:25:59.765175  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:59.745335  9497 consensus_queue.cc:1243] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:59.769862  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:25:59.733057  9446 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:25:59.777424  6453 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:59.777920  6456 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:59.774890  7335 raft_consensus.cc:2376] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
W20260709 12:25:59.783907  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 125240us
W20260709 12:25:59.785873  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 102167us
W20260709 12:25:59.788589  9494 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:25:59.791575  6452 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:25:59.784960  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1892) took 1126 ms. Trace:
I20260709 12:25:59.792065  6300 rpcz_store.cc:276] 0709 12:25:58.653394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.665875 (+ 12481us) service_pool.cc:224] Handling call
0709 12:25:59.769349 (+1103474us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54002d2dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:25:59.779398 (+ 10049us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453423488}
I20260709 12:25:59.792658  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:25:59.795118  6454 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:25:59.795399  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:25:59.796366  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.788256  9402 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:25:59.797966  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:25:59.800241  7319 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:59.776489  9313 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:25:59.851446  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.779858  9439 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:25:59.871507  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 84613.6us
I20260709 12:25:59.872999  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1919) took 1620 ms. Trace:
I20260709 12:25:59.873497  6451 rpcz_store.cc:276] 0709 12:25:58.252850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.406609 (+1153759us) service_pool.cc:224] Handling call
0709 12:25:59.430600 (+ 23991us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.457806 (+ 27206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.457837 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.458388 (+   551us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.534711 (+ 76323us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:59.534765 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.534871 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.872703 (+337832us) raft_consensus.cc:1685] finished
0709 12:25:59.872730 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.872916 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":75777,"threads_started":1,"wal-append.queue_time_us":167790}
I20260709 12:25:59.876497  6450 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:25:59.888935  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.889775  6454 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:25:59.891352  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:25:59.892613  6456 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:25:59.893676  9497 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.894441  9497 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.895246  9313 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.897626  9497 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:25:59.898784  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 371) took 1453 ms. Trace:
I20260709 12:25:59.899286  6305 rpcz_store.cc:276] 0709 12:25:58.445644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.248573 (+802929us) service_pool.cc:224] Handling call
0709 12:25:59.283434 (+ 34861us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.411537 (+128103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.411577 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.418504 (+  6927us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.717926 (+299422us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:59.717968 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.718039 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.898415 (+180376us) raft_consensus.cc:1685] finished
0709 12:25:59.898445 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.898675 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56345,"raft.queue_time_us":65689,"raft.run_cpu_time_us":1216,"raft.run_wall_time_us":101804,"thread_start_us":296627,"threads_started":1,"wal-append.queue_time_us":361732}
I20260709 12:25:59.903559  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:25:59.904520  6453 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
W20260709 12:25:59.916597  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 201943us
W20260709 12:25:59.917764  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 131845us
W20260709 12:25:59.921283  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:59.926033  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.925595  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 374) took 1145 ms. Trace:
W20260709 12:25:59.930219  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 58474.9us
I20260709 12:25:59.935078  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:25:59.939368  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 143990us
I20260709 12:25:59.941772  6453 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:59.928256  6304 rpcz_store.cc:276] 0709 12:25:58.780129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.309393 (+529264us) service_pool.cc:224] Handling call
0709 12:25:59.343995 (+ 34602us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:59.451671 (+107676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.451720 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:59.452621 (+   901us) log.cc:844] Serialized 78 byte log entry
0709 12:25:59.694598 (+241977us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b1800328240. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:25:59.750294 (+ 55696us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:59.914776 (+164482us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.915014 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.915087 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.915123 (+    36us) raft_consensus.cc:1685] finished
0709 12:25:59.925320 (+ 10197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.925510 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47767,"prepare.queue_time_us":35080,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2817,"raft.queue_time_us":67477,"spinlock_wait_cycles":139559040,"thread_start_us":201064,"threads_started":2,"wal-append.queue_time_us":449836}
W20260709 12:25:59.953055  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 153488us
I20260709 12:25:59.960889  9511 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:59.939199  9454 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:59.953954  9478 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:25:59.917407  9026 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:59.988830  6451 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:59.893888  9153 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:25:59.999290  9253 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:00.078601  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76472.8us
I20260709 12:26:00.079589  6453 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:25:59.991580  9511 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.076938  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:00.083500  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 309) took 1825 ms. Trace:
I20260709 12:26:00.091449  6451 rpcz_store.cc:276] 0709 12:25:58.257607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.912382 (+1654775us) service_pool.cc:224] Handling call
0709 12:25:59.983039 (+ 70657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.083188 (+100149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.083439 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.089087  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 106063us
W20260709 12:26:00.098928  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.099993  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:00.087319  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:00.101948  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 119720us
W20260709 12:26:00.104357  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 178764us
W20260709 12:26:00.105168  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 122585us
W20260709 12:26:00.088589  9364 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.001s
I20260709 12:26:00.109522  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:00.110198  6305 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.110831  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:00.112751  9305 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:00.114372  9362 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:00.114696  9153 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:00.104352  9509 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:26:00.115582  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2387) took 1667 ms. Trace:
I20260709 12:26:00.116027  6453 rpcz_store.cc:276] 0709 12:25:58.448021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.092802 (+1644781us) service_pool.cc:224] Handling call
0709 12:26:00.114656 (+ 21854us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.115185 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.115213 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.115295 (+    82us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:00.115330 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.115382 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.115509 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.117954  6453 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:00.118772  9231 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.120612  9362 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:00.121032  9253 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.122953  6453 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:26:00.123348  6453 pending_rounds.cc:85] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 6
I20260709 12:26:00.123631  6453 pending_rounds.cc:107] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:26:00.123896  6453 raft_consensus.cc:2889] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.126742  9446 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:00.129534  6454 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:00.130252  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 316) took 1855 ms. Trace:
I20260709 12:26:00.133960  6454 rpcz_store.cc:276] 0709 12:25:58.274909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.128870 (+1853961us) service_pool.cc:224] Handling call
0709 12:26:00.129286 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.129961 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.130190 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.138398  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57611.1us
I20260709 12:26:00.139441  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:00.142102  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.145294  6457 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:26:00.148317  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45868.4us
I20260709 12:26:00.095072  9511 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.102893  9407 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:00.150630  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:00.150988  9455 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:00.151022  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:00.151530  6454 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:59.969369  9444 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.107350  9478 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:00.156764  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:00.148056  9231 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:00.159574  7317 raft_consensus.cc:2376] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
I20260709 12:26:00.175475  6457 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:26:00.188894  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 206867us
I20260709 12:26:00.190970  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:00.157281  9446 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:00.204908  6451 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:00.205924  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1924) took 1150 ms. Trace:
I20260709 12:26:00.206367  6451 rpcz_store.cc:276] 0709 12:25:59.055586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.104984 (+1049398us) service_pool.cc:224] Handling call
0709 12:26:00.155388 (+ 50404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.205582 (+ 50194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.205852 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.188082  7317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 445) took 1874 ms (client timeout 1969 ms). Trace:
W20260709 12:26:00.210556  7317 rpcz_store.cc:269] 0709 12:25:58.295610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.295749 (+   139us) service_pool.cc:224] Handling call
0709 12:25:59.987720 (+1691971us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182e9c 00007f6fa0182a5b 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:26:00.168583 (+180863us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11581440}
I20260709 12:26:00.165382  9478 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.182709  9510 raft_consensus.cc:993] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
W20260709 12:26:00.220561  9508 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 12:26:00.225662  9221 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:00.235275  7332 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:26:00.236393  7335 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:00.236222  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:00.264148  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 96336.3us
I20260709 12:26:00.224004  9478 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:00.221210  9231 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:00.267993  9221 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:00.261269  9439 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:00.239585  9253 raft_consensus.cc:1081] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
W20260709 12:26:00.281982  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.280040  9486 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.544s	user 0.012s	sys 0.000s
W20260709 12:26:00.260712  9452 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.464s	user 0.000s	sys 0.007s
I20260709 12:26:00.249200  9364 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.232754  9512 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.303546  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:00.268302  9510 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:00.245702  9378 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.146s
I20260709 12:26:00.294929  9439 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.293128  9026 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.326887  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:00.327847  6378 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:00.329836  6451 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:26:00.330394  6451 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 3
I20260709 12:26:00.330690  6451 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:26:00.333424  6451 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:00.334568  9221 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.336819  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:00.337477  6300 raft_consensus.cc:606] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:00.337863  6300 raft_consensus.cc:614] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:00.338598  9485 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:26:00.339329  9351 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.339919  9351 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.339813  9512 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:00.338840  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:00.342931  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:00.342931  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49452.1us
W20260709 12:26:00.348280  6455 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.572s	user 0.011s	sys 0.000s
I20260709 12:26:00.348781  6455 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
W20260709 12:26:00.354607  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.343963  9351 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.357745  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:00.300068  9486 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.357913  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:00.358453  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 64734.9us
W20260709 12:26:00.358182  9252 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.001s
I20260709 12:26:00.354964  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1898) took 1411 ms. Trace:
W20260709 12:26:00.369026  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 216857us
I20260709 12:26:00.365231  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:00.374691  9519 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:00.375218  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:00.375495  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:00.365590  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 366) took 3773 ms. Trace:
I20260709 12:26:00.376106  6382 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.369910  9407 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:00.376113  6309 rpcz_store.cc:276] 0709 12:25:56.584708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.585879 (+  1171us) service_pool.cc:224] Handling call
0709 12:25:56.586212 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.680947 (+ 94735us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:56.687888 (+  6941us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.418637 (+1730749us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b18003285a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:25:58.669112 (+250475us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.669838 (+   726us) log.cc:844] Serialized 26 byte log entry
0709 12:25:58.669968 (+   130us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:58.669999 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.670062 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.259685 (+1589623us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:00.343860 (+ 84175us) raft_consensus.cc:1685] finished
0709 12:26:00.343882 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.358231 (+ 14349us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44287,"prepare.queue_time_us":20261,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":132525,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":26933,"spinlock_wait_cycles":18094848,"thread_start_us":339378,"threads_started":3,"wal-append.queue_time_us":2006809}
I20260709 12:26:00.381093  9407 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.388391  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:26:00.389003  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.391180  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.325317  9439 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.351706  9313 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:00.365125  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:00.397437  9313 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:00.372056  6377 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:26:00.316674  9364 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.392204  6378 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:00.370097  6303 rpcz_store.cc:276] 0709 12:25:58.943344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.337894 (+394550us) service_pool.cc:224] Handling call
0709 12:25:59.338416 (+   522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.499054 (+160638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.499101 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.512082 (+ 12981us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.718765 (+206683us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:25:59.740057 (+ 21292us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.747929 (+  7872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.764528 (+ 16599us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b54002e7a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:25:59.764593 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.258191 (+493598us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b18001d7fa0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:00.354708 (+ 96517us) raft_consensus.cc:1685] finished
0709 12:26:00.354737 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.354899 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7620,"prepare.queue_time_us":20823,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":12443,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":4570,"spinlock_wait_cycles":71114368,"thread_start_us":120040,"threads_started":1,"wal-append.queue_time_us":206692}
I20260709 12:26:00.367245  6376 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:00.398558  9313 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:00.412513  9211 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 12:26:00.413048  9211 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:00.401710  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.423346  6308 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:00.425599  9531 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:00.426211  6382 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:00.438884  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:00.440438  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:00.365793  9026 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.452154  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:00.375228  9519 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:00.456173  9455 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:00.459249  9455 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:00.456668  9224 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.680s	user 0.003s	sys 0.007s
I20260709 12:26:00.417883  9252 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.450773  9468 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:00.359997  9510 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.450582  9152 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:00.457384  9519 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:00.474191  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:00.371310  9231 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.464753  9455 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.474840  9446 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:00.486742  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:00.488279  9515 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 12:26:00.490646  9152 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.465536  9524 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:00.493144  6383 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:26:00.505182  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:00.476558  9468 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:00.513551  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:00.514554  6303 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:00.492889  9510 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:00.516991  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 208532us
I20260709 12:26:00.471266  9224 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.525966  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:00.520107  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:00.521672  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:00.520718  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:00.527665  6376 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.516953  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:00.530730  6300 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:26:00.531646  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 166681us
W20260709 12:26:00.526243  6272 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.735s (SENT) [suppressed 41 similar messages]
I20260709 12:26:00.536512  6304 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:00.538898  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.539597  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.540048  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.540426  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.540958  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.542927  7316 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:00.543010  6451 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:00.543923  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:00.518370  9486 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:00.520486  9211 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.526724  9524 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:00.549921  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:00.551162  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:00.551611  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:00.551990  6305 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:00.530210  9426 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:26:00.554359  6385 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:26:00.531700  9528 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.536108  9026 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.531400  9402 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:26:00.555059  9485 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:00.580928  9486 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:00.581791  9485 consensus_queue.cc:1243] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:00.550320  9253 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.587083  6303 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:00.547346  9407 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:00.582468  9524 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.573060  9528 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:00.588743  9444 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:00.592741  8908 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:00.587308  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:00.605736  9152 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.586983  9474 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.606230  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:00.651643  7320 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:00.651947  6385 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:26:00.652266  7320 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:00.653410  7320 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.596882  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:00.663757  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.595399  9328 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.850s	user 0.011s	sys 0.001s
I20260709 12:26:00.584659  9486 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:00.656924  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 130708us
I20260709 12:26:00.668601  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:00.668835  9407 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.669538  6376 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:00.669888  9252 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.594354  9512 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.677659  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:00.687016  6305 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:00.595949  9364 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:00.696920  6385 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:00.607586  9444 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:00.701824  9532 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:00.697925  9224 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:00.706741  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:00.709780  9528 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:00.713523  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:00.714558  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 294083us
I20260709 12:26:00.728338  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:00.729421  7319 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:00.731019  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:00.731904  7335 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:00.732146  6303 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:00.733450  6301 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 9. Ops: []
I20260709 12:26:00.736487  9407 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:00.695412  9511 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 12:26:00.743428  6381 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:00.737747  6305 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:00.744300  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:00.594637  9253 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.584796  9446 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:00.769968  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:00.663296  9468 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.671952  9474 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.701123  9328 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:00.779004  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:00.778487  6307 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:26:00.925259  6385 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:26:00.958606  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.932147  6376 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:01.036664  6457 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:00.963529  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:00.950928  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:00.929021  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.046463  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.049103  6301 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:01.049649  6381 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:00.769793  9546 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:01.052367  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:00.960335  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:00.783057  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.071069  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:01.073702  6385 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:01.080487  6459 raft_consensus.cc:2410] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:00.739596  9524 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.075922  7332 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:01.084574  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:01.088886  6452 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:00.819334  9528 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:01.095619  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.097411  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.098397  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.109233  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.110127  6457 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:00.778505  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:01.128837  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:00.761386  9511 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.066987  9224 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:26:00.774928  9455 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:01.134630  9534 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:26:01.152817  9401 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.356s	user 0.012s	sys 0.001s
I20260709 12:26:01.083928  9532 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:01.164672  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.206476  6385 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:01.217083  7318 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:01.245879  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.248113  6380 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:01.127691  9528 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:01.153276  9401 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.306314  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.248916  9512 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.310525  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:01.299906  9378 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:01.315742  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 388078us
I20260709 12:26:01.314757  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:01.316682  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.304694  9253 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.320040  6384 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:01.326961  6385 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:01.131336  9486 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.330386  7332 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:26:01.316159  9155 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:26:01.316063  9478 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.029s	user 0.008s	sys 0.005s
I20260709 12:26:01.312929  9445 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:01.237528  9444 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.401274  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.355792  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.401057  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:01.410336  6385 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:01.356496  9378 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:01.319152  9454 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.024s	sys 0.054s
I20260709 12:26:01.313405  9550 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:01.358106  9532 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.473822  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:01.358644  9224 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:01.480048  9445 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.311497  9528 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.475332  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.317034  9402 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:01.477228  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.314190  9512 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.322023  9537 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:01.484702  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.511164  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.475788  9553 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:01.488523  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.491070  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.492460  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:01.521085  7332 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:01.523802  7332 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:01.492584  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:01.493580  9439 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.100s	user 0.010s	sys 0.002s
I20260709 12:26:01.493846  9556 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.494073  9554 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:01.519325  9536 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.434s	user 0.001s	sys 0.000s
I20260709 12:26:01.498581  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.502234  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.485109  9497 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:01.511798  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.485579  9305 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:26:01.492657  9519 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.989s	user 0.006s	sys 0.004s
I20260709 12:26:01.512599  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.483894  9224 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.518149  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.480600  9561 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.528529  6304 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:01.525490  6302 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:01.529474  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.520195  9550 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.644088  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 347) took 1317 ms. Trace:
I20260709 12:26:01.537235  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.544713  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 348) took 1217 ms. Trace:
W20260709 12:26:01.477828  9454 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.025s	sys 0.055s
I20260709 12:26:01.472494  9478 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.730872  6305 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:26:01.731092  6453 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:01.730199  6384 rpcz_store.cc:276] 0709 12:26:00.326600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.338560 (+ 11960us) service_pool.cc:224] Handling call
0709 12:26:01.643856 (+1305296us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:01.644021 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2246118144}
I20260709 12:26:01.732180  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.732180  9478 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.732865  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:01.733446  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:01.733592  9478 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:01.733702  6384 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:01.548352  9378 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:01.734215  9478 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.724193  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.736290  6459 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
W20260709 12:26:01.737035  9529 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:26:01.737272  9478 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.737512  9328 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:01.725420  9401 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:01.725805  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.778574  9519 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.738157  9497 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.780465  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.781508  9497 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.567442  6307 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:01.781800  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.781903  9519 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:01.782866  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:01.537781  9556 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.739679  9550 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.742722  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:01.531286  9553 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.533631  9511 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:01.660543  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.540361  9554 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.744611  9512 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.533844  9439 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.516654  9402 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.748759  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.645277  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:01.799152  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.532025  6381 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:01.752575  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.800053  6385 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:01.800527  6305 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
W20260709 12:26:01.730165  9543 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.248s	user 0.000s	sys 0.000s
I20260709 12:26:01.730453  6379 rpcz_store.cc:276] 0709 12:26:00.326956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.343655 (+ 16699us) service_pool.cc:224] Handling call
0709 12:26:01.544476 (+1200821us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400065754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:01.544625 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355332352}
I20260709 12:26:01.758877  6384 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:01.804200  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.761632  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.643095  9445 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.668339  9563 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.758389  9404 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:01.787443  6307 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:01.787688  9478 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:01.829339  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.830675  6384 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:01.538585  9474 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.644223  9560 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:01.804522  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1808) took 1036 ms. Trace:
I20260709 12:26:01.834753  6381 rpcz_store.cc:276] 0709 12:26:00.763722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.963207 (+199485us) service_pool.cc:224] Handling call
0709 12:26:01.800146 (+836939us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.803503  9550 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.805073  6459 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:26:01.805456  6305 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:01.761799  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:01.808123  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:01.836936  6302 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
W20260709 12:26:01.809216  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.921s	user 0.027s	sys 0.031s
W20260709 12:26:01.839274  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.027s	sys 0.031s
I20260709 12:26:01.809705  6301 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:01.810776  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.784855  9305 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:01.823485  6455 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:01.823877  6424 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.068s	sys 0.135s
I20260709 12:26:01.530746  9537 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:01.854521  6424 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.068s	sys 0.136s
I20260709 12:26:01.855295  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2255) took 1280 ms. Trace:
I20260709 12:26:01.855919  6384 rpcz_store.cc:276] 0709 12:26:00.554064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.829014 (+1274950us) service_pool.cc:224] Handling call
0709 12:26:01.834426 (+  5412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.823983  9512 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:01.823032  9328 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:01.865846  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:01.866704  6459 raft_consensus.cc:2376] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7. Re-sending same reply.
I20260709 12:26:01.854832  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.868276  6457 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:26:01.870857  6379 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:01.826010  9364 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:01.845479  6385 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
W20260709 12:26:01.854887  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 629950us
W20260709 12:26:01.874469  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.875138  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.875475  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.847973  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.020s	sys 0.038s
W20260709 12:26:01.877648  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.020s	sys 0.038s
I20260709 12:26:01.879405  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2261) took 1175 ms. Trace:
I20260709 12:26:01.879580  6384 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:01.879961  6379 rpcz_store.cc:276] 0709 12:26:00.695871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.833071 (+1137200us) service_pool.cc:224] Handling call
0709 12:26:01.870679 (+ 37608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.871230 (+   551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.871386 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:01.887090  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 144471us
I20260709 12:26:01.888083  6385 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:01.832873  9478 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.891527  9478 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.865278  9563 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.894938  9563 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:01.895162  9402 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:01.836239  9404 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.896665  9224 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.897864  9404 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.860630  9512 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:01.898774  9512 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:01.899436  9439 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.901387  9556 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.902706  6381 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:01.905706  9155 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:26:01.793025  9561 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.888181  9328 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.908514  9561 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.864560  9560 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.909543  9328 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.891645  9554 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:01.910516  6310 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:26:01.911846  9560 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:01.864938  9550 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:01.915261  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 320) took 2254 ms. Trace:
I20260709 12:26:01.915769  6454 rpcz_store.cc:276] 0709 12:25:59.665163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.160545 (+495382us) service_pool.cc:224] Handling call
0709 12:26:00.160900 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.229629 (+ 68729us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:00.441312 (+211683us) log.cc:844] Serialized 14 byte log entry
0709 12:26:00.534862 (+ 93550us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.551156 (+ 16294us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.568575 (+ 17419us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:00.568652 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.568765 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.914963 (+1346198us) raft_consensus.cc:1685] finished
0709 12:26:01.914992 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.915177 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17755,"prepare.queue_time_us":276267,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":10390,"raft.queue_time_us":30669,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":11327,"thread_start_us":284454,"threads_started":2,"wal-append.queue_time_us":874145}
I20260709 12:26:01.876098  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:01.923938  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47949.8us
W20260709 12:26:01.929780  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.890270  9553 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.927317  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:01.933429  6379 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:26:01.927052  9328 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:01.940312  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1891) took 7330 ms. Trace:
I20260709 12:26:01.882916  9364 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:01.949896  6458 rpcz_store.cc:276] 0709 12:25:54.609543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.661359 (+ 51816us) service_pool.cc:224] Handling call
0709 12:25:54.661643 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.641123 (+4979480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.939820 (+2298697us) spinlock_profiling.cc:243] Waited 4.93 s on lock 0x7b54003c78d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:01.940197 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11345616768}
W20260709 12:26:01.945362  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 196818us
W20260709 12:26:01.951864  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.957922  6385 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:26:01.946657  6425 connection.cc:256] RPC call timeout handler was delayed by -4.3502371990002757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1757s for a response
W20260709 12:26:01.959712  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.960330  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:01.961169  6457 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:01.963429  6379 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:26:01.946179  9439 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:01.971508  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:01.972285  6458 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:01.973439  6458 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:01.952144  9311 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.964156  9155 tablet_bootstrap.cc:492] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:26:01.949740  9328 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:01.981927  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54755.2us
I20260709 12:26:01.986317  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:01.987052  6458 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:26:01.968693  9550 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:01.989012  6385 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:01.975731  9519 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:01.986991  9328 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:01.946225  9305 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.004189  6457 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:26:02.005415  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.006140  6457 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:01.978075  9439 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.010634  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.011029  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:02.011967  6457 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:26:02.012199  9439 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.012892  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.013540  6457 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
W20260709 12:26:02.024284  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1805) took 1696 ms (client timeout 1561 ms). Trace:
I20260709 12:26:02.025010  9402 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:02.026604  9445 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:02.027024  9445 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:02.027926  9445 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.029158  9445 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:02.024752  6383 rpcz_store.cc:269] 0709 12:26:00.328309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.364767 (+ 36458us) service_pool.cc:224] Handling call
0709 12:26:02.024110 (+1659343us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:02.024211 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275743616}
W20260709 12:26:02.030963  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56627.5us
I20260709 12:26:02.032481  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:02.033421  9439 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:02.033993  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:02.039317  9364 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.003590  9155 ts_tablet_manager.cc:1403] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 2.306s	user 0.030s	sys 0.032s
I20260709 12:26:02.043867  9305 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.063184  9519 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:02.063694  9399 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:02.064226  9537 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.045711  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2386) took 3601 ms. Trace:
W20260709 12:26:02.070880  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75155.9us
I20260709 12:26:01.993376  9311 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.077854  6458 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:02.079169  9439 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:02.079871  6380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 345) took 1754 ms (client timeout 2105 ms). Trace:
I20260709 12:26:02.080505  6456 rpcz_store.cc:276] 0709 12:25:58.444377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.981043 (+3536666us) service_pool.cc:224] Handling call
0709 12:26:02.000056 (+ 19013us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.009606 (+  9550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.009632 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.045377 (+ 35745us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:02.045433 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.045506 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.045647 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:02.092314  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 81457.4us
I20260709 12:26:02.082023  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.093657  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2411) took 1729 ms. Trace:
I20260709 12:26:02.084388  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.080139  9483 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:26:02.070997  9402 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:26:02.086951  7320 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:02.100476  9544 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.523s	user 0.007s	sys 0.014s
I20260709 12:26:02.101577  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.089632  9455 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.090615  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:02.091152  6380 rpcz_store.cc:269] 0709 12:26:00.325829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.325966 (+   137us) service_pool.cc:224] Handling call
0709 12:26:02.079646 (+1753680us) spinlock_profiling.cc:243] Waited 894 ms on lock 0x7b540023dcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:02.079802 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057057664}
I20260709 12:26:02.093772  6457 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:26:02.094478  6382 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:26:02.094122  6455 rpcz_store.cc:276] 0709 12:26:00.362119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.369659 (+  7540us) service_pool.cc:224] Handling call
0709 12:26:02.090774 (+1721115us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54001c1454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:02.090934 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3328553344}
I20260709 12:26:02.096711  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.099684  9311 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.102428  6377 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:02.082780  9364 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:26:02.107328  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.107957  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.107353  9512 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.111630  6457 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:26:02.112665  9512 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:02.115556  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 81696us
I20260709 12:26:02.117676  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.118695  6458 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:02.120625  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.122085  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:02.123996  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50379.1us
I20260709 12:26:02.126451  9444 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:02.131585  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.132875  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.116983  9537 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:02.132875  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:02.132915  6450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2399) took 2511 ms (client timeout 1795 ms). Trace:
W20260709 12:26:02.136713  6450 rpcz_store.cc:269] 0709 12:25:59.621337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.676887 (+ 55550us) service_pool.cc:224] Handling call
0709 12:26:02.132664 (+2455777us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:02.132809 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279351680}
I20260709 12:26:02.134419  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.133224  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.145193  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:02.146265  9364 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:02.146872  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:02.161625  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.172641  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:02.171897  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.186419  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:02.193805  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.200928  6450 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:02.187500  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.206827  6380 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.169421  9483 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:02.209188  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.209977  6377 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:02.212900  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:02.213075  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.213630  6301 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:02.214121  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.214691  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:02.214643  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:02.215636  6301 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.215975  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.217075  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.217576  6305 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:02.221143  9428 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.221690  9428 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.222640  9561 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:02.193532  9577 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:02.218268  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:02.224692  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.225206  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.235277  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.214744  6300 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:02.236106  6450 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:02.404927  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.405813  6309 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:02.405875  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.255600  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.255178  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.409169  6380 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:02.377506  6274 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.572s (SENT)
I20260709 12:26:02.409078  6377 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 3.
I20260709 12:26:02.252880  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.410437  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.410936  6385 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:02.233979  6301 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:02.256320  6379 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
W20260709 12:26:02.391490  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60545.1us
I20260709 12:26:02.252318  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:02.254667  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 85268.1us
I20260709 12:26:02.411145  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.318944  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.252746  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.414711  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.267675  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.416699  6457 raft_consensus.cc:2368] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:02.211898  9483 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.416705  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.415030  6383 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
W20260709 12:26:02.412149  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.419196  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.415344  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:02.422008  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.422772  6450 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:02.424424  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.425536  6348 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 2.164s (SENT)
I20260709 12:26:02.426024  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:02.198616  9537 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:02.426733  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:02.263361  9582 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:02.428181  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.428468  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.427450  6377 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:02.421142  9483 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:02.436039  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58585.3us
I20260709 12:26:02.436167  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.437557  6458 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:02.437476  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.433091  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.432931  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.229887  9364 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:02.435070  6380 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.392225  9328 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:02.248627  9561 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.443051  9567 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.443487  9567 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.443862  9567 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.444716  9561 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.435675  9519 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:26:02.446859  9519 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:02.447260  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 649) took 1918 ms. Trace:
I20260709 12:26:02.447751  7318 rpcz_store.cc:276] 0709 12:26:00.528931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.535808 (+  6877us) service_pool.cc:224] Handling call
0709 12:26:00.589625 (+ 53817us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.217590 (+627965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.446988 (+1229398us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b540043a3d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:02.447192 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1282052352}
I20260709 12:26:02.448370  9328 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.449846  6456 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:02.455503  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.457235  6380 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:02.323064  9512 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.324149  9428 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.438357  6300 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:02.412132  9550 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.439397  6455 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:26:02.471129  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.470757  9550 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.472455  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.473094  6455 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:02.440435  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.458285  6379 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:02.458745  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:02.421216  9446 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.464432  7320 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:02.422133  9528 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.323279  9311 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.115s
I20260709 12:26:02.474913  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.474027  6457 raft_consensus.cc:2376] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6. Re-sending same reply.
W20260709 12:26:02.503338  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 77804.6us
I20260709 12:26:02.503840  6455 raft_consensus.cc:2376] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12. Re-sending same reply.
I20260709 12:26:02.508584  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 648) took 1983 ms. Trace:
W20260709 12:26:02.508865  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70416.5us
I20260709 12:26:02.509974  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.473924  9550 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.513176  9311 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:26:02.513577  9481 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:26:02.509107  9446 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.514297  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:02.509824  7320 rpcz_store.cc:276] 0709 12:26:00.525710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.533738 (+  8028us) service_pool.cc:224] Handling call
0709 12:26:00.587275 (+ 53537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.086362 (+1499087us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b58004a57d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa329dfe7 00007f6fa329f7de 00007f6fa32a9b49 00007f6fa32aa416 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4
0709 12:26:02.506311 (+419949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.508484 (+  2173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105252608}
I20260709 12:26:02.515003  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.531339  6455 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:26:02.515429  6457 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:02.509231  9528 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.516100  9328 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:02.517372  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2267) took 1212 ms. Trace:
W20260709 12:26:02.523303  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.524554  9497 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:02.526871  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.527796  9474 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:02.481714  9439 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:02.532406  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.535848  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2410) took 2195 ms. Trace:
I20260709 12:26:02.537237  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:02.534605  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79306.9us
I20260709 12:26:02.541355  6451 rpcz_store.cc:276] 0709 12:26:00.336877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.356707 (+ 19830us) service_pool.cc:224] Handling call
0709 12:26:00.365952 (+  9245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.543523 (+177571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.531978 (+1988455us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540028cdd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:02.532268 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406749440}
I20260709 12:26:02.541839  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.536391  6384 rpcz_store.cc:276] 0709 12:26:01.304515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.902577 (+598062us) service_pool.cc:224] Handling call
0709 12:26:01.902962 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.903197 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:01.967854 (+ 64657us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.516116 (+548262us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b180034f000. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa6c9f3c7 00007f6fa32b19a7 00007f6fa32c10bf 00007f6fa32c1038
0709 12:26:02.516463 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.516742 (+   279us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.516826 (+    84us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:02.516940 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.517058 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.517254 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":507606,"prepare.queue_time_us":622946,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":85,"raft.queue_time_us":11194,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3199,"spinlock_wait_cycles":13380736,"thread_start_us":89094,"threads_started":2,"wal-append.queue_time_us":116682}
W20260709 12:26:02.536633  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 90842.3us
I20260709 12:26:02.539832  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:02.528002  9311 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:02.546952  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:02.535924  9573 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:02.552352  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.539605  9550 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.536104  9328 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:02.456549  9364 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:02.549723  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:02.553115  6451 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:02.540684  9511 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:26:02.542004  9497 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:02.556525  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.545296  9512 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:02.623147  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81393.5us
I20260709 12:26:02.626792  9556 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.557322  9550 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.631057  9556 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:02.609469  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49036.9us
I20260709 12:26:02.632234  9364 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:02.632404  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.610555  9558 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.608139  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.631733  9556 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.548228  9468 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.587433  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73202.5us
W20260709 12:26:02.601760  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62018.9us
I20260709 12:26:02.557057  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:02.609253  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62373.8us
I20260709 12:26:02.631247  9550 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.545902  9305 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.640193  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.648931  6301 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:02.649235  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:02.644989  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 72685.6us
I20260709 12:26:02.645175  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.551920  9311 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:02.650089  6455 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:02.627612  9589 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.632019  9512 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:02.658674  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:02.559114  9439 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:02.678260  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.663544  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.654973  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.668674  7317 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:02.655304  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:02.544449  9474 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.662953  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.682513  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.683389  6455 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:02.677959  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.638324  9573 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:02.619377  9566 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:02.689146  9328 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.653431  9558 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.562800  9444 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.122s
W20260709 12:26:02.701799  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46604.5us
I20260709 12:26:02.692284  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.552812  9404 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 12:26:02.601760  9580 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
W20260709 12:26:02.707989  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57005.7us
I20260709 12:26:02.709324  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.702930  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.706632  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.688476  9573 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.620492  9537 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.653556  9468 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.664700  9305 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.698863  9303 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:26:02.748569  6382 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:02.715922  7319 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:02.713397  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2397) took 3254 ms. Trace:
I20260709 12:26:02.749079  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:02.801761  6383 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:02.753005  6380 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:02.752560  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:02.749604  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.803253  6453 rpcz_store.cc:276] 0709 12:25:59.459016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.134803 (+675787us) service_pool.cc:224] Handling call
0709 12:26:00.156774 (+ 21971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.731559 (+1574785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.713067 (+981508us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:02.713317 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3620144384}
I20260709 12:26:02.787081  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:02.804936  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52463.6us
I20260709 12:26:02.751685  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:02.806039  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:02.751276  6384 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:02.806198  6381 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:02.803948  6383 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:02.812372  9565 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:02.812564  6383 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.813251  6451 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:26:02.813898  6383 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.815785  9155 raft_consensus.cc:359] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:02.813972  6451 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:02.816847  9155 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:26:02.817094  6451 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:02.817653  9155 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:02.819695  9311 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.821003  9497 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:02.822592  9474 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.824106  9558 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:02.799628  9588 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
W20260709 12:26:02.827805  6310 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1702) took 1339 ms (client timeout 1681 ms). Trace:
I20260709 12:26:02.827836  9455 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:02.737872  9593 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:02.830499  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 651) took 2254 ms. Trace:
I20260709 12:26:02.834888  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.704237  9512 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.836268  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.739112  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.713894  9483 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.837914  6451 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:02.885367  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50582.3us
I20260709 12:26:02.886132  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.887018  6451 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.887290  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:02.712289  9566 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:02.888614  9566 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.753173  9573 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:02.891059  7317 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:02.892040  9512 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.681622  9589 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.841593  6383 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:26:02.894742  6383 pending_rounds.cc:85] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 8
I20260709 12:26:02.895095  6383 pending_rounds.cc:107] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:26:02.895360  6383 raft_consensus.cc:2889] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.778666  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:02.896303  9589 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.740849  9528 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.834098  9364 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:02.896952  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 118383us
I20260709 12:26:02.897709  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:02.898533  9364 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:02.855753  6453 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:02.899220  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:02.788681  9328 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:02.889859  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:02.857507  6310 rpcz_store.cc:269] 0709 12:26:01.488335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.497677 (+  9342us) service_pool.cc:224] Handling call
0709 12:26:02.827606 (+1329929us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:02.827734 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351207296}
I20260709 12:26:02.807832  9439 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:02.859570  9155 ts_tablet_manager.cc:1434] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.790s	user 0.005s	sys 0.005s
W20260709 12:26:02.865482  6275 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:26:02.866528  9351 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:26:02.868999  9574 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.762s	user 0.002s	sys 0.000s
I20260709 12:26:02.870273  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.914525  7319 raft_consensus.cc:2410] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:02.847260  9565 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:02.874914  7318 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:02.878733  9531 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.920792  9483 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:02.743122  9567 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.924223  9567 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.871848  9455 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.924937  9455 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:02.695098  9454 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
W20260709 12:26:02.880364  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2260) took 2257 ms (client timeout 2154 ms). Trace:
I20260709 12:26:02.838568  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:02.881021  7331 rpcz_store.cc:276] 0709 12:26:00.575560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.575763 (+   203us) service_pool.cc:224] Handling call
0709 12:26:02.026507 (+1450744us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:26:02.026728 (+   221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.531399 (+504671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.531426 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.535804 (+  4378us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.694319 (+158515us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:02.694361 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.694422 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.830016 (+135594us) raft_consensus.cc:1685] finished
0709 12:26:02.830034 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.830262 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":543380,"raft.queue_time_us":504934,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":12091,"spinlock_wait_cycles":16880384,"thread_start_us":92745,"threads_started":1,"wal-append.queue_time_us":276358}
I20260709 12:26:02.927837  7335 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:02.838052  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.928429  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.929687  6457 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:02.929916  9446 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.930809  9305 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:02.712620  9595 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:02.914222  9404 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:02.753659  9590 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:26:02.907184  9481 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:02.931757  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.932035  9481 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:02.932523  9481 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:02.916021  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:02.911803  9426 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:02.934772  9426 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:02.935202  9426 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:02.914079  9155 ts_tablet_manager.cc:1467] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Tablet startup took 838ms
I20260709 12:26:02.919145  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.901372  9485 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:02.919644  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:02.938889  9485 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.939380  9485 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.911607  9444 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.942986  9444 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.943066  9483 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.943524  9444 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.943658  9483 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:02.944242  9483 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:02.945191  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.921097  9531 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.921363  9311 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:02.927457  6376 rpcz_store.cc:269] 0709 12:26:00.622689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.694411 (+ 71722us) service_pool.cc:224] Handling call
0709 12:26:02.880066 (+2185655us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:02.880248 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487865216}
I20260709 12:26:02.917297  9351 consensus_queue.cc:1243] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:02.917927  9565 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.946437  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:02.942238  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 385258us
I20260709 12:26:02.932471  6457 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 4.
I20260709 12:26:02.955127  6458 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:26:02.943907  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 387) took 1298 ms. Trace:
I20260709 12:26:02.956385  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:02.953666  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:02.959012  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:02.960433  9512 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:02.872677  9556 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.961133  9481 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.962700  9580 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:02.955847  6307 rpcz_store.cc:276] 0709 12:26:01.645507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.828901 (+183394us) service_pool.cc:224] Handling call
0709 12:26:01.843569 (+ 14668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.967276 (+123707us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:01.971313 (+  4037us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.991627 (+ 20314us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.991799 (+   172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.991848 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:01.991908 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.493725 (+501817us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b180041a8e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:02.519079 (+ 25354us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b180041a8e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f
0709 12:26:02.520185 (+  1106us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b4800002958. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:26:02.943667 (+423482us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54002dfad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:02.943721 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.943853 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":805,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":4650,"raft.queue_time_us":634,"raft.run_cpu_time_us":4017,"raft.run_wall_time_us":531458,"spinlock_wait_cycles":298994048,"thread_start_us":19657,"threads_started":1,"wal-append.queue_time_us":101652}
I20260709 12:26:02.963621  9439 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.964524  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:02.964747  9580 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:02.965554  6457 raft_consensus.cc:2368] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:02.931779  9404 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:02.964174  9531 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.965273  9595 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:02.966848  9404 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:02.910820  9558 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.911250  9303 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.968618  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:02.953804  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 246438us
I20260709 12:26:02.977022  7319 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:02.975697  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:02.985925  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.982584  9351 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:02.987550  9153 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:02.987845  9351 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.988265  9468 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.984308  9311 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:26:02.991416  9311 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:02.991740  9311 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:02.992492  9426 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.992681  9532 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:02.993731  9455 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:02.994643  7320 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:02.995846  9455 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:02.996047  7314 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:02.998049  9481 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:02.999575  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.000815  9402 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:03.001377  9402 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.001930  9560 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:02.979084  9305 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:03.003000  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.003453  9305 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:03.001189  9364 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.003917  9305 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.004309  9581 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:03.003966  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.004781  9581 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:02.988339  9351 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.002421  9402 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.011359  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:03.026618  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:03.013701  6459 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.019972  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:03.014564  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.038376  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:02.983446  9303 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:03.042455  6307 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:03.047960  6307 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.042996  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:03.045591  6452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 321) took 3263 ms (client timeout 2105 ms). Trace:
W20260709 12:26:03.049149  6452 rpcz_store.cc:269] 0709 12:25:59.782595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.792352 (+  9757us) service_pool.cc:224] Handling call
0709 12:26:03.045377 (+3253025us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400343854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:03.045510 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2978765568}
I20260709 12:26:02.982908  9556 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.051030  6303 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:26:03.052515  6303 pending_rounds.cc:85] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:26:03.052783  6303 pending_rounds.cc:107] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:26:03.053124  6303 raft_consensus.cc:2889] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.009395  9153 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:03.051673  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.053815  9303 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.054276  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.008127  9581 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.054250  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.054371  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.013242  9454 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:03.026472  9567 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:02.983045  9558 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:03.063720  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51359us
I20260709 12:26:03.054456  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:02.983266  9550 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.060099  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:03.062376  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52639us
I20260709 12:26:03.031797  9426 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:03.058544  6308 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:03.046187  9553 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.070192  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.070860  6452 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:03.065079  6300 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:03.028232  9531 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:26:03.069401  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.061970  9567 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.028975  9528 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:26:03.085294  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.085897  6456 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:03.076217  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.086861  6457 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:02.992542  9311 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.038712  9560 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:26:03.057276  9590 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:03.088878  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.061741  9454 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.085448  9567 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.115175  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:02.936637  9155 ts_tablet_manager.cc:1469] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Trace:
0709 12:25:50.163889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.313844 (+149955us) service_pool.cc:224] Handling call
0709 12:25:50.844909 (+531065us) ts_tablet_manager.cc:934] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
0709 12:25:53.865533 (+3020624us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:25:54.507799 (+642266us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:55.026378 (+518579us) tablet_metadata.cc:665] Metadata flushed
0709 12:25:55.138067 (+111689us) tablet_copy_client.cc:463] Replaced tombstoned tablet metadata.
0709 12:25:56.193090 (+1055023us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:25:56.193538 (+   448us) tablet_replica.cc:206] Creating consensus instance
0709 12:25:56.419919 (+226381us) spinlock_profiling.cc:243] Waited 30.8 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa01831a2 00007f6fa90c4889 00007f6fa81f5d88 00007f6fa82bc973 00007f6fa82b870f 00007f6fa82ba2d3 00007f6fa82ebf4e 00007f6fa82d0fdd 00007f6fa82d0f89
0709 12:25:58.277788 (+1857869us) ts_tablet_manager.cc:989] Replica registered.
0709 12:25:58.287454 (+  9666us) inbound_call.cc:177] Queueing success response
0709 12:25:59.697750 (+1410296us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:26:02.069745 (+2371995us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:26:02.079001 (+  9256us) tablet_replica.cc:261] Starting instrumentation
0709 12:26:02.079623 (+   622us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":51629,"mutex_wait_us":447927,"raft.queue_time_us":16248,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":70827904,"tablet-copy.queue_time_us":366289,"tablet-download-pool.queue_time_us":215,"tablet-download-pool.run_cpu_time_us":2351,"tablet-download-pool.run_wall_time_us":765656,"thread_start_us":303995,"threads_started":2}
I20260709 12:26:03.084014  9485 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.119285  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 4.
I20260709 12:26:03.124145  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:03.125417  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:03.126012  6457 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:03.127300  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62845.8us
I20260709 12:26:03.129235  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.131407  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.132207  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.133044  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.136176  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.137128  6450 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:03.130918  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:03.113276  9595 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:03.128021  9311 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.102192  9399 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
W20260709 12:26:03.149183  9605 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:03.143519  9566 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:03.150931  9566 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:03.151361  9566 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.152103  9528 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:03.152541  9528 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:03.152877  9528 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.131641  9573 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.157188  9573 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.159017  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.159667  6450 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.159763  9485 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.161512  9399 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.161990  9399 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.164218  9399 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:03.164871  9399 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:03.165236  9399 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.166138  6455 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:03.146059  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:03.169373  9590 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.169421  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:03.170332  9590 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.140861  9531 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:03.171856  9531 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.172187  9531 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.177076  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.178133  6455 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:03.179993  6307 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.181072  6307 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.183143  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:03.126740  9517 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:03.206640  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.141791  9537 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:03.217247  9351 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:03.217866  9606 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:03.218896  9512 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:26:03.223212  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 131338us
W20260709 12:26:03.228741  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45703.1us
I20260709 12:26:03.232852  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:03.236912  7329 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEARNER]: Advancing to term 8
I20260709 12:26:03.237807  6307 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:26:03.238281  6307 pending_rounds.cc:85] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:26:03.238623  6307 pending_rounds.cc:107] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:26:03.238787  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.226050  6300 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:03.240172  6457 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:03.239264  6307 raft_consensus.cc:2889] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.245993  7316 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:26:03.246670  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.247479  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.249421  9351 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.122119  9153 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.251665  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.228240  9537 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:03.246820  9399 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.218976  9550 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.248113  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.225016  6452 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:03.260110  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.276548  6304 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:03.265478  7316 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.266788  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.278118  6309 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:03.270090  6307 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.282791  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:03.283841  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:03.284473  7316 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:03.285499  7316 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:03.286945  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:03.287551  6454 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:26:03.287137  9444 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.288475  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.247349  9512 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:03.289038  6454 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:03.290076  9512 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.291321  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.241802  9590 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:03.293083  9590 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:03.294021  9590 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.272737  9560 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:03.274887  7331 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:03.254853  9537 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.258201  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.277140  7335 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:03.267263  9454 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.262036  9305 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:03.303920  9589 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
W20260709 12:26:03.272606  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136523us
I20260709 12:26:03.304865  9305 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.263445  9558 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.290745  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.306010  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:03.282101  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:03.323289  6310 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:03.323604  6452 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 9 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:26:03.272094  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.306841  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:03.324683  6300 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 4: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:03.310631  6456 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:03.310458  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:03.327310  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.314390  6455 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:03.319614  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:03.324716  6457 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:03.326189  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:03.309129  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:03.329103  6456 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:03.313374  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:03.327400  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.331956  6457 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:03.332532  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1952) took 1120 ms. Trace:
I20260709 12:26:03.332892  6457 rpcz_store.cc:276] 0709 12:26:02.211748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.328606 (+1116858us) service_pool.cc:224] Handling call
0709 12:26:03.328893 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.332305 (+  3412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.332482 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.258425  9399 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:03.332899  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.332499  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:03.336799  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.354173  6303 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:03.354358  6308 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.254467  9528 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.379417  9573 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.334436  9364 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:26:03.383495  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.377640  9537 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.384162  6305 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:03.384790  9613 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:03.384196  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73881.3us
I20260709 12:26:03.387490  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:03.383203  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.388305  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.361334  6454 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:03.389217  6452 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.390291  6454 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:03.357690  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2434) took 1148 ms. Trace:
W20260709 12:26:03.379552  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47128.8us
I20260709 12:26:03.391541  6459 rpcz_store.cc:276] 0709 12:26:02.209212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.830155 (+620943us) service_pool.cc:224] Handling call
0709 12:26:03.010256 (+180101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.031144 (+ 20888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.357470 (+326326us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54001c1e50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:03.357610 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":401693952}
I20260709 12:26:03.392045  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.396718  6454 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:03.358074  9600 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:03.408113  6459 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:03.369782  9483 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.376061  9444 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.400048  9362 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:03.434059  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2442) took 1027 ms. Trace:
I20260709 12:26:03.395479  9598 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:26:03.435057  6459 rpcz_store.cc:276] 0709 12:26:02.406914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.392988 (+986074us) service_pool.cc:224] Handling call
0709 12:26:03.407894 (+ 14906us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.433736 (+ 25842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.433991 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.436587  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1959) took 1023 ms. Trace:
I20260709 12:26:03.411670  9595 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.437037  6454 rpcz_store.cc:276] 0709 12:26:02.412974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.396276 (+983302us) service_pool.cc:224] Handling call
0709 12:26:03.396546 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.420016 (+ 23470us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.436523 (+ 16507us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:03.412413  9537 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.414052  9581 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.444120  9581 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:03.358870  9599 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:03.395921  9364 consensus_queue.cc:1243] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:03.417768  6458 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:03.398413  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.335996  9439 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:03.446002  9439 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:03.446019  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47698.3us
I20260709 12:26:03.446828  9439 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:03.447440  9439 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.448020  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.431815  6450 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:03.449885  6457 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:26:03.449981  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:03.450757  9311 raft_consensus.cc:2764] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:03.451772  9537 raft_consensus.cc:2764] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:26:03.411540  6307 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:03.453691  6307 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:03.457578  9537 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.446188  6308 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:26:03.458370  9537 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:03.464794  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 491656us
I20260709 12:26:03.466365  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:03.475716  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:03.477432  6452 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:26:03.478466  6459 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:03.479040  6452 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:03.418886  9553 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:03.554556  9558 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.555348  9558 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.556326  9483 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:03.557883  9357 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:03.558413  9357 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.558626  9517 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:03.559237  9517 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:03.559762  9517 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.559369  9357 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.560591  9357 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.563153  6302 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.420512  9614 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.612299  6348 leader_election.cc:400] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:03.422102  9600 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:03.660300  6348 leader_election.cc:403] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:03.666215  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 150670us
I20260709 12:26:03.668507  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:03.421588  9351 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.236s
I20260709 12:26:03.419059  9531 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.486105  9595 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:03.664804  9611 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 12:26:03.701840  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.420871  9303 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.706650  9303 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.707695  7332 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.473068  9404 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.202s
I20260709 12:26:03.659011  9614 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.656440  9558 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.470351  9560 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.202s
I20260709 12:26:03.669556  9600 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:03.715911  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 314910us
W20260709 12:26:03.716815  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 333662us
I20260709 12:26:03.726787  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.696871  9602 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.733843  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.566673  9599 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.144s
I20260709 12:26:03.701071  9311 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:03.713706  9595 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:03.726459  9593 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.841s	user 0.006s	sys 0.004s
W20260709 12:26:03.726718  9567 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.595s	user 0.008s	sys 0.000s
W20260709 12:26:03.751909  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 302010us
I20260709 12:26:03.721746  9598 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.758030  6308 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:03.722473  9614 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.760303  9615 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:03.766614  9615 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.767446  9615 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.779209  8735 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:03.779706  8735 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.780592  8735 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.784143  9531 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.756402  9311 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.786871  9311 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:03.787473  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.788151  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.788954  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62936.3us
I20260709 12:26:03.789507  9528 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.789077  9362 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:03.790715  9351 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:03.791595  9483 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:03.793612  9483 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:03.793958  9483 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:03.794545  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.750772  9602 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.795979  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.796630  6453 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.797557  9602 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.753501  9595 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:03.765549  9567 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.759915  9593 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:03.810946  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.811458  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-2483d7801887403e873adc62429fe046 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:26:03.811992  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-2483d7801887403e873adc62429fe046 on tablet 2483d7801887403e873adc62429fe046 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.813879  9620 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:03.814752  9620 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:26:03.815356  9620 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:03.816347  9620 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.819258  6456 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:26:03.794926  9483 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:03.820423  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:03.827466  6307 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:03.810995  6451 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:03.822371  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:03.855819  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:03.856712  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:03.858580  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:03.858973  6453 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:03.861697  9589 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.862432  9589 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.864822  9589 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:03.867020  9613 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:03.867286  9351 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:03.867991  9613 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.868245  9351 consensus_queue.cc:1243] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:03.868609  9444 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:03.867477  9362 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:26:03.869009  9613 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.870822  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:03.872215  9362 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:03.872730  9362 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.873765  9362 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.877786  9469 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:03.878419  9469 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.882643  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:03.806318  9567 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:03.894160  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:03.896339  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:03.900568  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.902727  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:03.909559  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:03.866971  9446 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:03.914769  6304 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:03.915391  6307 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:03.915721  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.886014  9399 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:03.940062  7314 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:03.940587  7331 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:03.916179  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:03.942205  6308 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:03.868893  9351 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:03.911888  9481 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:03.860769  9404 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:03.944926  9581 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:26:03.956890  7332 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:03.957520  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:03.943073  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 274622us
I20260709 12:26:03.912192  9305 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:03.971324  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.915478  9483 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:03.914377  9558 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:03.819865  9598 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:03.988780  6303 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:03.909040  9589 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:03.975297  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.998545  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.999914  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.991848  6348 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:03.976130  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.992883  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:03.938671  9446 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:04.010006  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.003010  6453 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:03.976383  9517 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.027316  9626 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.027582  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:03.995056  9558 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.049168  9558 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:04.049887  9566 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.898s	user 0.010s	sys 0.002s
I20260709 12:26:04.050380  9566 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.050717  9303 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:04.050875  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59096.8us
W20260709 12:26:04.051177  9618 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:26:04.051340  9303 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:03.987455  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:03.972841  9399 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.973367  9599 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:03.915655  9439 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.033466  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:04.033913  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 305353us
I20260709 12:26:03.954955  9351 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:03.879544  9469 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.028839  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:04.064734  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:03.915809  9454 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:03.975634  9481 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:04.003731  9589 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.070765  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:04.071806  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:04.074235  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:04.074998  6307 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:04.050163  9630 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:26:04.075918  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:04.076520  6453 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:04.079082  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.101140  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.104220  6384 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:04.104398  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.063656  9469 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:04.080121  7317 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:04.102532  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.102957  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.103081  6303 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:04.113859  6303 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:04.107177  6307 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:26:04.108414  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.108647  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.114684  6307 pending_rounds.cc:85] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 2
I20260709 12:26:04.114977  6378 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:04.115005  6307 pending_rounds.cc:107] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:26:04.115082  6380 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:04.115715  6307 raft_consensus.cc:2889] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.116037  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:04.134368  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.136502  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.137162  6384 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:04.074077  9481 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:04.138862  6377 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:04.074594  9589 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:04.141352  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.142632  6382 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:04.143862  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 234390us
I20260709 12:26:04.144600  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:04.144734  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:04.147109  6379 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:04.148823  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.149571  6378 raft_consensus.cc:2376] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6. Re-sending same reply.
I20260709 12:26:04.151108  7332 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:04.153366  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:04.154383  7331 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:04.058612  9626 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:04.158581  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 333825us
W20260709 12:26:04.159871  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4466693019999184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
W20260709 12:26:04.160283  6425 connection.cc:256] RPC call timeout handler was delayed by -1.2936693020001258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21000000000000002s for a response
W20260709 12:26:04.160964  6425 connection.cc:256] RPC call timeout handler was delayed by -1.2603693020000719s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2137s for a response
W20260709 12:26:04.161384  6425 connection.cc:256] RPC call timeout handler was delayed by -1.2108693019999919s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260709 12:26:04.161767  6425 connection.cc:256] RPC call timeout handler was delayed by -1.1154693019998376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2298s for a response
I20260709 12:26:04.162789  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.163638  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.138875  9580 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.165130  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:04.172871  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 222211us
W20260709 12:26:04.174162  6274 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.527s (SENT)
I20260709 12:26:04.174567  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:04.178206  9627 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
W20260709 12:26:04.178583  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.179515  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 333) took 1274 ms. Trace:
I20260709 12:26:04.179971  6454 rpcz_store.cc:276] 0709 12:26:02.904616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.440867 (+536251us) service_pool.cc:224] Handling call
0709 12:26:03.441147 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.450463 (+  9316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.450513 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.451155 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.736553 (+285398us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:04.167818 (+431265us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.168176 (+   358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.168258 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.179275 (+ 11017us) raft_consensus.cc:1685] finished
0709 12:26:04.179298 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.179441 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21744,"raft.queue_time_us":1128,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":120,"thread_start_us":686439,"threads_started":2,"wal-append.queue_time_us":603459}
I20260709 12:26:04.184710  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2440) took 1892 ms. Trace:
I20260709 12:26:04.185271  6457 rpcz_store.cc:276] 0709 12:26:02.292450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.333958 (+1041508us) service_pool.cc:224] Handling call
0709 12:26:03.449635 (+115677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.450321 (+   686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.184427 (+734106us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54000c17d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:04.184628 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265043200}
I20260709 12:26:04.189404  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 397) took 1155 ms. Trace:
I20260709 12:26:04.189914  6302 rpcz_store.cc:276] 0709 12:26:03.033534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.033688 (+   154us) service_pool.cc:224] Handling call
0709 12:26:03.034038 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.563652 (+529614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.189106 (+625454us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:04.189323 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1037383936}
W20260709 12:26:04.190553  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45902.4us
I20260709 12:26:04.192015  6457 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:04.195259  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:04.079594  9600 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.223976  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1972) took 1222 ms. Trace:
I20260709 12:26:04.224373  6457 rpcz_store.cc:276] 0709 12:26:02.974725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.186415 (+1211690us) service_pool.cc:224] Handling call
0709 12:26:04.191781 (+  5366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.196544 (+  4763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.196722 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:04.225685  6457 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:04.206884  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.170003  9351 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:26:04.213882  9628 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 12:26:04.229004  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66317.7us
I20260709 12:26:04.215427  9632 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
W20260709 12:26:04.216562  9623 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:26:04.217332  9619 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:26:04.226447  6380 raft_consensus.cc:2376] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7. Re-sending same reply.
I20260709 12:26:04.230180  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.231351  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1973) took 1256 ms. Trace:
I20260709 12:26:04.143551  6201 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 3 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:04.175669  9311 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:04.237178  6457 rpcz_store.cc:276] 0709 12:26:02.974996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.225210 (+1250214us) service_pool.cc:224] Handling call
0709 12:26:04.225512 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.231027 (+  5515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.231276 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:04.192711  9362 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:04.260157  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 144175us
I20260709 12:26:04.238045  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.261150  6457 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [10.5-11.6]
I20260709 12:26:04.261678  6382 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:04.192674  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:04.196789  9404 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:04.258900  9351 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.182077  9454 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:26:04.237610  9305 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:04.274708  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.295513  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.171777  9446 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.274322  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1974) took 1285 ms. Trace:
I20260709 12:26:04.176151  9399 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.171314  9303 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.311427  6457 rpcz_store.cc:276] 0709 12:26:02.978286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.243106 (+1264820us) service_pool.cc:224] Handling call
0709 12:26:04.259776 (+ 16670us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.261654 (+  1878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.263388 (+  1734us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:04.296448  9629 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
W20260709 12:26:04.319514  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 267816us
I20260709 12:26:04.316243  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 339) took 1189 ms. Trace:
I20260709 12:26:04.260614  9630 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 12:26:04.272820  9532 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.274730  9593 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:04.323541  6457 rpcz_store.cc:276] 0709 12:26:03.127346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.315432 (+1188086us) service_pool.cc:224] Handling call
0709 12:26:04.315731 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.315917 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.315944 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:04.315987 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:04.316042 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.316094 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.316191 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:04.324453  6201 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.268941  9642 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:04.330610  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1931) took 1010 ms. Trace:
I20260709 12:26:04.331079  6301 rpcz_store.cc:276] 0709 12:26:03.319904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.320082 (+   178us) service_pool.cc:224] Handling call
0709 12:26:03.332097 (+ 12015us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.332343 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.332371 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.356386 (+ 24015us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.726749 (+370363us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:04.140308 (+413559us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.140782 (+   474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.140887 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.330327 (+189440us) raft_consensus.cc:1685] finished
0709 12:26:04.330350 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.330526 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12668,"prepare.queue_time_us":215400,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":15124,"raft.queue_time_us":181,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5586,"thread_start_us":693667,"threads_started":2,"wal-append.queue_time_us":636814}
I20260709 12:26:04.334556  9566 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:04.335034  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.325757  9446 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:04.336920  9532 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.337462  9532 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.338052  9615 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:04.339322  9650 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:26:04.340096  9426 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:04.340562  9426 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:04.325888  9624 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:26:04.343817  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1929) took 1025 ms. Trace:
W20260709 12:26:04.344175  9625 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:26:04.344964  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2436) took 2135 ms. Trace:
I20260709 12:26:04.345664  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1933) took 1025 ms. Trace:
I20260709 12:26:04.346630  9598 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:04.348733  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 334) took 1417 ms. Trace:
I20260709 12:26:04.349360  6450 rpcz_store.cc:276] 0709 12:26:02.930878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.449851 (+518973us) service_pool.cc:224] Handling call
0709 12:26:03.450204 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.450697 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.450729 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.451309 (+   580us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.727391 (+276082us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:04.039360 (+311969us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.046609 (+  7249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.046708 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.348427 (+301719us) raft_consensus.cc:1685] finished
0709 12:26:04.348459 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.348645 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":486945,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":1152,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":7175,"spinlock_wait_cycles":27904,"thread_start_us":556243,"threads_started":2,"wal-append.queue_time_us":521046}
I20260709 12:26:04.351660  9626 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:04.346042  6305 rpcz_store.cc:276] 0709 12:26:03.320714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.385158 (+ 64444us) service_pool.cc:224] Handling call
0709 12:26:03.385577 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.385966 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.385998 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.388232 (+  2234us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.723680 (+335448us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:03.745995 (+ 22315us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.755244 (+  9249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.755328 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.343534 (+588206us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:04.345410 (+  1876us) raft_consensus.cc:1685] finished
0709 12:26:04.345502 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.345608 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11504,"prepare.queue_time_us":40860,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":7594,"raft.queue_time_us":10781,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":32130,"spinlock_wait_cycles":82986624,"thread_start_us":304894,"threads_started":1,"wal-append.queue_time_us":627711}
I20260709 12:26:04.359256  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:04.361511  6450 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:04.362848  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1970) took 1389 ms. Trace:
I20260709 12:26:04.363428  6454 rpcz_store.cc:276] 0709 12:26:02.973934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.181784 (+1207850us) service_pool.cc:224] Handling call
0709 12:26:04.182137 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.211995 (+ 29858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.212028 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.212555 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.314414 (+101859us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:04.314458 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.314573 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.362577 (+ 48004us) raft_consensus.cc:1685] finished
0709 12:26:04.362606 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.362774 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6067,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":101371,"threads_started":1,"wal-append.queue_time_us":140776}
I20260709 12:26:04.298974  9517 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:04.375290  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 114240us
I20260709 12:26:04.377142  6457 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:04.379106  6457 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:04.362270  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 341) took 1175 ms. Trace:
I20260709 12:26:04.385521  6450 rpcz_store.cc:276] 0709 12:26:03.187288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.360850 (+1173562us) service_pool.cc:224] Handling call
0709 12:26:04.361261 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.361985 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.362203 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:04.344203  6309 rpcz_store.cc:276] 0709 12:26:03.318427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.318632 (+   205us) service_pool.cc:224] Handling call
0709 12:26:03.355777 (+ 37145us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.387772 (+ 31995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.387807 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.400551 (+ 12744us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.891837 (+491286us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:04.197143 (+305306us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.197414 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.197485 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.343551 (+146066us) raft_consensus.cc:1685] finished
0709 12:26:04.343568 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.343751 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30586,"prepare.queue_time_us":98323,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":19215,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":15825,"thread_start_us":733324,"threads_started":2,"wal-append.queue_time_us":646380}
I20260709 12:26:04.345335  6451 rpcz_store.cc:276] 0709 12:26:02.209670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.887927 (+678257us) service_pool.cc:224] Handling call
0709 12:26:02.888334 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.829749 (+941415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.344742 (+514993us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54001e2dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:04.344904 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2121157376}
I20260709 12:26:04.272258  9474 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:26:04.391728  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66483.5us
W20260709 12:26:04.391991  6350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.016s	sys 0.029s
W20260709 12:26:04.394608  6350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.016s	sys 0.029s
I20260709 12:26:04.401636  6450 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:04.402598  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 671) took 1171 ms. Trace:
I20260709 12:26:04.407968  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.408702  6377 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:26:04.403224  7329 rpcz_store.cc:276] 0709 12:26:03.231267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.231465 (+   198us) service_pool.cc:224] Handling call
0709 12:26:03.236657 (+  5192us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.399325 (+162668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.399353 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.399699 (+   346us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.828906 (+429207us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:03.828966 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.829061 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.402209 (+573148us) raft_consensus.cc:1685] finished
0709 12:26:04.402243 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.402457 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":941,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":1532,"thread_start_us":356860,"threads_started":1,"wal-append.queue_time_us":587574}
I20260709 12:26:04.348774  9537 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:04.413745  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.415014  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.417856  6454 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:04.420801  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.421679  6385 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:04.428588  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.429425  6378 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:04.429129  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.429929  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.430639  6376 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.430919  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.431632  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.431967  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.432503  6382 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:04.432652  6380 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:26:04.433024  6382 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:04.434371  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 336) took 1486 ms. Trace:
I20260709 12:26:04.435331  6453 rpcz_store.cc:276] 0709 12:26:02.948064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.077335 (+1129271us) service_pool.cc:224] Handling call
0709 12:26:04.077698 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.077892 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:04.270226 (+192334us) log.cc:844] Serialized 14 byte log entry
0709 12:26:04.434111 (+163885us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:04.434158 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:04.434201 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.434246 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.434326 (+    80us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":136927,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"thread_start_us":351489,"threads_started":2}
I20260709 12:26:04.437080  6385 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:04.438045  6453 raft_consensus.cc:1217] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-11.6]   Dedup: 8.5->[11.6-11.6]
I20260709 12:26:04.397492  9642 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:04.411048  9519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.025s	user 0.012s	sys 0.006s
I20260709 12:26:04.318794  9305 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.402536  9630 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:26:04.442788  9652 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:04.443003  6379 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:26:04.434149  6382 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.456453  6382 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:04.400027  9598 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:26:04.399078  9404 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.445127  9566 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:04.418970  9517 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
W20260709 12:26:04.446053  9614 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.001s
W20260709 12:26:04.428987  9357 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.868s	user 0.014s	sys 0.000s
I20260709 12:26:04.446456  6201 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.438948  6453 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [7.4-11.6]
I20260709 12:26:04.453239  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.459954  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.470083  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:26:04.463477  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:04.460754  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54666.6us
I20260709 12:26:04.440016  9580 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.489338  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.490545  6385 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:04.491453  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.492242  6385 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:04.450601  9519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.013s	sys 0.006s
I20260709 12:26:04.494334  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.483346  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.494946  6379 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:04.488124  6380 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:04.493522  6376 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:04.498433  6380 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:04.499593  6376 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:04.500176  6381 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:26:04.508677  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 113894us
W20260709 12:26:04.468214  9590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.005s	sys 0.009s
I20260709 12:26:04.462054  9626 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:04.515314  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.514432  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:04.523005  6382 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:04.524660  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.527462  6381 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:04.524405  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:04.526571  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.516496  6380 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:04.530772  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:04.514467  9590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.141s	user 0.005s	sys 0.009s
W20260709 12:26:04.513012  9595 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.714s	user 0.004s	sys 0.006s
I20260709 12:26:04.535369  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.542526  6380 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:04.543669  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:04.542309  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59053us
I20260709 12:26:04.547470  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.518263  9654 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.551447  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.517591  9593 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:04.552217  6376 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:04.489952  9614 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.524385  9600 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:04.561053  6385 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:04.513198  9303 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:26:04.562629  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:04.526186  9512 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.154s	user 0.035s	sys 0.046s
I20260709 12:26:04.569537  6383 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:04.559543  6201 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:04.535271  9474 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:04.534969  9364 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
W20260709 12:26:04.574237  9512 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.202s	user 0.035s	sys 0.046s
I20260709 12:26:04.580631  6380 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:04.581966  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.584856  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:04.585975  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.531936  9357 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:26:04.535211  9399 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:04.535043  9613 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.042s
W20260709 12:26:04.593691  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68826.5us
I20260709 12:26:04.595258  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.567783  9626 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:04.599159  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.576139  9598 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:04.603875  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.551326  9558 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.605325  6376 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:04.551555  9595 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.606353  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.589725  9364 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:04.599355  9593 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:04.616916  9667 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:04.654800  6379 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:04.614538  9558 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:04.657068  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:04.657773  6379 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:26:04.657347  9474 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:04.658689  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.659104  6379 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:04.599252  9589 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.660315  6379 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:26:04.660657  9481 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.000s
I20260709 12:26:04.661067  9532 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.661653  6379 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:26:04.661765  9589 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:04.609805  9664 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
W20260709 12:26:04.625036  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.591575  6201 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 2, term changed from 4 to 6, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to <none>, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added, no longer has a pending config: VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65), VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66), VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67), NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 6 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:26:04.636073  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.636355  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.665432  6382 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.639418  6457 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:04.666987  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:04.640648  6450 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:04.606087  9598 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:04.668642  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:04.669194  6450 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.564568  9654 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.672469  6377 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:04.605526  9590 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:04.650947  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.650832  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:04.651523  6385 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:26:04.678169  6451 raft_consensus.cc:2368] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5.
W20260709 12:26:04.653204  9657 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:26:04.653993  9655 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:04.656291  7329 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:04.675561  6378 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:04.679195  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:04.646062  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 131734us
W20260709 12:26:04.686599  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50596.5us
I20260709 12:26:04.687362  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.683770  6201 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 4, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 4 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:04.689914  9658 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:04.638321  9593 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:04.692876  9593 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:04.556649  9642 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.697640  7332 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:26:04.677266  9590 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:04.602964  9641 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.271s
I20260709 12:26:04.698923  9656 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.699255  9590 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.699556  9656 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.700376  9590 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:04.701567  9595 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:04.701779  9656 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:04.704851  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.705668  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.698254  7320 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:26:04.713678  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.715989  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:04.716600  6450 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:04.718119  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:04.725241  7319 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:04.731653  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.732589  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.734256  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:04.736644  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.686337  9446 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:04.605396  9653 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.291s
I20260709 12:26:04.744320  6201 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 7, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 7 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.746752  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:04.752206  6300 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:04.727322  9328 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:04.723160  9378 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:26:04.735064  9311 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:04.749135  9469 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.000s
I20260709 12:26:04.757552  6201 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 4, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 4 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:04.661082  9481 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.760036  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:04.763746  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 93282.6us
I20260709 12:26:04.748123  9528 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:04.767590  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:04.779505  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.749912  9630 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.752548  9667 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:04.728077  9558 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:26:04.778780  6348 leader_election.cc:400] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5.
I20260709 12:26:04.755067  9512 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:04.786334  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.790536  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:04.757153  9378 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:04.846390  6382 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
W20260709 12:26:04.806703  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.763959  9573 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:26:04.848820  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 81338.2us
I20260709 12:26:04.853323  6348 leader_election.cc:403] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:04.854240  7314 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:04.781932  9652 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.847308  9446 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.883522  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:04.851038  9614 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:04.847283  9519 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:26:04.869096  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78722.7us
W20260709 12:26:04.888614  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 110049us
W20260709 12:26:04.898449  6348 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.663s (SENT)
I20260709 12:26:04.885984  9311 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:04.778393  9656 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:26:04.855114  9667 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:04.965288  9519 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:04.965703  9519 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:04.925287  9573 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:26:04.966126  9667 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.967269  9573 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:04.967855  9573 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:04.888901  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.898537  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:04.968923  9573 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:04.969719  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.970182  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.970060  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71619.4us
W20260709 12:26:04.970844  6275 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.643s (SENT)
I20260709 12:26:04.971087  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:04.850337  9580 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:04.972584  9580 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:04.972839  9652 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:04.972999  9580 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:04.973335  9652 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.973592  9580 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:04.973819  9652 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:04.877687  6201 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 5, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:04.975550  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:04.976754  9656 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.977232  9656 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.979283  9656 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:04.979792  9656 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:04.980136  9663 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:04.980520  9663 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:04.981448  9663 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.844326  9630 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.935711  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:04.983076  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:04.887676  9589 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.939896  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.940416  9669 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:04.895614  9653 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5.
I20260709 12:26:04.904688  9512 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:04.901309  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:04.952226  9631 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.781s	user 0.001s	sys 0.001s
W20260709 12:26:04.953402  9637 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.683s	user 0.000s	sys 0.002s
I20260709 12:26:04.899278  9528 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:04.756279  9598 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.975297  6201 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:05.037393  9305 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:05.036473  9613 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.980566  9656 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:04.890208  9665 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.072845  9665 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.073474  8735 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.074918  9665 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:05.023689  9378 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:04.943758  9674 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.075356  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.075970  6308 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:05.075778  9674 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.076990  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:04.970774  9654 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:04.987522  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.040683  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:05.078362  6308 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:26:05.092916  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.093961  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:05.081692  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:05.041054  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:05.095117  6310 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:05.036693  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 678) took 1164 ms. Trace:
I20260709 12:26:05.049769  6307 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:05.049890  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 148655us
I20260709 12:26:04.970300  9573 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.057864  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.025219  9593 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:04.926258  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:05.109557  6308 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:05.096671  7317 rpcz_store.cc:276] 0709 12:26:03.872645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.902226 (+ 29581us) service_pool.cc:224] Handling call
0709 12:26:03.941254 (+ 39028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.110089 (+168835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.985521 (+875432us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b540016dd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:05.036625 (+ 51104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307149184}
I20260709 12:26:05.069087  9630 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:05.111940  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 233233us
I20260709 12:26:05.070727  9305 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:05.071380  9528 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.058861  9663 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.060614  9446 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:05.041026  6300 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:05.100646  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:05.121667  6307 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:05.101517  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.111389  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:05.075825  9378 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:26:05.071034  9481 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:05.052030  9589 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.116417  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:04.957944  9426 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.078238  9674 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:05.112370  6201 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 5, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:05.163713  9426 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.164232  9589 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:05.164446  6201 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.127475  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2444) took 2475 ms. Trace:
I20260709 12:26:05.130239  6308 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:05.133996  7318 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:05.166101  6201 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:05.166702  6459 rpcz_store.cc:276] 0709 12:26:02.652407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.436022 (+783615us) service_pool.cc:224] Handling call
0709 12:26:03.478178 (+ 42156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.479086 (+   908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.125960 (+1646874us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b5400266950. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:05.127386 (+  1426us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":96038656}
I20260709 12:26:05.096972  9656 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:05.108410  9647 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:05.134163  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:05.169698  7335 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:05.137735  6310 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:05.046307  9519 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.112128  9404 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:26:05.180694  6201 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 2 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:05.148273  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:05.148566  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:05.155403  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:05.156116  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:05.118587  9567 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:26:05.159855  9580 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.161928  9614 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:05.162635  9311 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:05.175853  6300 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:05.177127  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:05.179580  6307 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:05.180828  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1969) took 2244 ms. Trace:
W20260709 12:26:05.182392  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66044.4us
I20260709 12:26:05.187824  9402 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:05.082003  9517 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:26:05.187610  6452 rpcz_store.cc:276] 0709 12:26:02.936395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.480741 (+544346us) service_pool.cc:224] Handling call
0709 12:26:03.481100 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.549726 (+ 68626us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.549777 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.669394 (+119617us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.988910 (+319516us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:03.988956 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.989028 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.951540 (+962512us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:05.167185 (+215645us) raft_consensus.cc:1685] finished
0709 12:26:05.167209 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.180761 (+ 13552us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2205,"raft.queue_time_us":83355,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":6398,"spinlock_wait_cycles":11440384,"thread_start_us":265752,"threads_started":1,"wal-append.queue_time_us":399781}
W20260709 12:26:05.191797  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.191525  6201 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.196070  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:05.194249  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:05.197897  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:05.196820  6201 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 2, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.200461  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:05.203208  6307 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:05.167042  9626 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.209916  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:05.210588  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:05.208742  6452 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:05.212165  6452 raft_consensus.cc:1217] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-11.6]   Dedup: 8.5->[11.6-11.6]
I20260709 12:26:05.212620  6452 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [7.4-11.6]
W20260709 12:26:05.185106  9620 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.019s	sys 0.014s
I20260709 12:26:05.213590  6201 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 4, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:05.178272  9446 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:05.109794  9613 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.216202  6307 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:05.190196  9600 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:26:05.180423  9328 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:26:05.210762  9667 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:26:05.222179  9667 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:05.223847  9446 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.205045  9647 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:26:05.227361  9647 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:05.227715  9647 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:05.228485  9613 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:05.228603  9647 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.230034  9647 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.189280  9531 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.188859  9481 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.236932  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:05.240298  6201 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:05.188463  9614 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:05.215114  9620 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.019s	sys 0.014s
I20260709 12:26:05.243167  7335 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:05.249497  9614 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.265496  9558 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:05.243228  9667 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.267206  9676 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:05.266778  9656 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.270637  9305 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.209481  9519 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:05.282096  6453 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:05.259509  6304 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:05.283597  6304 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:05.266979  9667 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.262457  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:05.260715  6201 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 3, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:05.288807  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 313382us
W20260709 12:26:05.304064  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103631us
I20260709 12:26:05.305568  9600 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.307485  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:05.309023  9311 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:05.311101  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:05.312059  9567 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.204945  9399 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.078s
I20260709 12:26:05.198082  9641 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:26:05.313751  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:05.315938  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.291252  6304 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:05.320015  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.235352  9626 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.290606  9404 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:05.339028  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:05.306396  9600 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.321022  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:05.337630  6304 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:05.323941  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.345031  6381 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:26:05.250808  9531 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.346377  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:05.351199  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.319530  9642 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:05.347958  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:05.355355  9531 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:05.357474  6303 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:26:05.321892  9311 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:05.380050  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2438) took 3168 ms. Trace:
W20260709 12:26:05.359145  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.361997  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:05.369738  6303 pending_rounds.cc:85] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 6
I20260709 12:26:05.381377  6303 pending_rounds.cc:107] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:26:05.380528  6455 rpcz_store.cc:276] 0709 12:26:02.211584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.328447 (+1116863us) service_pool.cc:224] Handling call
0709 12:26:03.328963 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.338181 (+  9218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.338226 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.371019 (+ 32793us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.728080 (+357061us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:03.728125 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.728200 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.354312 (+1626112us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b24001ea808. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f8fb0 00007f6fa2e6f488 00007f6fa2e6fbe8 00007f6fa2e47398 00007f6fa2e4276a 00007f6fa2e41bb6
0709 12:26:05.359421 (+  5109us) raft_consensus.cc:1685] finished
0709 12:26:05.359443 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.379966 (+ 20523us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20511,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"spinlock_wait_cycles":48621696,"thread_start_us":356166,"threads_started":1,"wal-append.queue_time_us":570879}
I20260709 12:26:05.352653  9404 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:05.375794  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112201us
I20260709 12:26:05.381771  6303 raft_consensus.cc:2889] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.383980  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:05.387741  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 1967) took 2451 ms (client timeout 1663 ms). Trace:
I20260709 12:26:05.386387  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:05.388362  6458 rpcz_store.cc:269] 0709 12:26:02.935922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.984512 (+ 48590us) service_pool.cc:224] Handling call
0709 12:26:05.387523 (+2403011us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b54000ebd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:05.387657 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878360832}
I20260709 12:26:05.344916  9512 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:05.387528  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.386327  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:05.359812  9632 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:05.369258  9446 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:05.378816  9620 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:05.379452  9328 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.413153  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2439) took 3137 ms. Trace:
I20260709 12:26:05.416508  6456 rpcz_store.cc:276] 0709 12:26:02.275471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.329737 (+1054266us) service_pool.cc:224] Handling call
0709 12:26:03.378988 (+ 49251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.819673 (+440685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.412915 (+1593242us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54001e23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:05.413095 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011985280}
I20260709 12:26:05.352898  9664 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:26:05.389397  9531 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:05.386898  9357 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.359982  9642 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:05.430846  9531 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:05.432339  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:05.396955  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:05.441340  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.409029  9311 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:05.413656  9620 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:05.457350  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.401454  9656 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:26:05.455513  9446 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:05.447777  9528 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:05.420795  9595 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:26:05.426419  9652 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.426003  9328 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.454835  9580 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:05.482129  9632 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.480698  9620 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:05.442862  9357 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.482589  9311 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:05.557600  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.497438  9439 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:05.537570  9710 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
W20260709 12:26:05.567909  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 181588us
W20260709 12:26:05.331166  9589 consensus_queue.cc:1175] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 5 status { last_received { term: 4 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:26:05.578150  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:05.601552  9696 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:05.604050  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.604889  6309 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:05.605909  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.606899  9600 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:05.607219  9692 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.193s	user 0.000s	sys 0.000s
I20260709 12:26:05.607570  9600 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.608084  9600 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:05.608773  9694 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:26:05.612759  9652 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:05.564359  9705 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:05.614279  9705 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:05.616119  9697 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:05.615466  9705 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.617220  9705 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.623965  9311 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.626920  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:05.628057  7320 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:05.629236  9700 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:05.497874  9404 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.637792  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.638541  6310 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:05.639657  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:05.645510  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.491319  9699 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:05.645517  6309 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:05.650439  6456 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:26:05.626168  9481 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:05.556874  9667 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:05.685622  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:05.677537  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:05.687099  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:05.693852  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:05.699450  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:05.680621  9573 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.568s	user 0.014s	sys 0.000s
I20260709 12:26:05.699752  7317 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:05.702886  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:05.701545  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.568614  9641 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260709 12:26:05.704071  6309 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:05.555316  9632 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.599115  9642 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.704741  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:05.695200  9378 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.002s
I20260709 12:26:05.494685  9512 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.767825  6301 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:26:05.712095  9641 consensus_queue.cc:1243] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:05.707191  9714 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:05.783071  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.763453  9573 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:05.757663  9626 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:05.570458  9531 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.763382  9567 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:05.795199  9713 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:05.795771  9713 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.796118  9713 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:05.791875  9573 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.799413  9573 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.769858  9446 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:05.781373  9714 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.817888  9714 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:05.835305  9614 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.571s	user 0.007s	sys 0.005s
I20260709 12:26:05.842453  9614 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.851049  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1832) took 1444 ms. Trace:
I20260709 12:26:05.851591  6383 rpcz_store.cc:276] 0709 12:26:04.406225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.406336 (+   111us) service_pool.cc:224] Handling call
0709 12:26:04.430603 (+ 24267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.574364 (+143761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.850846 (+1276482us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b540022ea54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:05.850995 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319017728}
I20260709 12:26:05.852433  9444 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:05.852908  9444 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:05.856892  9519 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.565s	user 0.001s	sys 0.008s
I20260709 12:26:05.857640  6456 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:05.857535  9446 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:05.859238  6455 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:26:05.859364  9517 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:05.860881  6458 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:26:05.861984  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.854480  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:05.834899  9647 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.604s	user 0.015s	sys 0.001s
I20260709 12:26:05.862864  6457 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:26:05.866957  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.872262  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:05.873073  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.873363  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:05.874023  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:05.875167  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.875566  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.876864  6384 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:05.877303  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.878314  6382 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:05.874049  6381 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:05.887687  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:05.887125  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.853845  9444 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.892025  9444 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:05.893587  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:05.931528  9632 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:05.937968  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:05.935220  9573 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:05.959920  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.962807  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 413) took 2014 ms. Trace:
I20260709 12:26:05.963459  6303 rpcz_store.cc:276] 0709 12:26:03.948089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.140111 (+192022us) service_pool.cc:224] Handling call
0709 12:26:04.140514 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.385442 (+1244928us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.962215 (+576773us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54000b3954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:05.962709 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2797802240}
W20260709 12:26:05.967231  9660 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.398s	user 0.004s	sys 0.000s
W20260709 12:26:05.968936  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.973380  9531 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:05.983687  9531 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:05.999174  9573 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:05.999804  9593 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:06.000607  9531 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:06.001767  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:06.002241  9709 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.245s	user 0.000s	sys 0.000s
I20260709 12:26:06.002655  7331 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:06.004033  9706 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:26:05.982040  7320 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:26:05.988912  9714 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.005609  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 486) took 1331 ms. Trace:
W20260709 12:26:06.006325  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.981431  9620 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:06.005954  9703 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.515s	user 0.000s	sys 0.001s
I20260709 12:26:06.010996  6376 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.030246  9662 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:06.030776  9662 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:06.031915  9662 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.034842  9662 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:06.035074  9614 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:06.035836  6383 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:06.036458  6385 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:26:06.039762  9717 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:26:06.041038  9600 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:06.009699  9668 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 12:26:06.042506  9707 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.553s	user 0.004s	sys 0.000s
W20260709 12:26:06.044234  6424 connection.cc:256] RPC call timeout handler was delayed by -1.5704883779999363s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15100000000000002s for a response
W20260709 12:26:06.044727  6424 connection.cc:256] RPC call timeout handler was delayed by -1.3400195709996297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
W20260709 12:26:06.045080  6424 connection.cc:256] RPC call timeout handler was delayed by -1.3310291439997854s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 12:26:06.045579  6424 connection.cc:256] RPC call timeout handler was delayed by -1.2443957859995862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.193s for a response
I20260709 12:26:06.045686  9600 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.045938  6424 connection.cc:256] RPC call timeout handler was delayed by -1.0089048959998763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
I20260709 12:26:06.046212  9600 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.048617  9600 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:06.006136  7314 rpcz_store.cc:276] 0709 12:26:04.673891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.674142 (+   251us) service_pool.cc:224] Handling call
0709 12:26:04.691338 (+ 17196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.885796 (+194458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.005354 (+1119558us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:06.005536 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374083840}
W20260709 12:26:06.053531  6274 connection.cc:256] RPC call timeout handler was delayed by -1.7594565539998257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
W20260709 12:26:06.053970  6274 connection.cc:256] RPC call timeout handler was delayed by -1.7099152029995821s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
I20260709 12:26:06.054177  6379 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 3. Current term is 4. Ops: []
I20260709 12:26:06.013789  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:06.054908  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.056080  6381 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:06.054421  6274 connection.cc:256] RPC call timeout handler was delayed by -1.7018152030000238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20379999999999998s for a response
W20260709 12:26:06.059898  6274 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.642s (SENT)
I20260709 12:26:06.060297  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.027115  9454 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:06.052979  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.062068  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.062455  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.058869  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.055320  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:06.063935  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.064428  6379 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:06.064045  6274 leader_election.cc:336] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.709s (SENT)
I20260709 12:26:06.065084  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:06.066700  6274 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.795s (SENT)
I20260709 12:26:06.067154  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.069540  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:06.069923  6274 connection.cc:256] RPC call timeout handler was delayed by -1.6802152029999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
W20260709 12:26:06.011302  9729 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.075s	user 0.003s	sys 0.000s
W20260709 12:26:06.070598  6274 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.856s (SENT)
W20260709 12:26:06.072036  6274 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.856s (SENT)
I20260709 12:26:06.072496  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:06.007839  9718 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 12:26:06.074729  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.084578  6385 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 5. Ops: []
I20260709 12:26:06.011835  9653 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:06.086858  9531 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:06.096073  9378 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:06.083760  9558 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:26:06.062095  9454 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:06.071060  9404 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:06.107271  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.111685  9679 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:06.126528  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.128535  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.068889  9647 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.071388  9303 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:26:06.086995  9614 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:06.087497  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.077813  6382 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:26:06.074667  9667 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
W20260709 12:26:06.128926  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56896.4us
I20260709 12:26:06.065433  9615 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:06.084903  9600 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:06.133371  9679 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:06.006956  9713 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.139618  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:06.065939  9715 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:26:06.072089  9599 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 12:26:06.142746  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55416us
I20260709 12:26:06.143126  6382 pending_rounds.cc:85] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 5
I20260709 12:26:06.128929  9714 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:06.156179  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1740) took 1168 ms. Trace:
W20260709 12:26:06.146168  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.029s (SENT) [suppressed 48 similar messages]
I20260709 12:26:06.133939  6383 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:06.165019  6379 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:06.086735  9653 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:06.142815  9602 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:06.152976  6382 pending_rounds.cc:107] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:06.196359  6382 raft_consensus.cc:2889] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:06.197923  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2301) took 1759 ms (client timeout 2154 ms). Trace:
I20260709 12:26:06.164278  6381 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:06.151454  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1938) took 1435 ms. Trace:
I20260709 12:26:06.164997  6301 rpcz_store.cc:276] 0709 12:26:04.988238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.039980 (+ 51742us) service_pool.cc:224] Handling call
0709 12:26:06.156004 (+1116024us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:06.156113 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541892864}
I20260709 12:26:06.167874  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:06.200170  6385 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:06.168051  9712 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.378s	user 0.000s	sys 0.003s
W20260709 12:26:06.198441  6378 rpcz_store.cc:269] 0709 12:26:04.438112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.439445 (+  1333us) service_pool.cc:224] Handling call
0709 12:26:06.197740 (+1758295us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:06.197862 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470015872}
I20260709 12:26:06.167510  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.160413  9454 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:06.170219  9723 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:26:06.172539  9713 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:06.154503  9599 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:06.215199  9599 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:06.216534  9305 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.216239  9599 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.217610  9599 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:06.174721  9672 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:06.194864  9667 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:26:06.179913  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.179859  9653 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.181871  9328 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:06.187412  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:06.190609  6376 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:26:06.191916  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.235733  6376 pending_rounds.cc:85] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 7
I20260709 12:26:06.192451  6452 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:26:06.236100  6376 pending_rounds.cc:107] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:26:06.236207  6452 pending_rounds.cc:85] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 4
I20260709 12:26:06.236608  6376 raft_consensus.cc:2889] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.193931  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:06.152666  9679 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.163028  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.199538  6305 rpcz_store.cc:276] 0709 12:26:04.716295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.716492 (+   197us) service_pool.cc:224] Handling call
0709 12:26:04.741932 (+ 25440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.782915 (+ 40983us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.782952 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:04.783001 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:04.783046 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.151202 (+1368156us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b54001b2bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:06.151253 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.151392 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29770,"raft.queue_time_us":13775,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":5923,"spinlock_wait_cycles":82995968}
I20260709 12:26:06.208660  6379 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
W20260709 12:26:06.210011  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.212260  9532 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.213146  6458 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [10.5-11.6]
I20260709 12:26:06.242594  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:06.243837  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:06.214023  6455 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:06.245517  9668 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.246694  9668 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.249010  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:06.214787  6459 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.249716  6301 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:06.179366  9674 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:06.251031  9731 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:26:06.251381  9674 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.225791  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:06.254712  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:06.255570  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:06.256364  6309 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:06.257059  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:06.257308  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:06.258136  6458 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:06.225610  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:06.228127  6310 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:06.222812  9672 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:06.233104  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:06.226691  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:06.201299  9469 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:26:06.234452  6456 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:06.193377  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.238420  6453 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:06.166217  9620 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:06.258147  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.236737  6452 pending_rounds.cc:107] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260709 12:26:06.233422  9690 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.744s	user 0.000s	sys 0.004s
I20260709 12:26:06.256119  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:06.258160  6309 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:06.259457  6307 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:06.263039  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.263687  6452 raft_consensus.cc:2889] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.267690  6453 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:26:06.266599  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103669us
I20260709 12:26:06.260910  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:06.273065  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:06.259816  6310 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:06.273465  6451 raft_consensus.cc:2410] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:26:06.260040  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68194.1us
I20260709 12:26:06.276960  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.277122  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:06.197393  9531 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.096s
I20260709 12:26:06.270782  9705 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.261883  9566 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.291594  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.280135  6452 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:26:06.294261  6452 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:06.270820  9600 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:06.282831  9573 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:06.283687  9650 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:06.235518  9404 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.238785  9378 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:06.298789  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:06.299095  9672 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.299569  9567 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:06.300117  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.300549  6305 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:06.300805  9602 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:06.301111  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:06.246776  9558 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:26:06.303395  6452 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:06.321666  6452 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.305099  6384 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:26:06.291160  9653 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:06.332656  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:26:06.319204  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.324892  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.316592  6303 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:06.280153  9532 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:06.337039  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:06.340579  6458 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:06.298849  9715 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:06.306330  9573 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:06.346455  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 200297us
W20260709 12:26:06.347708  6274 connection.cc:256] RPC call timeout handler was delayed by -1.6541152029999466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 12:26:06.350214  9446 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:06.351249  6274 connection.cc:256] RPC call timeout handler was delayed by -1.6072797299998456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1693s for a response
I20260709 12:26:06.307695  9705 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.301826  9303 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:26:06.339766  9713 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:06.353480  6274 connection.cc:256] RPC call timeout handler was delayed by -1.597415202999855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
I20260709 12:26:06.354009  9713 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:06.354741  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.360756  6274 connection.cc:256] RPC call timeout handler was delayed by -1.4032653919998666s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 12:26:06.360997  6458 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:06.361179  6274 connection.cc:256] RPC call timeout handler was delayed by -1.3669797299999118s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
W20260709 12:26:06.361724  6274 connection.cc:256] RPC call timeout handler was delayed by -1.2925898600001346s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17520000000000002s for a response
W20260709 12:26:06.362183  6274 connection.cc:256] RPC call timeout handler was delayed by -1.1665564599998106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21610000000000001s for a response
I20260709 12:26:06.356997  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.302073  9699 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:06.343137  9672 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:06.365886  6458 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:26:06.315496  9714 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 12:26:06.364708  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.344787  9328 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:06.380069  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.383811  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:06.345045  9567 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:06.348982  9573 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:06.333767  9378 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:06.401705  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.361744  9303 consensus_queue.cc:1243] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:06.383535  9602 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:06.365368  9647 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:06.398260  9714 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:06.393911  9537 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:06.320276  9404 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:06.420174  9705 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:06.365603  9713 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.422537  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.432930  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1751) took 1064 ms. Trace:
I20260709 12:26:06.447167  9567 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.401427  9444 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.450081  9378 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:06.454574  9647 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.409303  9602 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:06.469357  6381 pending_rounds.cc:85] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 8
I20260709 12:26:06.539515  6381 pending_rounds.cc:107] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:26:06.507201  9620 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.539973  6381 raft_consensus.cc:2889] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:06.511600  6302 rpcz_store.cc:276] 0709 12:26:05.368043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.368174 (+   131us) service_pool.cc:224] Handling call
0709 12:26:05.368556 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.383501 (+ 14945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.383539 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.387006 (+  3467us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.164787 (+777781us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b1800168c40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:06.339926 (+175139us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:06.339973 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.340039 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.432531 (+ 92492us) raft_consensus.cc:1685] finished
0709 12:26:06.432553 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.432836 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6099,"raft.queue_time_us":20368,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":15069,"spinlock_wait_cycles":10453888,"thread_start_us":174758,"threads_started":1,"wal-append.queue_time_us":989281}
I20260709 12:26:06.471958  9614 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:06.479562  9642 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:06.416277  9714 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:06.489677  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:06.367130  9615 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 12:26:06.573457  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.573951  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.574256  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.543802  9602 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:06.555234  9378 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:06.528013  9679 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.551528  9620 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.608757  9662 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.657759  9599 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:06.678123  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.621259  9714 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.666955  9647 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.669296  9600 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.649976  9662 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:26:06.715375  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 331646us
I20260709 12:26:06.711107  9600 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.738864  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:06.744601  6382 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:06.744601  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:06.745893  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.746924  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:06.747576  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 325128us
I20260709 12:26:06.747903  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.748493  6379 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:06.749241  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:06.753011  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:06.755168  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:06.757679  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 361546us
I20260709 12:26:06.709687  9404 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.688143  9599 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:06.761435  6458 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:06.771585  6382 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:06.777119  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:06.779150  6379 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:26:06.716456  9328 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:06.792966  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:06.800951  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:06.772933  9647 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:06.750722  9531 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 12:26:06.806070  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.780781  9710 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.790871  9404 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:26:06.810909  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.804342  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:06.799903  9737 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
W20260709 12:26:06.832409  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.803717  9444 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:06.839174  9444 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:06.804894  9679 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.806136  9378 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:06.811785  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:06.805840  9647 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:06.838109  9620 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.839766  9444 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:06.860154  9647 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:06.863759  9598 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:06.943557  9311 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:06.940168  9736 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.281s	user 0.002s	sys 0.000s
W20260709 12:26:06.940423  9737 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:06.842957  9710 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:06.942725  9444 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:06.974562  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 139796us
I20260709 12:26:06.951882  9750 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:06.889009  9378 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.080s
I20260709 12:26:06.975481  9662 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.976166  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:06.975839  9741 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
I20260709 12:26:06.862985  9328 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:06.987620  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.980938  9750 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:07.006856  9653 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.002s
W20260709 12:26:06.994385  9746 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:26:06.994544  9599 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:06.995050  9668 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.039829  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:06.995648  9674 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.040788  7316 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:07.001673  9378 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.003754  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.006928  6379 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:07.044284  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.046835  6305 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:07.043931  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.007616  9600 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:07.008380  9615 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:06.967689  9311 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:06.984678  9537 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.558s	user 0.006s	sys 0.006s
I20260709 12:26:07.019012  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:07.019362  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.067629  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.021598  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:06.978688  9404 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:26:07.068418  6302 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
W20260709 12:26:07.024260  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.031327  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.070271  6451 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:07.071601  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.072748  6451 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:26:07.073673  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.031667  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.074875  6452 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:07.076038  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.077067  6452 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 12:26:06.983052  9740 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.233s	user 0.001s	sys 0.001s
I20260709 12:26:07.032225  9765 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:07.079732  9765 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:06.987721  9748 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:26:07.034060  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:07.067570  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:26:07.015633  9573 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.004s
I20260709 12:26:06.976714  9357 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:07.000453  6380 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:07.015177  9749 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:26:07.040571  6459 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:07.040004  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.106041  6304 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:07.042202  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.107245  6309 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
W20260709 12:26:07.057513  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53848.2us
W20260709 12:26:07.037360  9747 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:26:07.035168  9620 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:26:07.090567  6452 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:07.034976  9590 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:26:07.111615  6453 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:26:07.034058  9759 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:26:07.068877  9558 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.057971  9653 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:26:07.092051  9599 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.094429  9537 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:07.120818  6379 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:07.116076  9662 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.118475  9672 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.733s	user 0.013s	sys 0.000s
I20260709 12:26:07.099047  9674 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:07.139806  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 156233us
I20260709 12:26:07.099552  9573 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.153504  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:07.116499  9765 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:07.161283  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.164551  9756 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
I20260709 12:26:07.166672  9766 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:07.154732  9672 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.182633  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 147434us
W20260709 12:26:07.175302  9602 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.002s
I20260709 12:26:07.145381  9599 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.184242  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:07.183614  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:07.151312  9662 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.156805  9404 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:26:07.154947  9674 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.187115  9469 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:07.191347  9710 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.197764  9590 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:07.180881  9444 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:07.232985  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49424.7us
I20260709 12:26:07.187837  9672 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.247218  9772 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.248270  9772 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:07.249378  9772 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:07.250543  9378 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:07.252030  9620 consensus_queue.cc:1243] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 6.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:26:07.253003  9620 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:07.255713  7319 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:07.258307  9714 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:07.259130  9620 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:07.194180  9602 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.264050  9573 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:07.265146  9311 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:07.266947  9444 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.266899  9311 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.267877  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 77781.9us
I20260709 12:26:07.220108  9590 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:07.260547  9620 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.274426  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:07.327550  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53233.3us
W20260709 12:26:07.276582  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75114.9us
I20260709 12:26:07.328858  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.330091  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.273307  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:07.332233  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:07.333563  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60350.9us
W20260709 12:26:07.337167  6384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2322) took 1284 ms (client timeout 1591 ms). Trace:
I20260709 12:26:07.337581  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:07.337598  6384 rpcz_store.cc:269] 0709 12:26:06.052526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.052679 (+   153us) service_pool.cc:224] Handling call
0709 12:26:07.336928 (+1284249us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001f54d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:07.337100 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556367232}
I20260709 12:26:07.250398  9647 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.275394  9766 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:26:07.336939  9653 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:07.345209  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:07.338639  9620 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.338770  9778 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:07.267678  9311 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.343102  9755 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.011s
I20260709 12:26:07.347776  9653 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:07.362581  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.351619  9778 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:07.391396  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53366.3us
I20260709 12:26:07.365110  9653 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:07.363521  9779 raft_consensus.cc:2764] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:07.392226  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:07.396685  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64546us
I20260709 12:26:07.350585  9590 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.370783  9662 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.397837  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.360793  9404 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:26:07.368041  9778 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.403856  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:07.410923  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73416.5us
I20260709 12:26:07.395244  9775 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:07.395478  9599 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:07.365499  9777 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:26:07.402459  9674 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.408072  9779 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:07.409665  9755 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.410338  9662 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.436463  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91342.8us
I20260709 12:26:07.445322  9773 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.446136  9773 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.446338  9311 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:07.446687  9773 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.449273  9773 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.423599  9672 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.423892  9590 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:07.450060  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46313.7us
I20260709 12:26:07.423383  9602 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:07.451056  9674 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.452885  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:07.455006  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.443812  9775 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:07.457413  9674 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.464514  9647 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:07.464916  9598 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.503s	user 0.012s	sys 0.001s
W20260709 12:26:07.465673  9642 consensus_meta.cc:287] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.911s	user 0.012s	sys 0.003s
W20260709 12:26:07.479733  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.463004  9785 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:07.495919  6377 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:07.457729  9662 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.480120  9303 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:07.481649  9773 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:07.473968  9775 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.480441  9558 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.468276  9672 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:26:07.498524  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:07.470171  9777 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.026s
W20260709 12:26:07.502542  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 104361us
I20260709 12:26:07.508527  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.510455  6450 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:26:07.495525  9647 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.494618  9785 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:07.508944  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.512809  6451 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:07.511439  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.512336  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:07.508814  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.513650  6310 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.497406  9648 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.497350  9668 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.518989  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.519383  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.519904  6458 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.520051  6459 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:07.522913  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:07.523519  6301 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.525972  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.527639  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:07.528465  6300 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.530037  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.530622  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:07.513654  6307 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:07.534036  6304 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:07.505764  9775 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.511512  9766 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.542402  6309 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:07.548902  9614 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:07.549479  9614 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:07.521764  9785 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:07.550381  6303 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:07.560597  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:07.562772  6310 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:07.533893  9792 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.527966  9598 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.566780  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:07.565418  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:07.567523  6300 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:07.571451  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:07.578327  6301 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:07.503263  9303 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.512513  9558 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:07.600993  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.525946  9648 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.514024  9784 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:26:07.521744  9647 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.524166  9668 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.540259  9793 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.557677  9614 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.619360  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:07.620587  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.579686  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:07.710963  6383 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:07.702226  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 196429us
I20260709 12:26:07.678373  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:07.711932  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.718752  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.718840  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:07.719583  6458 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:07.720662  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:07.664131  6310 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:07.629519  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:07.537868  9674 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:07.658619  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:07.628892  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.727864  6309 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 4.
I20260709 12:26:07.727502  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:07.629019  9791 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:07.699375  9598 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.744697  6301 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.660285  9602 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.627674  9311 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:07.712806  9672 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:07.753082  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:07.766588  6303 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:07.753777  6309 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:07.715550  9788 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.731431  9705 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.564563  9599 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.187s
I20260709 12:26:07.730566  9765 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:07.775504  9311 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.570363  9766 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.740633  9614 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:07.783097  9714 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.002s
I20260709 12:26:07.564538  9790 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.207s
I20260709 12:26:07.745394  9600 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:07.715058  9793 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.805089  9759 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:07.629189  9792 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:07.814126  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.845043  6303 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:07.856153  6310 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:07.766119  9672 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:07.776774  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 205448us
I20260709 12:26:07.886399  9573 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.885768  6300 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:07.893262  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 172698us
W20260709 12:26:07.912523  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 200263us
I20260709 12:26:07.766636  9598 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.771442  9602 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.947371  6450 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.949580  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.884781  9753 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:07.922613  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.930195  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:07.935195  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:07.935799  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:07.920409  6456 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:26:07.965301  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.966758  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:07.894112  9759 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:07.913378  9672 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:07.793727  9705 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.905388  9714 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.950376  6458 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:07.963203  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:07.977572  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:07.888003  9796 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:07.960932  6451 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 4.
I20260709 12:26:07.914250  9662 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:07.910440  9600 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:07.937402  9444 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:07.984592  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48913.2us
I20260709 12:26:07.979394  9404 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:07.978708  9765 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:07.785691  9788 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:07.959275  9674 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:26:08.038100  7314 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:07.947458  9573 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.992519  9794 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.993005  9620 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:08.046865  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 129903us
I20260709 12:26:08.052284  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:08.060887  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:08.069926  7316 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:07.986232  9755 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 12:26:08.013901  9404 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.112502  9303 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:08.059347  9778 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.004s
I20260709 12:26:07.994709  9714 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:07.992607  9789 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 12:26:08.051961  9782 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.122089  9404 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.045939  9806 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:08.036131  9328 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260709 12:26:07.972868  9753 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.060575  9620 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:08.119153  9674 consensus_queue.cc:1243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:26:08.141376  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 166392us
I20260709 12:26:08.010135  9444 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.127398  9647 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.125198  9558 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:08.004750  9796 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:08.137881  9747 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:26:08.211457  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.261152  9814 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 12:26:08.261430  9558 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.262071  9800 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:26:08.262187  9558 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.263928  9573 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.264395  9793 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:08.264696  9558 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:08.265467  9825 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.265894  9813 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.266866  9590 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.816s	user 0.011s	sys 0.002s
I20260709 12:26:08.267067  9311 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.270632  9765 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.405946  9573 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:08.406164  9765 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.406924  9573 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:08.276099  9788 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:08.276185  9714 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:08.284771  9819 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:08.200264  9808 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.409262  9808 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.410825  9598 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.151844  9782 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.184499  9806 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:08.456609  9815 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 12:26:08.411903  9808 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:08.458048  9826 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:26:08.072203  9794 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:08.351066  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53630.3us
I20260709 12:26:08.180940  9620 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:08.258387  9775 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.693s	user 0.000s	sys 0.018s
W20260709 12:26:08.392760  9798 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.426s	user 0.000s	sys 0.001s
I20260709 12:26:08.344234  9674 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:08.386023  9816 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.405567  9311 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.408397  9653 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.008s	user 0.009s	sys 0.002s
I20260709 12:26:08.400450  9614 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.455196  9444 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.455492  9782 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.346309  6300 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:08.131311  9778 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.137504  9600 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:08.707018  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.741513  6459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.035s	sys 0.024s
W20260709 12:26:08.742233  6459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.036s	sys 0.025s
W20260709 12:26:08.743372  6456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.013s	sys 0.048s
W20260709 12:26:08.643678  6450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.047s	sys 0.054s
W20260709 12:26:08.744428  6450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.766s	user 0.055s	sys 0.054s
I20260709 12:26:08.746251  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:08.743958  6456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.013s	sys 0.048s
I20260709 12:26:08.784482  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:08.785631  6450 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:08.787633  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:08.791802  6450 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:26:08.787351  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:08.682195  9747 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:26:08.459774  9705 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.732191  9573 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:08.241873  9805 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:08.793569  9674 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.810456  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2335) took 1249 ms. Trace:
I20260709 12:26:08.914237  6381 rpcz_store.cc:276] 0709 12:26:07.560509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.740525 (+180016us) service_pool.cc:224] Handling call
0709 12:26:07.767050 (+ 26525us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:07.767613 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.767659 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:07.793630 (+ 25971us) log.cc:844] Serialized 52 byte log entry
0709 12:26:08.051230 (+257600us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:08.051642 (+   412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.054052 (+  2410us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.810147 (+756095us) raft_consensus.cc:1685] finished
0709 12:26:08.810176 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.810372 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3855,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":7087,"thread_start_us":90323,"threads_started":1,"wal-append.queue_time_us":737171}
I20260709 12:26:08.915903  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 1989) took 2723 ms. Trace:
W20260709 12:26:08.842226  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.843021  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.937s	user 0.001s	sys 0.010s
I20260709 12:26:08.916507  6453 rpcz_store.cc:276] 0709 12:26:06.192456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.272870 (+ 80414us) service_pool.cc:224] Handling call
0709 12:26:07.111263 (+838393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.123467 (+ 12204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.915585 (+1792118us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b54000c1cd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:08.915822 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1927464704}
W20260709 12:26:08.917357  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.001s	sys 0.010s
W20260709 12:26:08.920166  6379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.099s	sys 0.152s
W20260709 12:26:08.843489  6304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.005s	sys 0.005s
W20260709 12:26:08.920630  6379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.099s	sys 0.152s
I20260709 12:26:08.803821  9602 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:08.888777  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44928.7us
I20260709 12:26:08.804766  9311 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:08.878930  6458 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.004s	sys 0.006s
I20260709 12:26:08.877533  9808 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.645483  9598 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:08.932910  6458 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.005s	sys 0.006s
W20260709 12:26:08.933658  9791 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.175s	user 0.013s	sys 0.000s
I20260709 12:26:08.933154  9808 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:08.919610  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.878238  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90981us
I20260709 12:26:08.934078  9791 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:08.875131  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:08.458660  9792 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:08.921996  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.001s	sys 0.017s
I20260709 12:26:08.924032  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:08.812453  9785 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.195s	user 0.007s	sys 0.004s
I20260709 12:26:08.412560  9813 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.933276  9598 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.935479  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:08.935634  9790 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.936087  9705 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.920763  6304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.006s	sys 0.005s
I20260709 12:26:08.810083  9806 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:08.936928  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.937044  9765 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.804745  9653 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.890759  9782 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:08.894742  9573 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:08.794189  9753 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.181280  9759 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:08.945199  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.002s	sys 0.018s
I20260709 12:26:08.938114  9714 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.886680  9775 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:08.891899  9823 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:08.918809  9444 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.953377  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:08.956346  6272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.005s	sys 0.022s
I20260709 12:26:08.956239  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2480) took 1017 ms. Trace:
W20260709 12:26:08.984941  6272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.005s	sys 0.022s
W20260709 12:26:08.985628  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.00641e+06us
W20260709 12:26:08.957901  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.019s	sys 0.009s
W20260709 12:26:08.987454  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.019s	sys 0.009s
I20260709 12:26:08.987138  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:08.958007  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:08.401371  9825 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:08.954442  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1767) took 1419 ms. Trace:
I20260709 12:26:08.886883  9614 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:08.348143  9773 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:08.965054  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 782890us
I20260709 12:26:08.963328  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:08.799726  9816 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:08.951028  9829 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:08.553880  9796 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:08.960793  9782 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:08.987892  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 948995us
I20260709 12:26:09.014744  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:08.953521  9705 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:08.954633  9598 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.954001  9837 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:08.955502  9808 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:08.954963  9806 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:08.953188  9790 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.963125  9765 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:26:09.043252  6450 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:08.995117  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:08.985077  6456 rpcz_store.cc:276] 0709 12:26:07.939177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.964071 (+ 24894us) service_pool.cc:224] Handling call
0709 12:26:07.964426 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.146238 (+181812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.146259 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.146681 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.777945 (+631264us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:08.778009 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.778117 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:08.956031 (+177914us) raft_consensus.cc:1685] finished
0709 12:26:08.956050 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.956181 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25175,"raft.queue_time_us":91986,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":49152,"thread_start_us":801485,"threads_started":2,"wal-append.queue_time_us":802092}
I20260709 12:26:08.990633  6304 rpcz_store.cc:276] 0709 12:26:07.535437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.535592 (+   155us) service_pool.cc:224] Handling call
0709 12:26:07.546359 (+ 10767us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.546578 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:07.898091 (+351513us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.953944 (+1055853us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.954023 (+    79us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.954110 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.954195 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:08.954363 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10413,"prepare.queue_time_us":30982,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":23,"raft.queue_time_us":2206,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":2325,"thread_start_us":1389584,"threads_started":2,"wal-append.queue_time_us":1057552}
I20260709 12:26:08.963001  9839 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.989113  7314 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:09.023447  9782 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:08.992452  9823 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:09.058823  9838 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:26:09.058961  9705 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.044802  9598 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:09.059777  9598 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.059504  9823 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.061174  9823 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.061592  9825 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:09.061892  9598 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:08.964221  9602 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:26:08.976052  9714 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.922096  9805 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:08.954432  9831 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 12:26:09.052634  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:08.965595  9573 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.014277  9813 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:09.055233  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60232.1us
I20260709 12:26:09.056404  9791 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:09.056676  9311 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:09.056879  9835 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:26:09.013441  9829 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.057998  9614 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:09.043205  9837 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:08.997812  9792 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:08.949566  9785 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.044656  9806 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.075690  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1875) took 1031 ms. Trace:
I20260709 12:26:09.003450  9444 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.005977  9773 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.082587  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:09.094897  9829 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.096642  6381 rpcz_store.cc:276] 0709 12:26:08.044032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.039942 (+995910us) service_pool.cc:224] Handling call
0709 12:26:09.075584 (+ 35642us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.051903  9808 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:09.070468  9753 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
W20260709 12:26:09.107430  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 158449us
W20260709 12:26:09.100349  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.035s	user 0.135s	sys 0.313s
I20260709 12:26:09.108853  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:09.109859  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.045s	user 0.135s	sys 0.313s
W20260709 12:26:09.113615  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.04891e+06us
I20260709 12:26:09.095403  9785 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.116616  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:09.097664  9806 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.116982  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:09.121634  6381 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:09.098179  9790 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.064653  9839 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.090065  9837 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.101001  9791 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:09.102025  9778 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.129690  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2018) took 1127 ms. Trace:
W20260709 12:26:09.130419  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 122747us
I20260709 12:26:09.130239  6459 rpcz_store.cc:276] 0709 12:26:08.002320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.126901 (+124581us) service_pool.cc:224] Handling call
0709 12:26:08.127291 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.138211 (+ 10920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.138259 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.186712 (+ 48453us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.742831 (+556119us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:26:08.742906 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.743010 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.129423 (+386413us) raft_consensus.cc:1685] finished
0709 12:26:09.129446 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.129617 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10443,"raft.queue_time_us":156692,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":555706,"threads_started":1,"wal-append.queue_time_us":768848}
I20260709 12:26:09.134111  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.135133  6459 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:09.135603  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.136669  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:09.134095  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.116467  9808 consensus_queue.cc:1243] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:26:09.142076  6301 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.039s	sys 0.115s
W20260709 12:26:09.149735  6301 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.039s	sys 0.115s
W20260709 12:26:09.142921  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 117480us
I20260709 12:26:09.151448  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 435) took 1396 ms. Trace:
I20260709 12:26:09.121322  9653 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.124290  9829 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:26:09.147550  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 212207us
I20260709 12:26:09.152058  6301 rpcz_store.cc:276] 0709 12:26:07.754103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.754326 (+   223us) service_pool.cc:224] Handling call
0709 12:26:07.761288 (+  6962us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:07.761499 (+   211us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:07.765293 (+  3794us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.150336 (+1385043us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.150413 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:09.150509 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.150612 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.150786 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6832,"prepare.queue_time_us":3900,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":30532,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":33513,"thread_start_us":1381259,"threads_started":2,"wal-append.queue_time_us":350833}
I20260709 12:26:09.126498  9614 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:09.152694  7316 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:09.158722  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.163817  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.164490  7314 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:09.126463  9674 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.127074  9841 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.132611  9816 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.128214  9791 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:09.165334  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.168639  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:09.168700  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 206266us
I20260709 12:26:09.186777  6458 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:09.168102  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:09.187783  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1870) took 1145 ms. Trace:
I20260709 12:26:09.187808  7317 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:09.188364  6381 rpcz_store.cc:276] 0709 12:26:08.042704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.100784 (+1058080us) service_pool.cc:224] Handling call
0709 12:26:09.187713 (+ 86929us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.169207  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.153318  9794 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.190003  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:09.191722  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.133178  9311 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:26:09.191815  6458 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.191648  6459 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:09.196050  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.196698  6459 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:09.197645  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.198167  6459 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:09.189407  9791 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:09.199791  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.151366  9849 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.207870  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:09.152034  9602 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:26:09.209201  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:09.184244  9842 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.184561  9829 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:09.188141  9614 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:09.192495  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.252591  9664 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:09.330233  9664 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:09.331207  9664 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:09.333904  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 146111us
I20260709 12:26:09.335649  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:09.336367  9404 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.152s	user 0.012s	sys 0.000s
I20260709 12:26:09.337954  6458 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:09.237849  9444 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.339493  9444 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.261682  9668 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.326249  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2019) took 1323 ms. Trace:
I20260709 12:26:09.233330  9620 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:09.238348  9738 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.484s	user 0.009s	sys 0.006s
I20260709 12:26:09.208921  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.342774  6459 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:09.344817  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:09.265245  9672 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.271s	user 0.013s	sys 0.000s
I20260709 12:26:09.346570  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 366) took 3157 ms. Trace:
I20260709 12:26:09.236413  9773 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.348063  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.347743  9672 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.250137  9573 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.349017  9773 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.267733  9714 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.350014  9773 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.347163  6457 rpcz_store.cc:276] 0709 12:26:06.189374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.193451 (+  4077us) service_pool.cc:224] Handling call
0709 12:26:06.193788 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:06.194251 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.194283 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:06.194884 (+   601us) log.cc:844] Serialized 52 byte log entry
0709 12:26:06.527513 (+332629us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:06.835871 (+308358us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.875779 (+ 39908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.875904 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.463192 (+587288us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b1800492b20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:09.237582 (+1774390us) spinlock_profiling.cc:243] Waited 19.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:09.345946 (+108364us) raft_consensus.cc:1685] finished
0709 12:26:09.345998 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.346287 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42689,"prepare.queue_time_us":163789,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":40270,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":13090,"spinlock_wait_cycles":53525504,"thread_start_us":612648,"threads_started":2,"wal-append.queue_time_us":498579}
I20260709 12:26:09.353052  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.353088  9714 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:09.353266  9773 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.340449  9444 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.215518  9775 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.355420  6381 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:09.356033  9778 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.322901  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.356578  9573 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:09.356781  9778 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.357671  9573 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.359910  9778 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.360002  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.361335  9573 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.360556  9778 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:09.327620  9614 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.363070  9661 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.340453  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.364457  9661 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.365527  6304 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:09.336520  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.366987  9672 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.363035  9778 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.337442  9404 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:09.367619  9661 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.354049  9714 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:09.340667  6453 rpcz_store.cc:276] 0709 12:26:08.002687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.939830 (+937143us) service_pool.cc:224] Handling call
0709 12:26:08.940260 (+   430us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.127707 (+187447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.127741 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.133485 (+  5744us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.325261 (+191776us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:09.325401 (+   140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.325551 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.325640 (+    89us) raft_consensus.cc:1685] finished
0709 12:26:09.325677 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.325950 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2861,"raft.queue_time_us":19060,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14684,"thread_start_us":325831,"threads_started":2,"wal-append.queue_time_us":130107}
I20260709 12:26:09.371011  6457 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:09.373926  9714 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.371853  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.393756  6303 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:26:09.394711  6453 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:09.373235  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.356845  9620 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.396241  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.375881  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.396965  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:09.397719  6457 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 5.
I20260709 12:26:09.379230  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:09.381009  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:09.406797  9785 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:09.361367  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.407770  9785 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.409124  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:09.411013  9573 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:26:09.235984  9805 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.346046  6459 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:09.389945  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.415081  9833 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:09.415673  9833 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.416078  9833 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.422044  6301 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:09.421293  9785 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.356688  9444 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.281908  9782 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.426998  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:09.429320  9782 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.400084  6307 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:26:09.401976  9781 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.970s	user 0.000s	sys 0.004s
I20260709 12:26:09.433395  6307 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:09.369659  9778 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:09.236896  9812 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.965s	user 0.001s	sys 0.008s
I20260709 12:26:09.340472  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:09.411954  9797 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.673s	user 0.004s	sys 0.006s
I20260709 12:26:09.265942  9829 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:09.237195  9828 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:26:09.445628  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2332) took 1886 ms. Trace:
I20260709 12:26:09.446244  6378 rpcz_store.cc:276] 0709 12:26:07.558994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.559147 (+   153us) service_pool.cc:224] Handling call
0709 12:26:07.572491 (+ 13344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.617865 (+ 45374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.617895 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.658447 (+ 40552us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.982416 (+323969us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:07.982464 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.982562 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.236145 (+1253583us) spinlock_profiling.cc:243] Waited 34.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:09.444731 (+208586us) raft_consensus.cc:1685] finished
0709 12:26:09.444782 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.445253 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":55924,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":40026,"spinlock_wait_cycles":80115072,"thread_start_us":284340,"threads_started":1,"wal-append.queue_time_us":574108}
I20260709 12:26:09.448402  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2337) took 1887 ms. Trace:
I20260709 12:26:09.406675  6304 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:09.412469  9573 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:09.287719  9806 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.413426  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.369926  9404 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.265144  9837 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.257275  9839 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.186947  9841 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.237636  9842 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.432667  9794 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:09.430114  9808 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:09.401795  9791 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.435809  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.436628  9840 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:09.437151  9789 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:09.449663  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.445043  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:09.247330  9846 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.205850  9849 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.626369  9849 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:09.251084  9853 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:09.631749  9853 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:09.471153  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.473153  6303 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:09.473338  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2011) took 1966 ms. Trace:
I20260709 12:26:09.474210  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.477212  9444 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.478549  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.607085  6310 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:09.492158  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.448956  6379 rpcz_store.cc:276] 0709 12:26:07.560982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.740832 (+179850us) service_pool.cc:224] Handling call
0709 12:26:07.780476 (+ 39644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.907034 (+126558us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:07.907083 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.907780 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.921299 (+1013519us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.921351 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.921429 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.447786 (+526357us) raft_consensus.cc:1685] finished
0709 12:26:09.447838 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.448156 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":139489,"raft.queue_time_us":127733,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":34156,"thread_start_us":924697,"threads_started":1,"wal-append.queue_time_us":1105369}
I20260709 12:26:09.596833  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.488247  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.499006  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.487649  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:09.612880  9808 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.603014  6307 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.465426  9785 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:09.640841  7317 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:09.641393  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.641657  9602 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:09.642741  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.606597  9404 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:09.636631  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1871) took 1581 ms (client timeout 2084 ms). Trace:
I20260709 12:26:09.617852  9446 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:09.646005  6378 rpcz_store.cc:269] 0709 12:26:08.042986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.449141 (+1406155us) service_pool.cc:224] Handling call
0709 12:26:09.624749 (+175608us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.643504  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.655705  6458 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.479045  9573 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.659722  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:09.637996  9853 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.647145  9808 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.673493  9404 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.680608  9446 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.686102  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:09.686978  6457 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:09.674870  6310 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:09.688230  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.689339  6310 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:09.674690  9602 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.689411  9404 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.691550  9454 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:09.692710  9602 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:09.683017  9862 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:09.683480  6456 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:09.696647  9785 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.700076  9785 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.681191  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.633546  6455 rpcz_store.cc:276] 0709 12:26:07.506911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.507034 (+   123us) service_pool.cc:224] Handling call
0709 12:26:07.523666 (+ 16632us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:07.531564 (+  7898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.531586 (+    22us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:07.531970 (+   384us) log.cc:844] Serialized 52 byte log entry
0709 12:26:07.672162 (+140192us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:07.672224 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.672407 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.410337 (+1737930us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:09.472746 (+ 62409us) raft_consensus.cc:1685] finished
0709 12:26:09.472778 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.473140 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9771,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":3276,"spinlock_wait_cycles":23347072,"thread_start_us":139874,"threads_started":1,"wal-append.queue_time_us":84803}
I20260709 12:26:09.699668  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:09.700677  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.672144  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2342) took 1536 ms. Trace:
W20260709 12:26:09.702548  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1869) took 1660 ms (client timeout 1906 ms). Trace:
I20260709 12:26:09.703625  9765 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:26:09.665385  9444 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.704519  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:09.716158  6385 rpcz_store.cc:276] 0709 12:26:08.136299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.498436 (+1362137us) service_pool.cc:224] Handling call
0709 12:26:09.672060 (+173624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:09.715209  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.720199  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.721916  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.722360  9806 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.722668  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.723624  6378 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:09.724282  9837 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.724803  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.725734  6378 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:09.726135  9444 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.726773  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.727686  6379 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:09.728637  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:09.729130  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:09.731030  9829 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:09.732519  6455 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:09.733300  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:09.733598  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:09.734280  6303 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:09.734858  6455 raft_consensus.cc:2410] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260709 12:26:09.735819  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:09.716910  6381 rpcz_store.cc:269] 0709 12:26:08.042342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.191451 (+1149109us) service_pool.cc:224] Handling call
0709 12:26:09.702182 (+510731us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540023d7d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:09.702407 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372371712}
I20260709 12:26:09.715665  6459 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:09.731213  9833 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.670763  9794 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:09.675685  9846 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.649813  9840 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:10.504539  6458 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.795s	user 0.010s	sys 0.002s
I20260709 12:26:10.031657  9849 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:10.649149  6458 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:10.649986  9840 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:10.650183  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2030) took 1160 ms. Trace:
I20260709 12:26:10.650686  9840 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.650777  6459 rpcz_store.cc:276] 0709 12:26:09.489987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.491736 (+  1749us) service_pool.cc:224] Handling call
0709 12:26:10.650125 (+1158389us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.651401  9840 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:09.719192  9404 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:09.796285  9842 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:09.854635  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 242626us
I20260709 12:26:10.652992  9794 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:09.884753  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:10.045110  9808 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:10.654480  6456 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:10.655233  7320 raft_consensus.cc:2368] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7.
I20260709 12:26:10.655303  9840 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:10.655660  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:09.794016  9648 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.898711  9766 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:09.966815  9664 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.635s	user 0.006s	sys 0.013s
I20260709 12:26:10.656383  6456 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:09.792486  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:10.657176  9664 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:10.658143  6455 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:10.041671  6457 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:09.766688  6453 raft_consensus.cc:2410] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:26:10.650004  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2029) took 1206 ms. Trace:
I20260709 12:26:10.659832  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2035) took 1058 ms. Trace:
I20260709 12:26:10.660216  6458 rpcz_store.cc:276] 0709 12:26:09.444124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.444516 (+   392us) service_pool.cc:224] Handling call
0709 12:26:10.649906 (+1205390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.660498  6457 rpcz_store.cc:276] 0709 12:26:09.601893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.685747 (+ 83854us) service_pool.cc:224] Handling call
0709 12:26:10.659752 (+974005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.666519  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 384) took 1297 ms. Trace:
I20260709 12:26:10.666906  6453 rpcz_store.cc:276] 0709 12:26:09.368827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.397662 (+ 28835us) service_pool.cc:224] Handling call
0709 12:26:10.666401 (+1268739us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:09.780862  9811 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.479s	user 0.003s	sys 0.000s
I20260709 12:26:10.668923  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:10.670073  9328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.006s	sys 0.006s
W20260709 12:26:10.670693  9328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.007s	sys 0.006s
I20260709 12:26:10.671255  9808 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.893793  9750 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:10.651100  9849 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.913802  9676 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.653008  9404 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:10.656337  9846 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:09.787498  9446 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:10.638746  9791 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:10.645869  9830 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 2.369s	user 0.004s	sys 0.047s
I20260709 12:26:10.687207  9783 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:10.687762  9783 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:10.688979  9783 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:10.690176  9783 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:26:10.692363  9378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.003s	sys 0.003s
I20260709 12:26:10.692818  9664 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:10.695184  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:10.682368  9676 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:10.707772  9378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.004s	sys 0.004s
I20260709 12:26:10.747437  7332 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
W20260709 12:26:10.755590  9653 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.000s	sys 0.008s
I20260709 12:26:10.749816  9841 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:10.764050  9653 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.000s	sys 0.009s
W20260709 12:26:10.765965  9807 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.392s	user 0.004s	sys 0.016s
W20260709 12:26:10.776409  9807 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.502s	user 0.004s	sys 0.017s
I20260709 12:26:10.785207  9833 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:10.795029  9787 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 3.367s	user 0.006s	sys 0.000s
I20260709 12:26:10.798568  9404 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:10.842132  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.852092  9853 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.169s	user 0.014s	sys 0.000s
I20260709 12:26:10.853425  9853 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:10.861050  9853 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:10.855079  9783 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.862727  9783 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:10.865134  9783 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:09.784427  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:10.912185  6308 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:26:10.915007  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1960) took 1311 ms. Trace:
I20260709 12:26:09.862841  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:10.915467  6308 rpcz_store.cc:276] 0709 12:26:09.603179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.784051 (+180872us) service_pool.cc:224] Handling call
0709 12:26:10.914908 (+1130857us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.916023  6385 raft_consensus.cc:2368] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7.
W20260709 12:26:10.916672  6385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1881) took 1530 ms (client timeout 1693 ms). Trace:
I20260709 12:26:10.916695  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:10.917244  6385 rpcz_store.cc:269] 0709 12:26:09.385960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.738393 (+352433us) service_pool.cc:224] Handling call
0709 12:26:10.916560 (+1178167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.917549  6308 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:10.918987  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1787) took 1424 ms. Trace:
I20260709 12:26:10.919379  6308 rpcz_store.cc:276] 0709 12:26:09.495039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.916452 (+1421413us) service_pool.cc:224] Handling call
0709 12:26:10.918907 (+  2455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.919539  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:10.920524  6385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1883) took 1524 ms (client timeout 2015 ms). Trace:
I20260709 12:26:10.920516  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:10.921525  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1963) took 1317 ms. Trace:
W20260709 12:26:10.921455  6385 rpcz_store.cc:269] 0709 12:26:09.401398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.919215 (+1517817us) service_pool.cc:224] Handling call
0709 12:26:10.920471 (+  1256us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.921932  6308 rpcz_store.cc:276] 0709 12:26:09.604225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.920277 (+1316052us) service_pool.cc:224] Handling call
0709 12:26:10.921472 (+  1195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.922721  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:10.922883  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:10.923588  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1970) took 1218 ms. Trace:
I20260709 12:26:10.923519  6385 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:10.924003  6308 rpcz_store.cc:276] 0709 12:26:09.705213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.922648 (+1217435us) service_pool.cc:224] Handling call
0709 12:26:10.923507 (+   859us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.924237  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1887) took 1517 ms. Trace:
I20260709 12:26:10.924751  6385 rpcz_store.cc:276] 0709 12:26:09.406504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.922318 (+1515814us) service_pool.cc:224] Handling call
0709 12:26:10.924187 (+  1869us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.924991  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:10.925822  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1966) took 1319 ms. Trace:
I20260709 12:26:10.925964  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:10.926230  6308 rpcz_store.cc:276] 0709 12:26:09.606040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.924742 (+1318702us) service_pool.cc:224] Handling call
0709 12:26:10.925766 (+  1024us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.927692  6385 raft_consensus.cc:2410] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:10.928580  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1885) took 1526 ms. Trace:
I20260709 12:26:10.929003  6385 rpcz_store.cc:276] 0709 12:26:09.402246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.925626 (+1523380us) service_pool.cc:224] Handling call
0709 12:26:10.928492 (+  2866us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.930012  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:10.925964  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2488) took 1303 ms. Trace:
I20260709 12:26:10.932273  6385 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:10.932322  6451 rpcz_store.cc:276] 0709 12:26:09.622015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.924674 (+1302659us) service_pool.cc:224] Handling call
0709 12:26:10.925121 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.925420 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.925448 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.925501 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:10.925536 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.925608 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.925743 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.932986  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1884) took 1531 ms. Trace:
I20260709 12:26:10.933836  6385 rpcz_store.cc:276] 0709 12:26:09.401838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.929683 (+1527845us) service_pool.cc:224] Handling call
0709 12:26:10.932927 (+  3244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.935163  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:10.936122  6385 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:10.936618  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 408) took 1554 ms. Trace:
I20260709 12:26:10.937005  6385 rpcz_store.cc:276] 0709 12:26:09.382827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.934776 (+1551949us) service_pool.cc:224] Handling call
0709 12:26:10.936566 (+  1790us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.937153  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:10.938022  6310 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:10.938586  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1969) took 1287 ms. Trace:
I20260709 12:26:10.939079  6310 rpcz_store.cc:276] 0709 12:26:09.655118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.737734 (+ 82616us) service_pool.cc:224] Handling call
0709 12:26:10.938528 (+1200794us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:10.940464  7318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.000s	sys 0.017s
W20260709 12:26:10.940912  7318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.002s	sys 0.018s
I20260709 12:26:10.941684  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 555) took 1462 ms. Trace:
I20260709 12:26:10.938550  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:10.942101  7318 rpcz_store.cc:276] 0709 12:26:09.479018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.597092 (+118074us) service_pool.cc:224] Handling call
0709 12:26:09.626688 (+ 29596us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.663944 (+ 37256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.663981 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.786193 (+122212us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.941274 (+1155081us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:10.941313 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.941380 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.941424 (+    44us) raft_consensus.cc:1685] finished
0709 12:26:10.941436 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.941574 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":78256,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":252,"thread_start_us":1050062,"threads_started":2,"wal-append.queue_time_us":981032}
I20260709 12:26:10.943946  6385 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 13 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:10.946236  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 400) took 1764 ms. Trace:
I20260709 12:26:10.946882  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1791) took 1450 ms. Trace:
I20260709 12:26:10.947459  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1790) took 1451 ms. Trace:
I20260709 12:26:10.947641  6310 rpcz_store.cc:276] 0709 12:26:09.496303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.939977 (+1443674us) service_pool.cc:224] Handling call
0709 12:26:10.940477 (+   500us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.940975 (+   498us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.941007 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.941740 (+   733us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.944226 (+  2486us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:10.945069 (+   843us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.945543 (+   474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.945625 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.946607 (+   982us) raft_consensus.cc:1685] finished
0709 12:26:10.946629 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.946766 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":184,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":2244,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":227,"spinlock_wait_cycles":1328384,"thread_start_us":1391,"threads_started":1,"wal-append.queue_time_us":2178}
I20260709 12:26:09.767910  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:10.950986  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1890) took 1543 ms (client timeout 1713 ms). Trace:
I20260709 12:26:10.947844  6308 rpcz_store.cc:276] 0709 12:26:09.496005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.927223 (+1431218us) service_pool.cc:224] Handling call
0709 12:26:10.927831 (+   608us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:10.928986 (+  1155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.929017 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:10.929687 (+   670us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.945452 (+ 15765us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:10.946001 (+   549us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.947133 (+  1132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.947205 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.947244 (+    39us) raft_consensus.cc:1685] finished
0709 12:26:10.947255 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.947370 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":139,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":225,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":203,"thread_start_us":15308,"threads_started":1,"wal-append.queue_time_us":6371}
W20260709 12:26:10.951399  6379 rpcz_store.cc:269] 0709 12:26:09.407673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.730288 (+322615us) service_pool.cc:224] Handling call
0709 12:26:10.950920 (+1220632us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.947304  6385 rpcz_store.cc:276] 0709 12:26:09.181897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.938040 (+1756143us) service_pool.cc:224] Handling call
0709 12:26:10.946156 (+  8116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.953251  6379 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:10.954072  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2340) took 3392 ms. Trace:
I20260709 12:26:10.954452  6379 rpcz_store.cc:276] 0709 12:26:07.561614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.952331 (+3390717us) service_pool.cc:224] Handling call
0709 12:26:10.952803 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.953709 (+   906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.953966 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.956501  6379 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:10.956760  9378 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:10.957172  9378 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:10.957265  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1867) took 2915 ms. Trace:
I20260709 12:26:10.957901  6379 rpcz_store.cc:276] 0709 12:26:08.041718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.955637 (+2913919us) service_pool.cc:224] Handling call
0709 12:26:10.956072 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.956942 (+   870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.957208 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.954880  6385 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:10.959997  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2341) took 2951 ms. Trace:
I20260709 12:26:10.960798  6385 rpcz_store.cc:276] 0709 12:26:08.008429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.953289 (+2944860us) service_pool.cc:224] Handling call
0709 12:26:10.954652 (+  1363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.959740 (+  5088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.959928 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.963877  6385 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:10.965817  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1872) took 2922 ms. Trace:
I20260709 12:26:10.966187  6385 rpcz_store.cc:276] 0709 12:26:08.043253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.962797 (+2919544us) service_pool.cc:224] Handling call
0709 12:26:10.963477 (+   680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.965195 (+  1718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.965737 (+   542us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.966562  9378 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:10.967499  9759 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:10.968014  6385 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:10.972287  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:10.973044  6456 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:26:10.973127  7331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 502) took 1813 ms (client timeout 1784 ms). Trace:
I20260709 12:26:10.969692  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1873) took 2926 ms. Trace:
W20260709 12:26:10.973896  7331 rpcz_store.cc:269] 0709 12:26:09.160006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.160144 (+   138us) service_pool.cc:224] Handling call
0709 12:26:10.971981 (+1811837us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:26:10.973035 (+  1054us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10056064}
I20260709 12:26:10.967767  9378 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:10.974320  6385 rpcz_store.cc:276] 0709 12:26:08.043524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.967158 (+2923634us) service_pool.cc:224] Handling call
0709 12:26:10.967661 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.969467 (+  1806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.969649 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.986735  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2497) took 1023 ms. Trace:
I20260709 12:26:10.987178  6456 rpcz_store.cc:276] 0709 12:26:09.963083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.655314 (+692231us) service_pool.cc:224] Handling call
0709 12:26:10.986644 (+331330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:10.995580  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1792) took 1499 ms. Trace:
I20260709 12:26:10.996157  6310 rpcz_store.cc:276] 0709 12:26:09.496597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.950241 (+1453644us) service_pool.cc:224] Handling call
0709 12:26:10.950616 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.962017 (+ 11401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.962047 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.964400 (+  2353us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.990779 (+ 26379us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:10.990831 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.990929 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.995277 (+  4348us) raft_consensus.cc:1685] finished
0709 12:26:10.995301 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.995472 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":968,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":143,"thread_start_us":25841,"threads_started":1,"wal-append.queue_time_us":26837}
I20260709 12:26:10.998921  6310 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:26:10.999778  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1964) took 1395 ms. Trace:
I20260709 12:26:11.000170  6310 rpcz_store.cc:276] 0709 12:26:09.604964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.998111 (+1393147us) service_pool.cc:224] Handling call
0709 12:26:10.998606 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.999427 (+   821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.999681 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.001848  6310 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:11.001919  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 388) took 1389 ms. Trace:
I20260709 12:26:11.002635  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1965) took 1397 ms. Trace:
I20260709 12:26:11.003080  6310 rpcz_store.cc:276] 0709 12:26:09.605610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.001075 (+1395465us) service_pool.cc:224] Handling call
0709 12:26:11.001555 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.002370 (+   815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.002585 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.005095  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:11.006140  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2034) took 1404 ms. Trace:
I20260709 12:26:11.006685  6459 rpcz_store.cc:276] 0709 12:26:09.601393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.664920 (+1063527us) service_pool.cc:224] Handling call
0709 12:26:11.006066 (+341146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.009209  6459 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:11.009814  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2492) took 1365 ms. Trace:
I20260709 12:26:11.010120  6459 rpcz_store.cc:276] 0709 12:26:09.644283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.008625 (+1364342us) service_pool.cc:224] Handling call
0709 12:26:11.009005 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.009566 (+   561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.009756 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.002748  6453 rpcz_store.cc:276] 0709 12:26:09.612624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.667359 (+1054735us) service_pool.cc:224] Handling call
0709 12:26:11.001195 (+333836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.001507 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.001540 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.001588 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.001641 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.001712 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.001844 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.015094  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1793) took 1518 ms. Trace:
W20260709 12:26:11.016185  9827 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.743s	user 0.013s	sys 0.000s
I20260709 12:26:11.015645  6308 rpcz_store.cc:276] 0709 12:26:09.496901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.955185 (+1458284us) service_pool.cc:224] Handling call
0709 12:26:10.955647 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.956272 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.956312 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.956954 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.014084 (+ 57130us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:11.014135 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.014811 (+   676us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.014874 (+    63us) raft_consensus.cc:1685] finished
0709 12:26:11.014890 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.015031 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"thread_start_us":56612,"threads_started":1,"wal-append.queue_time_us":37418}
I20260709 12:26:11.020740  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2495) took 1282 ms. Trace:
I20260709 12:26:11.021173  6458 rpcz_store.cc:276] 0709 12:26:09.745740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.664910 (+919170us) service_pool.cc:224] Handling call
0709 12:26:11.020662 (+355752us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.026778  9668 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.027776  9668 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.042541  6379 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:26:11.044552  9676 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.052021  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1967) took 1445 ms. Trace:
I20260709 12:26:11.052582  6310 rpcz_store.cc:276] 0709 12:26:09.606401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.003937 (+1397536us) service_pool.cc:224] Handling call
0709 12:26:11.004522 (+   585us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.008630 (+  4108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.008660 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.009134 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.050846 (+ 41712us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:11.050910 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.051021 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.051773 (+   752us) raft_consensus.cc:1685] finished
0709 12:26:11.051790 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.051951 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3322,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":151,"thread_start_us":41247,"threads_started":1,"wal-append.queue_time_us":40071}
I20260709 12:26:11.065132  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 1972) took 1359 ms. Trace:
I20260709 12:26:11.065594  6308 rpcz_store.cc:276] 0709 12:26:09.705968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.019788 (+1313820us) service_pool.cc:224] Handling call
0709 12:26:11.020318 (+   530us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:11.022501 (+  2183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.022544 (+    43us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:11.025751 (+  3207us) log.cc:844] Serialized 78 byte log entry
0709 12:26:11.060204 (+ 34453us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:11.060589 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.061129 (+   540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.061205 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.064855 (+  3650us) raft_consensus.cc:1685] finished
0709 12:26:11.064876 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.065069 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":198,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":1417,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":257,"thread_start_us":33940,"threads_started":1,"wal-append.queue_time_us":36897}
W20260709 12:26:10.034616  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1855) took 4161 ms (client timeout 1713 ms). Trace:
W20260709 12:26:11.073191  6377 rpcz_store.cc:269] 0709 12:26:05.872946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.873440 (+   494us) service_pool.cc:224] Handling call
0709 12:26:10.034035 (+4160595us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b5400038754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:10.034298 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3693982336}
W20260709 12:26:11.100435  7320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 551) took 1668 ms (client timeout 1693 ms). Trace:
I20260709 12:26:09.797613  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.104703  6380 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
W20260709 12:26:11.105316  6380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1889) took 1698 ms (client timeout 1776 ms). Trace:
W20260709 12:26:11.105788  6380 rpcz_store.cc:269] 0709 12:26:09.407270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.796979 (+389709us) service_pool.cc:224] Handling call
0709 12:26:11.105246 (+1308267us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:11.108246  7320 rpcz_store.cc:269] 0709 12:26:09.432012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.473685 (+ 41673us) service_pool.cc:224] Handling call
0709 12:26:11.100125 (+1626440us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:11.100355 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2348332288}
I20260709 12:26:11.111436  9779 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:11.112026  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 399) took 3029 ms. Trace:
I20260709 12:26:11.112421  9779 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.112486  6385 rpcz_store.cc:276] 0709 12:26:08.082900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.976723 (+2893823us) service_pool.cc:224] Handling call
0709 12:26:10.977122 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.977443 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:10.977794 (+   351us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.111593 (+133799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.111651 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.111707 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.111770 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.111941 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63790,"prepare.queue_time_us":551,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":68,"raft.queue_time_us":65090,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":69433,"threads_started":1,"wal-append.queue_time_us":75592}
I20260709 12:26:11.117535  9782 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.118798  9598 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.119403  9598 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.120035  9779 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.120589  9779 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:11.121085  9779 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.121618  9598 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.122133  9598 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:11.123134  9779 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.123649  9779 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.124106  9779 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.124774  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2338) took 3563 ms. Trace:
I20260709 12:26:11.126099  9779 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.127246  9788 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.127804  9788 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.130033  9788 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.118132  9782 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.135659  9662 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:11.141858  9662 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:10.025923  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.143262  6382 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:11.142977  9662 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.138576  9653 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:11.143867  6382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 405) took 1818 ms (client timeout 2063 ms). Trace:
I20260709 12:26:11.139150  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 401) took 1952 ms. Trace:
I20260709 12:26:11.144222  9662 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:11.144806  9378 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.144170  9653 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.145399  9378 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.145838  9378 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.147222  9653 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:11.140159  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.4611e+06us
W20260709 12:26:11.148624  7287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.644s (SENT) [suppressed 10 similar messages]
I20260709 12:26:11.148779  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.150061  9884 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.150605  9884 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.150995  9884 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:11.137912  6351 connection.cc:256] RPC call timeout handler was delayed by -2.138991919000091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260709 12:26:11.140120  9598 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.141414  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 402) took 1818 ms. Trace:
I20260709 12:26:11.154847  6380 rpcz_store.cc:276] 0709 12:26:09.323565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.108179 (+1784614us) service_pool.cc:224] Handling call
0709 12:26:11.111050 (+  2871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.141075 (+ 30025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.141099 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.141149 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.141177 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.141220 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.141355 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.125160  6376 rpcz_store.cc:276] 0709 12:26:07.561194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.007918 (+446724us) service_pool.cc:224] Handling call
0709 12:26:08.008238 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.011177 (+  2939us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:08.055052 (+ 43875us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.133600 (+ 78548us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.134091 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.134259 (+   168us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:08.134290 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.134352 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.090219 (+1955867us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:11.124293 (+1034074us) raft_consensus.cc:1685] finished
0709 12:26:11.124411 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.124578 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45126,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":2308,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":90136,"spinlock_wait_cycles":43586944,"thread_start_us":78126,"threads_started":1,"wal-append.queue_time_us":221671}
W20260709 12:26:11.144351  6382 rpcz_store.cc:269] 0709 12:26:09.325268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.024659 (+699391us) service_pool.cc:224] Handling call
0709 12:26:11.143786 (+1119127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.158038  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.144682  6377 rpcz_store.cc:276] 0709 12:26:09.186331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.077542 (+1891211us) service_pool.cc:224] Handling call
0709 12:26:11.077981 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.078365 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:11.079032 (+   667us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.113011 (+ 33979us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.113069 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:11.113127 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.113201 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.139071 (+ 25870us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22606,"prepare.queue_time_us":203,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":54,"raft.queue_time_us":22777,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":10790,"threads_started":1,"wal-append.queue_time_us":10236}
I20260709 12:26:11.162052  6382 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:11.162691  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1877) took 1778 ms. Trace:
I20260709 12:26:11.163014  6382 rpcz_store.cc:276] 0709 12:26:09.384013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.156808 (+1772795us) service_pool.cc:224] Handling call
0709 12:26:11.161598 (+  4790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.162443 (+   845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.162638 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.143451  9782 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.170799  9783 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:11.171293  9783 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.171675  9783 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:11.172041  9884 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.179618  6380 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:11.181599  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1876) took 1796 ms. Trace:
I20260709 12:26:11.182135  6380 rpcz_store.cc:276] 0709 12:26:09.383521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.156189 (+1772668us) service_pool.cc:224] Handling call
0709 12:26:11.175120 (+ 18931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.180043 (+  4923us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.180246 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.184437  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1888) took 1777 ms. Trace:
I20260709 12:26:11.189684  9598 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.196118  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.196491  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.196916  6308 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:11.182276  9884 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.226536  6308 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:11.257552  9817 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 2.978s	user 0.010s	sys 0.005s
I20260709 12:26:11.264225  9884 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:11.264950  9884 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:11.271433  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118961us
I20260709 12:26:11.270228  9378 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.291414  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 404) took 1966 ms. Trace:
W20260709 12:26:11.293200  6351 connection.cc:256] RPC call timeout handler was delayed by -1.8815541520002625s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 12:26:11.291971  6385 rpcz_store.cc:276] 0709 12:26:09.324724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.115043 (+1790319us) service_pool.cc:224] Handling call
0709 12:26:11.115404 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.115892 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:11.185283 (+ 69391us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.290981 (+105698us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.291052 (+    71us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:11.291123 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.291210 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.291354 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2195,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":2279,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"thread_start_us":171890,"threads_started":2}
W20260709 12:26:11.293661  6351 connection.cc:256] RPC call timeout handler was delayed by -1.697091919000286s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2681s for a response
W20260709 12:26:11.295614  6351 connection.cc:256] RPC call timeout handler was delayed by -1.6727541520003797s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 12:26:11.296437  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4756906020002134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 12:26:11.292451  9786 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 3.558s	user 0.006s	sys 0.018s
W20260709 12:26:11.296736  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4549541520004823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23570000000000002s for a response
I20260709 12:26:11.297298  6310 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:11.298642  6351 connection.cc:256] RPC call timeout handler was delayed by -1.26863015500021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 12:26:11.301035  6351 connection.cc:256] RPC call timeout handler was delayed by -1.1066919190002409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.33370000000000005s for a response
W20260709 12:26:11.301795  6351 connection.cc:256] RPC call timeout handler was delayed by -1.0580541520002953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2798s for a response
W20260709 12:26:11.302505  6351 connection.cc:256] RPC call timeout handler was delayed by -1.0498073360004128s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:26:11.303211  6351 connection.cc:256] RPC call timeout handler was delayed by -1.0490301550003096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1784s for a response
I20260709 12:26:11.304456  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:11.289674  9783 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.306854  9598 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:11.308748  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:10.047737  6384 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:26:11.310989  9598 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:11.311411  9598 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:11.311784  9598 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:11.312772  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2329) took 3754 ms. Trace:
I20260709 12:26:11.313315  6384 rpcz_store.cc:276] 0709 12:26:07.557949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.558133 (+   184us) service_pool.cc:224] Handling call
0709 12:26:10.046797 (+2488664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.311065 (+1264268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.312513 (+  1448us) spinlock_profiling.cc:243] Waited 2.48 s on lock 0x7b5400038750. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:11.312699 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5709472000}
W20260709 12:26:11.318877  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.322443  9598 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.324815  9598 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.298815  9782 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:11.332063  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.335177  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:11.336228  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.66701e+06us
I20260709 12:26:11.336678  9779 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.336596  9782 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.336999  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.337255  9779 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.337725  9779 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.337874  9662 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:11.338029  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.338356  9662 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:11.338832  9662 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:11.339097  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.342909  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1878) took 1958 ms. Trace:
I20260709 12:26:11.343392  6377 rpcz_store.cc:276] 0709 12:26:09.384504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.160003 (+1775499us) service_pool.cc:224] Handling call
0709 12:26:11.160579 (+   576us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.170897 (+ 10318us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.170929 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.171504 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.190725 (+ 19221us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:11.190806 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.190990 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.342664 (+151674us) raft_consensus.cc:1685] finished
0709 12:26:11.342686 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.342828 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1410,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":118,"thread_start_us":18745,"threads_started":1,"wal-append.queue_time_us":104664}
W20260709 12:26:09.793345  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.346771  9908 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:11.347620  9784 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:11.348047  9784 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.348474  9784 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.348151  9908 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:11.350030  9820 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 3.075s	user 0.002s	sys 0.017s
I20260709 12:26:11.350791  9909 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.351070  9378 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.351398  9909 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:10.039464  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.353422  6383 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:11.353749  9909 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:11.353826  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1879) took 1969 ms (client timeout 2004 ms). Trace:
I20260709 12:26:11.351786  9378 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:11.355422  6383 rpcz_store.cc:269] 0709 12:26:09.384989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.038801 (+653812us) service_pool.cc:224] Handling call
0709 12:26:11.353756 (+1314955us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.375183  9908 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.374833  9378 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.377767  9378 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.338902  9782 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.377300  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.383884  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:11.346787  9788 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:11.362758  9598 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:26:11.374833  9906 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:11.399165  9788 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:11.406586  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53927.2us
I20260709 12:26:11.407557  9788 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.408340  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:11.409525  6307 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:10.085759  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:11.410852  9378 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:11.411664  9909 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:11.411929  9378 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:11.399348  9759 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:11.413478  9662 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.414508  9759 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:11.415081  9759 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:11.415585  9759 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.400158  9906 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.424387  9906 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.413513  9378 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:11.425546  9378 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:11.426815  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.429073  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:11.442530  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:11.443423  6377 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:11.457496  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:11.502956  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 147431us
I20260709 12:26:11.504091  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.505312  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:11.405943  9784 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.508924  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.511158  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.511785  6385 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:26:11.511801  6384 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:11.519869  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.499791  9906 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.509673  9909 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.535741  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.539702  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.542042  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.551009  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:11.552904  6384 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:11.555394  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.540653  9906 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.559813  6380 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:11.561270  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.551785  9779 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.562784  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.567278  9662 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:11.570786  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:11.573676  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:11.573520  9759 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.574894  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:11.575479  9913 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:11.525112  9378 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.582476  6384 raft_consensus.cc:2368] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:11.581215  6302 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:11.560127  9912 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:11.586208  6307 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:26:11.602149  6307 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 3
I20260709 12:26:11.601994  9912 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:11.591069  9779 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.602162  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:11.589437  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.603477  9912 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.604744  6385 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:11.605739  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.606083  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:11.606469  6385 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:11.606966  7329 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:11.607435  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:11.603928  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.608287  6302 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:11.609256  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:11.610018  6302 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:11.598201  6380 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:11.611585  6377 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:11.614061  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:11.614768  9784 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.614873  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:11.553534  9911 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:11.615654  9911 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.616047  9911 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:11.619374  9378 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:11.623831  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:11.624337  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:11.624558  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:11.625142  6308 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:11.593420  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:11.627138  6384 raft_consensus.cc:2376] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8. Re-sending same reply.
I20260709 12:26:11.604328  6307 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:11.630602  6307 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.547341  9909 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:11.636114  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:11.626531  9915 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:11.639991  9915 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:11.641433  9915 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:11.635576  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:11.647542  7331 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:11.649348  9906 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:11.650343  9848 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:11.650418  9909 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:11.654009  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 958993us
I20260709 12:26:11.649139  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.656657  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.585502  9913 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:11.666814  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.667660  6385 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:11.655812  9848 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:12.249650  6310 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.000s
I20260709 12:26:11.688369  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.543640  6310 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:26:11.836272  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.682648  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:11.701643  7331 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:11.759325  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1780) took 2336 ms. Trace:
I20260709 12:26:11.688788  9759 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:12.547472  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:11.865777  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.389279  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800262220 after lost signal to thread 5140
I20260709 12:26:11.694888  9909 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.855s
I20260709 12:26:12.550714  6274 leader_election.cc:400] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
W20260709 12:26:12.039788  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.551254  6274 leader_election.cc:403] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:11.676074  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.552491  6384 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:11.676520  9913 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:12.555127  9913 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:12.556159  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.557097  6380 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:12.558256  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:11.733990  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 496) took 3697 ms. Trace:
W20260709 12:26:11.695257  7316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 549) took 2739 ms (client timeout 2004 ms). Trace:
I20260709 12:26:11.669988  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:12.548832  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.562356  7316 rpcz_store.cc:269] 0709 12:26:08.955602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.955801 (+   199us) service_pool.cc:224] Handling call
0709 12:26:11.694855 (+2739054us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54003bc4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:11.695024 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355610880}
I20260709 12:26:12.561177  7319 rpcz_store.cc:276] 0709 12:26:08.036741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.037011 (+   270us) service_pool.cc:224] Handling call
0709 12:26:08.037474 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.393858 (+356384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.393907 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:08.393970 (+    63us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:08.394027 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.733440 (+3339413us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400566b54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:11.733521 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.733779 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818887040}
I20260709 12:26:12.547456  6307 rpcz_store.cc:276] 0709 12:26:09.398666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.399215 (+   549us) service_pool.cc:224] Handling call
0709 12:26:09.399756 (+   541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.409229 (+2009473us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b58007f1fd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa329dfe7 00007f6fa329f7de 00007f6fa32a9b49 00007f6fa32aa416 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4
0709 12:26:11.734680 (+325451us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.734959 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4004993792}
I20260709 12:26:12.572420  6302 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:12.575896  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:11.691906  9884 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:11.757685  9357 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:26:12.582656  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.589473  6381 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:12.590245  6380 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:12.590660  9909 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.591282  9909 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.591754  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.593232  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.593871  9783 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:12.594066  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2361) took 1001 ms. Trace:
I20260709 12:26:11.670148  9848 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.594776  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.598523  6380 rpcz_store.cc:276] 0709 12:26:11.593103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.592948 (+999845us) service_pool.cc:224] Handling call
0709 12:26:12.594005 (+  1057us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.599852  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:12.600698  6380 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:12.602919  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.603808  6307 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:12.599057  9848 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:12.602236  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 416) took 1032 ms. Trace:
I20260709 12:26:12.608026  6380 rpcz_store.cc:276] 0709 12:26:11.569842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.599639 (+1029797us) service_pool.cc:224] Handling call
0709 12:26:12.602183 (+  2544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.595264  9884 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.606782  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.609176  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.607348  9909 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.611892  6380 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:26:12.612586  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2360) took 1019 ms. Trace:
I20260709 12:26:12.612869  6380 rpcz_store.cc:276] 0709 12:26:11.592821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.608894 (+1016073us) service_pool.cc:224] Handling call
0709 12:26:12.612538 (+  3644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.608759  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.615327  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2355) took 1024 ms. Trace:
I20260709 12:26:12.615618  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:12.617341  6307 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:26:12.616169  9909 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.624732  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.615725  6381 rpcz_store.cc:276] 0709 12:26:11.591377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.600904 (+1009527us) service_pool.cc:224] Handling call
0709 12:26:12.615255 (+ 14351us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.630126  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 415) took 1060 ms. Trace:
I20260709 12:26:12.630707  6380 rpcz_store.cc:276] 0709 12:26:11.569559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.613583 (+1044024us) service_pool.cc:224] Handling call
0709 12:26:12.630043 (+ 16460us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.631465  6348 connection.cc:256] RPC call timeout handler was delayed by -1.2681897049997133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2238s for a response
I20260709 12:26:12.623020  9404 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.638939  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.640043  6380 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:12.642984  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 409) took 1090 ms. Trace:
I20260709 12:26:12.643517  6380 rpcz_store.cc:276] 0709 12:26:11.552293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.638559 (+1086266us) service_pool.cc:224] Handling call
0709 12:26:12.640548 (+  1989us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.637773  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.619787  9783 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:12.652465  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.653107  6380 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:12.639166  9915 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.667479  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:12.648511  9911 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.032s	user 0.008s	sys 0.005s
I20260709 12:26:12.621212  9884 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.651450  9404 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.683868  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.684556  6380 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:12.683064  9912 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.075s	user 0.028s	sys 0.038s
I20260709 12:26:12.649152  9759 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:12.702247  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 156990us
I20260709 12:26:12.701155  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.703732  6380 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:12.704707  6302 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
W20260709 12:26:12.701885  9912 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.094s	user 0.029s	sys 0.039s
I20260709 12:26:12.706396  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1795) took 1218 ms. Trace:
W20260709 12:26:12.704339  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.714440  6307 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:12.716248  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.716579  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1978) took 1230 ms. Trace:
I20260709 12:26:12.717419  6307 rpcz_store.cc:276] 0709 12:26:11.486550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.607727 (+1121177us) service_pool.cc:224] Handling call
0709 12:26:12.716493 (+108766us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.716629  6275 leader_election.cc:336] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.621s (SENT)
I20260709 12:26:12.715379  6302 rpcz_store.cc:276] 0709 12:26:11.488328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.575570 (+1087242us) service_pool.cc:224] Handling call
0709 12:26:12.706330 (+130760us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.719288  6275 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.622s (SENT)
I20260709 12:26:12.719944  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.720747  6302 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:12.721360  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1984) took 1112 ms. Trace:
I20260709 12:26:12.721730  6302 rpcz_store.cc:276] 0709 12:26:11.608605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.719690 (+1111085us) service_pool.cc:224] Handling call
0709 12:26:12.721293 (+  1603us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.719671  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:12.664513  9783 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.724218  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.728009  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:12.728499  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64433.6us
I20260709 12:26:12.683343  9911 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:12.733507  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 181201us
W20260709 12:26:12.735786  7287 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.015s	sys 0.035s
I20260709 12:26:12.735541  6307 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
W20260709 12:26:12.739444  7287 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.017s	sys 0.035s
I20260709 12:26:12.702273  9915 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:12.739274  6275 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.628s (SENT)
W20260709 12:26:12.739977  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 639799us
I20260709 12:26:12.738320  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:12.741739  6380 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:12.715378  9759 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.739918  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1993) took 1049 ms. Trace:
I20260709 12:26:12.744378  6307 rpcz_store.cc:276] 0709 12:26:11.690905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.723844 (+1032939us) service_pool.cc:224] Handling call
0709 12:26:12.739848 (+ 16004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.747084  6380 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 6. Ops: []
I20260709 12:26:12.747773  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2352) took 1196 ms. Trace:
I20260709 12:26:12.747668  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.748176  6380 rpcz_store.cc:276] 0709 12:26:11.551347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.746622 (+1195275us) service_pool.cc:224] Handling call
0709 12:26:12.746920 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.747540 (+   620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.747722 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.748555  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1990) took 1134 ms. Trace:
I20260709 12:26:12.749048  6307 rpcz_store.cc:276] 0709 12:26:11.614451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.745370 (+1130919us) service_pool.cc:224] Handling call
0709 12:26:12.748497 (+  3127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.749922  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 410) took 1182 ms. Trace:
I20260709 12:26:12.750075  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.750288  6380 rpcz_store.cc:276] 0709 12:26:11.567939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.749117 (+1181178us) service_pool.cc:224] Handling call
0709 12:26:12.749365 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.749583 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.749607 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.749671 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:12.749707 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.749760 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.749850 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.752723  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 412) took 1184 ms. Trace:
I20260709 12:26:12.753057  6380 rpcz_store.cc:276] 0709 12:26:11.568698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.752021 (+1183323us) service_pool.cc:224] Handling call
0709 12:26:12.752263 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.752459 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.752481 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.752518 (+    37us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:12.752551 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.752597 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.752679 (+    82us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.755980  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 1.07874e+06us
I20260709 12:26:12.757606  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1980) took 1270 ms. Trace:
I20260709 12:26:12.759087  6307 rpcz_store.cc:276] 0709 12:26:11.487019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.749723 (+1262704us) service_pool.cc:224] Handling call
0709 12:26:12.757549 (+  7826us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.764545  6381 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:12.795478  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 43625.4us
I20260709 12:26:12.797822  9929 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:12.798635  9929 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.799120  9929 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:12.802380  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800246fe0 after lost signal to thread 6240
I20260709 12:26:12.803535  6380 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.801970  9404 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.806782  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.807863  6307 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:12.808509  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1799) took 1319 ms. Trace:
I20260709 12:26:12.808956  6307 rpcz_store.cc:276] 0709 12:26:11.489574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.797181 (+1307607us) service_pool.cc:224] Handling call
0709 12:26:12.808444 (+ 11263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.809639  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 436) took 5040 ms. Trace:
I20260709 12:26:12.810115  6309 rpcz_store.cc:276] 0709 12:26:07.768770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.769047 (+   277us) service_pool.cc:224] Handling call
0709 12:26:07.795841 (+ 26794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.796167 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.796205 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.796823 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.012031 (+215208us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:08.012081 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.012173 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.254122 (+3241949us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:12.793552 (+1539430us) raft_consensus.cc:1685] finished
0709 12:26:12.793601 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.809421 (+ 15820us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79559552,"thread_start_us":214753,"threads_started":1,"wal-append.queue_time_us":473387,"wal-append.run_cpu_time_us":17970,"wal-append.run_wall_time_us":4065934}
W20260709 12:26:12.826375  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84719.5us
I20260709 12:26:12.827780  9926 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.828265  9933 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:12.819010  9404 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:12.828950  9933 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.807307  9928 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.828431  9926 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:12.831867  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73831.3us
I20260709 12:26:12.816203  9884 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:12.825913  9921 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
W20260709 12:26:12.834813  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.837390  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.834008  9920 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:26:12.832978  9851 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:26:12.844480  9909 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:12.851356  9851 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:12.851987  9851 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:12.853216  9851 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:12.855185  9851 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:12.857242  9573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.192s	user 0.015s	sys 0.030s
I20260709 12:26:12.837172  9928 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.868496  9911 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:12.870687  9759 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.871076  6302 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:11.186580  6376 rpcz_store.cc:276] 0709 12:26:09.406899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.166660 (+1759761us) service_pool.cc:224] Handling call
0709 12:26:11.167130 (+   470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.167561 (+   431us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:11.167952 (+   391us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.183925 (+ 15973us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.184005 (+    80us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:11.184076 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.184182 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.184356 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":193,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":2473,"thread_start_us":13022,"threads_started":1}
I20260709 12:26:12.860661  9783 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.871600  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.866240  9915 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.873673  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.875026  9928 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.876194  7331 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:12.877537  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.878312  7329 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:12.879241  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.877175  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:12.879923  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.880539  7318 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:12.882673  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.883291  7317 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:12.884151  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.885268  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.882259  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.886840  9848 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:12.887074  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.888115  7318 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:12.886294  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:12.889210  6275 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.722s (SENT)
I20260709 12:26:12.889858  9926 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.888077  7329 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:12.889168  7317 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:12.891414  9759 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:12.897943  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:12.898831  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60126.1us
I20260709 12:26:12.899156  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 508) took 1294 ms. Trace:
I20260709 12:26:12.899942  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.899941  7314 rpcz_store.cc:276] 0709 12:26:11.604667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.897508 (+1292841us) service_pool.cc:224] Handling call
0709 12:26:12.899099 (+  1591us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.893364  9573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.283s	user 0.016s	sys 0.032s
I20260709 12:26:12.901968  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.893236  9915 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:26:12.903261  9915 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:26:12.903738  9915 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:12.904901  6376 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:12.904804  9915 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.905694  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 414) took 1336 ms. Trace:
I20260709 12:26:12.906572  9915 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:12.908198  9933 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.906085  6376 rpcz_store.cc:276] 0709 12:26:11.569270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.904301 (+1335031us) service_pool.cc:224] Handling call
0709 12:26:12.904703 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.905451 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.905639 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.910787  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.911515  6307 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:12.911948  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1989) took 1297 ms. Trace:
I20260709 12:26:12.912230  6307 rpcz_store.cc:276] 0709 12:26:11.614176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.910507 (+1296331us) service_pool.cc:224] Handling call
0709 12:26:12.911895 (+  1388us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.913089  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.913625  6307 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:12.914050  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1995) took 1222 ms. Trace:
I20260709 12:26:12.914255  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:12.903470  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.920591  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:12.921581  9912 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:12.914404  6307 rpcz_store.cc:276] 0709 12:26:11.691477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.912859 (+1221382us) service_pool.cc:224] Handling call
0709 12:26:12.914004 (+  1145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.923287  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.924345  6307 raft_consensus.cc:2410] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:12.924801  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1987) took 1314 ms. Trace:
I20260709 12:26:12.925163  6307 rpcz_store.cc:276] 0709 12:26:11.609894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.923052 (+1313158us) service_pool.cc:224] Handling call
0709 12:26:12.924749 (+  1697us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.926029  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:12.926967  6307 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:12.928754  6455 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 2.264s	user 0.021s	sys 0.022s
I20260709 12:26:12.931358  6455 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:12.922636  9933 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.922405  9912 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:12.933139  9912 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:12.933049  9933 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:12.936790  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.938007  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.938412  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:12.939376  6301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1788) took 3444 ms (client timeout 2420 ms). Trace:
W20260709 12:26:12.939638  6455 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2036) took 3304 ms (client timeout 2851 ms). Trace:
I20260709 12:26:12.939587  9848 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:12.940272  6455 rpcz_store.cc:269] 0709 12:26:09.629767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.748384 (+118617us) service_pool.cc:224] Handling call
0709 12:26:12.931876 (+3183492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.941286  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1880) took 3555 ms. Trace:
I20260709 12:26:12.943121  6380 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:12.943856  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 413) took 1374 ms. Trace:
I20260709 12:26:12.944242  6380 rpcz_store.cc:276] 0709 12:26:11.568987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.756418 (+1187431us) service_pool.cc:224] Handling call
0709 12:26:12.756667 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.943609 (+186942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.943800 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.945176  9929 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:12.945622  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.946866  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.947624  9929 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:12.947983  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.948282  9933 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:12.948915  7317 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:12.949082  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.949906  7335 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:12.944022  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1974) took 1515 ms (client timeout 1522 ms). Trace:
W20260709 12:26:12.951880  6308 rpcz_store.cc:269] 0709 12:26:11.428926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.485121 (+ 56195us) service_pool.cc:224] Handling call
0709 12:26:12.943837 (+1458716us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540010fc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:12.943954 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248309504}
I20260709 12:26:12.955116  7319 raft_consensus.cc:2368] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
W20260709 12:26:12.939766  6301 rpcz_store.cc:269] 0709 12:26:09.495366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.791174 (+295808us) service_pool.cc:224] Handling call
0709 12:26:12.939219 (+3148045us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.956760  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.957527  6301 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:12.962962  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1976) took 1476 ms. Trace:
I20260709 12:26:12.963565  6301 rpcz_store.cc:276] 0709 12:26:11.486019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.956452 (+1470433us) service_pool.cc:224] Handling call
0709 12:26:12.962874 (+  6422us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.957399  9930 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:26:12.961530  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:12.965155  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:12.967015  6308 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:12.967487  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1986) took 1358 ms. Trace:
W20260709 12:26:12.967459  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46953.9us
I20260709 12:26:12.968041  6308 rpcz_store.cc:276] 0709 12:26:11.609587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.952617 (+1343030us) service_pool.cc:224] Handling call
0709 12:26:12.967444 (+ 14827us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.966341  9936 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:26:12.968664  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:12.941684  6382 rpcz_store.cc:276] 0709 12:26:09.385492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.163710 (+1778218us) service_pool.cc:224] Handling call
0709 12:26:11.164005 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.179573 (+ 15568us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:11.188105 (+  8532us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.295820 (+107715us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.295879 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:11.327406 (+ 31527us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.941029 (+1613623us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b5400038254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:12.941083 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.941221 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":309,"prepare.queue_time_us":8381,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":1398,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":3028,"spinlock_wait_cycles":4867840,"thread_start_us":44591,"threads_started":1,"wal-append.queue_time_us":96217,"wal-append.run_cpu_time_us":1523,"wal-append.run_wall_time_us":1021230}
I20260709 12:26:12.975204  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:12.975739  6301 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:12.977167  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.980130  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:12.980652  9934 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:12.982065  6382 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.987434  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:12.988113  7331 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:12.988914  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:12.990064  7317 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:12.996220  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.997231  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.004416  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:12.993206  9913 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.006389  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.007047  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.007450  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2364) took 1412 ms. Trace:
I20260709 12:26:13.012251  6376 rpcz_store.cc:276] 0709 12:26:11.593931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.918794 (+1324863us) service_pool.cc:224] Handling call
0709 12:26:12.919109 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.919297 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:12.960566 (+ 41269us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.003869 (+ 43303us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.003934 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:13.004001 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.006380 (+  2379us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.006630 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41208,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":42,"thread_start_us":83520,"threads_started":3,"wal-append.queue_time_us":40859}
I20260709 12:26:13.014396  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.015142  6455 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 4. Current term is 6. Ops: []
I20260709 12:26:13.016469  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.017582  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.018195  6455 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:13.019048  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.019197  6458 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:13.019568  6455 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.020412  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.021703  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.015308  6308 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:13.023003  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.023298  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.023844  6453 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:13.023976  6308 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.024752  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.025488  6453 raft_consensus.cc:2368] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:26:13.026775  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.027383  7335 raft_consensus.cc:2376] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8. Re-sending same reply.
I20260709 12:26:13.027549  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2476) took 5100 ms. Trace:
I20260709 12:26:13.028349  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:13.029438  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.030051  7335 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:13.030593  9933 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:13.030979  6308 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:13.026275  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.022346  6458 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:13.032385  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:13.027992  6454 rpcz_store.cc:276] 0709 12:26:07.927134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.928359 (+  1225us) service_pool.cc:224] Handling call
0709 12:26:07.928822 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:07.929339 (+   517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:07.929380 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:07.953620 (+ 24240us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.055702 (+102082us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:08.055819 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.055960 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.015274 (+2959314us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:13.027078 (+2011804us) raft_consensus.cc:1685] finished
0709 12:26:13.027098 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.027345 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1707,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":23294,"spinlock_wait_cycles":43166976,"thread_start_us":55917,"threads_started":1,"wal-append.queue_time_us":292213,"wal-append.run_cpu_time_us":15226,"wal-append.run_wall_time_us":3766374}
I20260709 12:26:12.993352  9911 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:12.989971  9934 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:13.035216  6455 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:13.036391  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.053831  6308 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:13.036087  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.037468  6382 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:13.038659  6307 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:13.055346  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1994) took 1364 ms. Trace:
I20260709 12:26:13.055744  6307 rpcz_store.cc:276] 0709 12:26:11.691211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.925804 (+1234593us) service_pool.cc:224] Handling call
0709 12:26:13.055276 (+129472us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.031895  9913 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.056680  9913 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.047839  9911 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:13.056968  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.057266  9911 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.040881  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.058593  6453 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:13.058867  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:13.059459  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.060266  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:13.060647  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.061260  6453 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.060274  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.041430  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.055800  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.046627  7318 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:13.046977  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.047860  6301 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:13.062705  9928 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:13.062742  9926 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:13.071061  9926 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:13.071273  6453 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:13.040493  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.072088  9926 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.072574  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.073165  6454 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:13.074015  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.062988  9783 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:13.074803  9934 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.064033  6455 raft_consensus.cc:2376] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10. Re-sending same reply.
I20260709 12:26:13.075481  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:13.062467  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.054128  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.076454  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:13.076972  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.077140  6455 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
W20260709 12:26:13.077548  6272 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.551s (SENT)
I20260709 12:26:13.064082  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.067806  6308 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:13.068564  7335 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:13.068836  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1983) took 1561 ms. Trace:
I20260709 12:26:13.063385  6458 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.068852  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.080847  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:13.063000  7317 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:13.082644  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.080075  6301 rpcz_store.cc:276] 0709 12:26:11.507293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.964412 (+1457119us) service_pool.cc:224] Handling call
0709 12:26:13.068776 (+104364us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.069707  6382 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:26:13.086790  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.087489  9912 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.088073  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.088249  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.088922  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1981) took 1601 ms. Trace:
I20260709 12:26:13.068264  9848 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:13.089577  6301 rpcz_store.cc:276] 0709 12:26:11.487252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.087812 (+1600560us) service_pool.cc:224] Handling call
0709 12:26:13.088858 (+  1046us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.090222  9937 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:13.090425  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.090898  9937 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:13.091075  6455 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:13.091957  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.091845  9937 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.092249  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.092984  6384 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:13.093633  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2358) took 1501 ms. Trace:
I20260709 12:26:13.093037  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:13.094238  6274 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.551s (SENT)
I20260709 12:26:13.096374  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.097826  9928 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:13.099145  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:13.099730  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 442) took 5183 ms (client timeout 2184 ms). Trace:
I20260709 12:26:13.099857  6307 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:13.100189  6300 rpcz_store.cc:269] 0709 12:26:07.916002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.921179 (+  5177us) service_pool.cc:224] Handling call
0709 12:26:13.099365 (+5178186us) spinlock_profiling.cc:243] Waited 358 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:13.099506 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":823697280}
I20260709 12:26:13.101493  6307 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:13.102046  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1796) took 1613 ms. Trace:
I20260709 12:26:13.076237  9934 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.102981  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.103674  6454 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:13.088956  6308 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.104781  6455 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:13.105021  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2366) took 1510 ms. Trace:
I20260709 12:26:13.102371  6307 rpcz_store.cc:276] 0709 12:26:11.488788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.100985 (+1612197us) service_pool.cc:224] Handling call
0709 12:26:13.101224 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.101804 (+   580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.102000 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.105623  6382 rpcz_store.cc:276] 0709 12:26:11.605809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.056041 (+1450232us) service_pool.cc:224] Handling call
0709 12:26:13.069468 (+ 13427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.104706 (+ 35238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.104966 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.093991  6384 rpcz_store.cc:276] 0709 12:26:11.592304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.091903 (+1499599us) service_pool.cc:224] Handling call
0709 12:26:13.093566 (+  1663us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.125069  6458 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:13.133637  6307 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:13.135726  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.149762  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.151223  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.151701  6301 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:13.152012  6454 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:26:13.152238  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1985) took 1543 ms. Trace:
I20260709 12:26:13.151459  6376 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:13.152689  6458 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:13.153163  6308 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:13.152668  6301 rpcz_store.cc:276] 0709 12:26:11.609015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.090463 (+1481448us) service_pool.cc:224] Handling call
0709 12:26:13.152164 (+ 61701us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.152964  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.154759  6454 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:13.158939  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.159060  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.089707  9912 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:13.139060  9943 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:26:13.069373  9913 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:13.167618  7332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 547) took 6134 ms (client timeout 2292 ms). Trace:
I20260709 12:26:13.169428  7316 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:26:13.171144  6458 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 12:26:13.172600  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1961) took 3569 ms (client timeout 2115 ms). Trace:
W20260709 12:26:13.175348  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.173414  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:13.174131  7332 rpcz_store.cc:269] 0709 12:26:07.033567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.033772 (+   205us) service_pool.cc:224] Handling call
0709 12:26:13.167298 (+6133526us) spinlock_profiling.cc:243] Waited 3.66 s on lock 0x7b54007a0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:13.167506 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8422203776}
W20260709 12:26:13.175186  6303 rpcz_store.cc:269] 0709 12:26:09.603568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.735347 (+131779us) service_pool.cc:224] Handling call
0709 12:26:13.172517 (+3437170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.176911  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.098420  9911 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:13.180141  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 191320us
W20260709 12:26:13.182475  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.182991  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.183339  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:13.185353  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75105.1us
I20260709 12:26:13.190025  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.191476  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.192751  9759 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:13.179921  9930 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.193135  9759 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:13.193306  6310 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1982) took 1691 ms (client timeout 1602 ms). Trace:
I20260709 12:26:13.193596  9759 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:13.193699  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.194648  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:13.195261  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 421) took 1624 ms (client timeout 2063 ms). Trace:
W20260709 12:26:13.195613  6377 rpcz_store.cc:269] 0709 12:26:11.571255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.193412 (+1622157us) service_pool.cc:224] Handling call
0709 12:26:13.195187 (+  1775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.195962  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:13.193740  6310 rpcz_store.cc:269] 0709 12:26:11.502066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.593167 (+ 91101us) service_pool.cc:224] Handling call
0709 12:26:13.193113 (+1599946us) spinlock_profiling.cc:243] Waited 70.6 ms on lock 0x7b54001b21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:13.193223 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":162477440}
W20260709 12:26:13.197260  6274 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.639s (SENT)
I20260709 12:26:13.198693  9915 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:13.198820  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:13.199220  9915 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:13.199605  7318 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:13.199659  9915 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.200472  9912 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:13.187058  9926 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:13.201719  9926 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.223570  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.225507  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:13.231916  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.231591  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45208.2us
I20260709 12:26:13.232975  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:13.233800  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.235325  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2365) took 1641 ms. Trace:
I20260709 12:26:13.235736  6380 rpcz_store.cc:276] 0709 12:26:11.594251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.945331 (+1351080us) service_pool.cc:224] Handling call
0709 12:26:12.945638 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.945879 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:12.965254 (+ 19375us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.229518 (+264264us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.234949 (+  5431us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.235005 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:13.235059 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.235130 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.235262 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15009,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":4219,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":52,"thread_start_us":254397,"threads_started":3,"wal-append.queue_time_us":113843}
I20260709 12:26:13.237910  9950 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:13.238584  9950 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:13.239161  9950 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.240693  9937 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:13.241747  9937 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:13.242713  9937 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.254633  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.256963  9942 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:13.280711  9911 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:13.281567  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49698.5us
I20260709 12:26:13.282673  9928 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:13.283761  9926 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:13.229202  9936 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
W20260709 12:26:13.285595  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.287140  9913 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:13.288769  6272 leader_election.cc:400] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:26:13.289168  6272 leader_election.cc:403] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:13.292548  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:13.293517  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:13.295576  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:13.296523  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71116.3us
I20260709 12:26:13.234687  9930 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.301124  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:13.304816  6450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2023) took 4952 ms (client timeout 2238 ms). Trace:
W20260709 12:26:13.305227  6450 rpcz_store.cc:269] 0709 12:26:08.352215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.874759 (+522544us) service_pool.cc:224] Handling call
0709 12:26:13.304468 (+4429709us) spinlock_profiling.cc:243] Waited 91.1 ms on lock 0x7b54001635d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:13.304631 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209436416}
I20260709 12:26:13.306649  9933 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:13.295002  9926 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7.
I20260709 12:26:13.307511  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:13.308295  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2363) took 1714 ms (client timeout 2226 ms). Trace:
W20260709 12:26:13.308678  6383 rpcz_store.cc:269] 0709 12:26:11.593642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.307203 (+1713561us) service_pool.cc:224] Handling call
0709 12:26:13.308238 (+  1035us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:13.306365  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 1977) took 1820 ms (client timeout 2111 ms). Trace:
W20260709 12:26:13.309768  6302 rpcz_store.cc:269] 0709 12:26:11.486264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.722720 (+1236456us) service_pool.cc:224] Handling call
0709 12:26:13.306215 (+583495us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b540010f254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:13.306310 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324588800}
I20260709 12:26:13.311137  6380 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:13.313084  6376 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:13.313251  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:13.312655  9909 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:13.300880  9783 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.314594  9909 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:13.314888  9783 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:13.315052  9909 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.315322  9783 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:09.841882  6304 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
W20260709 12:26:13.316172  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1776) took 3941 ms (client timeout 1905 ms). Trace:
W20260709 12:26:13.316557  6304 rpcz_store.cc:269] 0709 12:26:09.374353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.378776 (+  4423us) service_pool.cc:224] Handling call
0709 12:26:13.316103 (+3937327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:13.315804  9950 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.324522  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:13.330076  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 406) took 4004 ms (client timeout 1774 ms). Trace:
W20260709 12:26:13.330660  6378 rpcz_store.cc:269] 0709 12:26:09.325730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.726462 (+400732us) service_pool.cc:224] Handling call
0709 12:26:13.324311 (+3597849us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:26:13.329988 (+  5677us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14992640}
W20260709 12:26:13.340484  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.322742  9933 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:13.342394  9766 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:13.342168  9933 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.343000  9766 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:13.343098  9930 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:13.344103  9930 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:13.344084  9766 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:13.345758  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53289us
I20260709 12:26:13.342113  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:13.341090  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 151106us
W20260709 12:26:13.359035  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 125036us
I20260709 12:26:13.322575  9770 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.363584  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:13.368399  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:13.339371  9915 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.323132  9909 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.357040  9931 raft_consensus.cc:2764] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:13.382917  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.373955  9770 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.381472  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.339920  9759 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.369782  9930 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:13.387923  6383 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:13.376734  9928 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:13.391513  6237 debug-util.cc:398] Leaking SignalData structure 0x7b080019df60 after lost signal to thread 6334
W20260709 12:26:13.391901  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08001bfec0 after lost signal to thread 6335
I20260709 12:26:13.393324  9944 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.402050  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1798) took 1912 ms. Trace:
I20260709 12:26:13.402671  6300 rpcz_store.cc:276] 0709 12:26:11.489335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.101059 (+1611724us) service_pool.cc:224] Handling call
0709 12:26:13.101414 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.101751 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.101779 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.102338 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.210707 (+108369us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800251920. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:13.376283 (+165576us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:13.376328 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.376402 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.400275 (+ 23873us) raft_consensus.cc:1685] finished
0709 12:26:13.400296 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.401955 (+  1659us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1799,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":22271,"spinlock_wait_cycles":27452672,"thread_start_us":165357,"threads_started":1,"wal-append.queue_time_us":288115}
I20260709 12:26:13.405619  9933 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.406212  9933 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:13.406757  9933 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.416931  9909 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:13.419201  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:13.420251  6384 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:13.390004  9930 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:13.442970  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96201.2us
I20260709 12:26:13.420372  9931 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:13.495515  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:13.494016  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:13.495915  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:13.498318  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:13.549886  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.550705  6384 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:26:13.556253  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81899.6us
I20260709 12:26:13.558729  9964 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:13.559211  9964 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.559541  9964 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:13.559793  7320 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:13.561966  9950 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:13.559945  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:13.492219  9931 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.568888  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.579598  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.580552  6384 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:13.582091  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.491815  9932 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:26:13.420962  9944 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.593159  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2493) took 3930 ms. Trace:
I20260709 12:26:13.594027  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:13.576346  9931 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.574298  9446 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:13.601594  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.603672  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:13.577920  9759 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:13.586544  9926 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:13.584052  9963 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:13.607115  7332 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:13.600481  6459 rpcz_store.cc:276] 0709 12:26:09.663169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.010938 (+1347769us) service_pool.cc:224] Handling call
0709 12:26:11.011311 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.011671 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:11.012013 (+   342us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.592712 (+2580699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.592769 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:13.592825 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.592919 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.593080 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2569341,"prepare.queue_time_us":165,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":4671,"threads_started":1,"wal-append.queue_time_us":11000,"wal-append.run_cpu_time_us":1427,"wal-append.run_wall_time_us":1015606}
I20260709 12:26:13.610570  9829 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:13.644793  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 411) took 2076 ms (client timeout 2729 ms). Trace:
I20260709 12:26:13.645735  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.608775  9926 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:13.642228  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.643167  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:13.645231  6381 rpcz_store.cc:269] 0709 12:26:11.568351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.637296 (+1068945us) service_pool.cc:224] Handling call
0709 12:26:13.644605 (+1007309us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:13.644730 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275083648}
I20260709 12:26:13.652980  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.653702  6459 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.656236  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.657270  6376 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:13.658270  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:13.658973  6376 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:13.659798  9966 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:26:13.614552  9446 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.654994  9926 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:13.661895  9446 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:13.663146  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:13.638311  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71040.9us
I20260709 12:26:13.665714  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.587889  9930 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:13.682410  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800356260 after lost signal to thread 6409
I20260709 12:26:13.688458  6450 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:26:13.678838  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.689476  6381 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:13.644146  9963 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:13.682971  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.707211  6453 raft_consensus.cc:1217] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Deduplicated request from leader. Original: 4.3->[5.4-7.5]   Dedup: 5.4->[7.5-7.5]
I20260709 12:26:13.707847  6453 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:26:13.708339  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2033) took 4216 ms. Trace:
W20260709 12:26:13.692828  6237 debug-util.cc:398] Leaking SignalData structure 0x7b080040e180 after lost signal to thread 6483
I20260709 12:26:13.709137  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.708886  6457 rpcz_store.cc:276] 0709 12:26:09.491502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.664327 (+1172825us) service_pool.cc:224] Handling call
0709 12:26:10.668391 (+  4064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.546901 (+878510us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:26:11.558796 (+ 11895us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.707248 (+2148452us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.707319 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:13.707980 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.708097 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.708256 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1909530,"prepare.queue_time_us":10054,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"thread_start_us":198396,"threads_started":2,"wal-append.queue_time_us":1057152}
W20260709 12:26:13.714157  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08003edbc0 after lost signal to thread 7371
I20260709 12:26:13.718938  6455 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:13.723400  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08002e7200 after lost signal to thread 7373
W20260709 12:26:13.723887  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800357e60 after lost signal to thread 9311
W20260709 12:26:13.724467  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08003ce820 after lost signal to thread 9602
W20260709 12:26:13.724790  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08003bc280 after lost signal to thread 9642
W20260709 12:26:13.672890  9962 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:26:13.684697  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2481) took 5677 ms. Trace:
I20260709 12:26:13.685038  9968 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:26:13.663800  6384 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
W20260709 12:26:13.675612  9955 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:26:13.654506  9915 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:13.754356  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1897) took 1120 ms. Trace:
I20260709 12:26:13.754974  6382 rpcz_store.cc:276] 0709 12:26:12.634342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.137889 (+503547us) service_pool.cc:224] Handling call
0709 12:26:13.138208 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.163807 (+ 25599us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:13.179248 (+ 15441us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.369900 (+190652us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.370366 (+   466us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.370522 (+   156us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:13.370556 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.370620 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.754005 (+383385us) raft_consensus.cc:1685] finished
0709 12:26:13.754034 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.754238 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10971,"prepare.queue_time_us":15400,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":11644,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":3770,"thread_start_us":156426,"threads_started":1,"wal-append.queue_time_us":208169}
I20260709 12:26:13.766808  9930 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:13.769615  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1868) took 5727 ms. Trace:
I20260709 12:26:13.770090  6379 rpcz_store.cc:276] 0709 12:26:08.042038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.958858 (+2916820us) service_pool.cc:224] Handling call
0709 12:26:10.959236 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.042959 (+ 83723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.769357 (+2726398us) spinlock_profiling.cc:243] Waited 82.9 ms on lock 0x7b5400109ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:13.769551 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190586624}
I20260709 12:26:13.773450  6379 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:13.726289  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.726315  6453 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.778379  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.784695  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.786429  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.787223  6302 raft_consensus.cc:2376] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6. Re-sending same reply.
I20260709 12:26:13.791779  6379 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:26:13.693143  6425 connection.cc:256] RPC call timeout handler was delayed by -2.746858314000292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
W20260709 12:26:13.730907  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 123939us
I20260709 12:26:13.708155  9765 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:13.734809  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.736038  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:13.802662  6381 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:13.740056  9668 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:13.803279  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.749547  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.750043  6450 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:13.677464  9944 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:13.792924  6425 connection.cc:256] RPC call timeout handler was delayed by -2.7078886809999858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2015s for a response
I20260709 12:26:13.808243  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.793509  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:13.795703  6459 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:13.814616  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.814756  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:13.796725  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:13.800498  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.816114  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.801528  6308 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:13.802906  6377 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:13.805262  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.718001  9963 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:13.817373  6450 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:13.818713  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.818863  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.819583  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.823159  6376 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:13.822978  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.825244  6304 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:13.826790  6453 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:13.820389  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.823318  6301 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:13.831859  6303 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:13.832270  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.828590  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.731848  6452 rpcz_store.cc:276] 0709 12:26:07.956223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:07.968314 (+ 12091us) service_pool.cc:224] Handling call
0709 12:26:07.968741 (+   427us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.052853 (+ 84112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.052916 (+    63us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:08.061392 (+  8476us) log.cc:844] Serialized 52 byte log entry
0709 12:26:11.044417 (+2983025us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800492be0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:11.059493 (+ 15076us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:11.059908 (+   415us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.608663 (+2548755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.608745 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.608784 (+    39us) raft_consensus.cc:1685] finished
0709 12:26:13.614741 (+  5957us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.633708 (+ 18967us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2626673,"prepare.queue_time_us":178,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":2768502,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":47779,"spinlock_wait_cycles":28625792,"thread_start_us":91485,"threads_started":2,"wal-append.queue_time_us":2937882,"wal-append.run_cpu_time_us":2409,"wal-append.run_wall_time_us":1040824}
I20260709 12:26:13.828896  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.852442  6458 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:13.853996  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.860852  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:13.861667  6307 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.818475  9765 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.802270  9783 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:13.864210  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.865480  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.829911  9963 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:13.869932  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.870782  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.871768  6459 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:13.867318  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.874017  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.857354  9766 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:13.878803  9783 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:13.878643  9964 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:13.885726  6458 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:13.827934  9980 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:13.888837  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2490) took 4256 ms. Trace:
I20260709 12:26:13.890147  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:13.891311  6309 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 5: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:13.898715  6307 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:13.902951  6304 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:13.904884  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.907883  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.909018  6459 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:13.889227  6451 rpcz_store.cc:276] 0709 12:26:09.632929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.934106 (+1301177us) service_pool.cc:224] Handling call
0709 12:26:10.934568 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.585819 (+2651251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.585854 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.590670 (+  4816us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.718941 (+128271us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:13.718982 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.719115 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:13.888640 (+169525us) raft_consensus.cc:1685] finished
0709 12:26:13.888663 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.888782 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2646131,"raft.queue_time_us":2684536,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":4682,"thread_start_us":115793,"threads_started":1,"wal-append.queue_time_us":287541}
I20260709 12:26:13.913496  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.916759  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.921530  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.922722  6458 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:13.891919  9404 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:13.898444  9964 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:13.928077  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.927078  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:13.935360  6451 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:13.936154  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:13.931167  6457 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:13.993237  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.031266  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 695) took 1151 ms. Trace:
I20260709 12:26:13.928992  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:14.051460  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 84257.6us
I20260709 12:26:14.031955  7329 rpcz_store.cc:276] 0709 12:26:12.879592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.879695 (+   103us) service_pool.cc:224] Handling call
0709 12:26:14.030984 (+1151289us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b54005c1054. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:14.031184 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16793856}
I20260709 12:26:14.060981  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.065712  6451 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:14.068027  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.068740  6459 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.067163  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.071287  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:14.071765  6451 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.070933  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:14.083348  6459 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:14.084455  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.085084  6459 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:14.087795  6274 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.984s (SENT)
I20260709 12:26:14.088286  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:14.091085  6425 connection.cc:256] RPC call timeout handler was delayed by -2.585272939999868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
W20260709 12:26:14.091725  6425 connection.cc:256] RPC call timeout handler was delayed by -2.4981886810001015s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22480000000000003s for a response
W20260709 12:26:14.092271  6425 connection.cc:256] RPC call timeout handler was delayed by -2.343388681000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.242s for a response
W20260709 12:26:14.092670  6425 connection.cc:256] RPC call timeout handler was delayed by -2.1915799270000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 12:26:14.093091  6425 connection.cc:256] RPC call timeout handler was delayed by -2.1915799270000207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1937s for a response
W20260709 12:26:14.093468  6425 connection.cc:256] RPC call timeout handler was delayed by -2.1903886810000586s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.259s for a response
W20260709 12:26:14.093842  6425 connection.cc:256] RPC call timeout handler was delayed by -2.1289245640000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
W20260709 12:26:14.094236  6425 connection.cc:256] RPC call timeout handler was delayed by -2.0128752059999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260709 12:26:14.094805  6425 connection.cc:256] RPC call timeout handler was delayed by -1.9340679450001517s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
I20260709 12:26:14.094959  6459 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:26:14.095623  6425 connection.cc:256] RPC call timeout handler was delayed by -1.9012729399996715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2292s for a response
I20260709 12:26:14.100859  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.101727  6452 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:26:14.111675  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 63868us
I20260709 12:26:14.114651  7317 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:14.023383  9404 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:14.156528  6272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.012s	sys 0.002s
W20260709 12:26:14.157070  6272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.012s	sys 0.002s
W20260709 12:26:14.157799  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 789435us
I20260709 12:26:14.161365  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.167232  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.168349  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:14.169250  6385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2371) took 2561 ms (client timeout 1984 ms). Trace:
W20260709 12:26:14.169672  6385 rpcz_store.cc:269] 0709 12:26:11.607462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.169029 (+2561567us) service_pool.cc:211] Skipping call since client already timed out
0709 12:26:14.169177 (+   148us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:26:14.139024  9783 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.174499  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:14.176471  6452 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:14.171752  9766 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:14.179015  9766 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:14.179240  9979 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:26:14.179853  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 449) took 2697 ms. Trace:
I20260709 12:26:14.180018  9766 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.180246  6305 rpcz_store.cc:276] 0709 12:26:11.193193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.195114 (+  1921us) service_pool.cc:224] Handling call
0709 12:26:11.195668 (+   554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.692289 (+2496621us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b2c00123048. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1557291 00007f6fa15720ca 00007f6fa32a04ad 00007f6fa32a6e1b 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:26:13.731154 (+ 38865us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:13.735361 (+  4207us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.890059 (+154698us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.890121 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:13.890177 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.890264 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.890439 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":163860,"prepare.queue_time_us":5375,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":54,"raft.queue_time_us":89,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":11794,"spinlock_wait_cycles":171732864,"thread_start_us":154147,"threads_started":1,"wal-append.queue_time_us":314841}
I20260709 12:26:14.158993  9980 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:26:14.186697  9967 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:26:14.191205  9926 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.530s	user 0.010s	sys 0.000s
I20260709 12:26:14.139837  9964 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:26:14.177467  9976 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:14.211210  9976 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:14.204568  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:14.205930  9933 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.799s	user 0.008s	sys 0.005s
W20260709 12:26:14.203796  9931 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.594s	user 0.000s	sys 0.009s
I20260709 12:26:14.206933  9770 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:14.199270  9783 raft_consensus.cc:2764] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:14.210448  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 691) took 1335 ms. Trace:
I20260709 12:26:14.240507  7316 rpcz_store.cc:276] 0709 12:26:12.874681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.874827 (+   146us) service_pool.cc:224] Handling call
0709 12:26:13.168957 (+294130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:13.174849 (+  5892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.210230 (+1035381us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54007a0dd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:14.210384 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675570048}
I20260709 12:26:14.210695  9909 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:14.211652  9976 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:14.191089  6385 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:14.232867  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.239717  7314 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:14.243579  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.247051  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.247581  6458 raft_consensus.cc:2376] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10. Re-sending same reply.
I20260709 12:26:14.212227  9967 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:14.249408  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.241052  9931 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.249843  9944 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:14.252363  7319 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:14.242954  9909 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:14.261163  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.225893  9926 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.265820  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.251144  9770 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:26:14.241159  9933 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.270723  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.273636  6310 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:14.271621  6458 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:14.280547  6451 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:14.267084  9944 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:14.281800  6303 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:14.283435  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.286940  6457 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:14.286948  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.288321  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.289642  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:14.290251  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:14.287873  6458 raft_consensus.cc:2376] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6. Re-sending same reply.
I20260709 12:26:14.292513  6458 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:14.293025  6451 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:14.294786  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:14.295537  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 127220us
I20260709 12:26:14.296376  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:14.268533  9783 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:26:14.297063  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:14.297808  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.296365  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:14.304579  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.298321  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.298899  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:14.309619  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:14.311045  6378 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:14.311702  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.312516  6305 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:26:14.303946  9357 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:14.315850  9357 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:14.304435  9980 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:14.317028  9980 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:14.316726  9357 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.317430  9980 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.300928  9964 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:14.322845  9964 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:14.323249  9964 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:14.327602  9933 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:14.312391  9766 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.304960  9770 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:26:14.329706  9967 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.329974  9909 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.303525  9884 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.318284  9357 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.338757  9944 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.340377  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2491) took 4707 ms. Trace:
W20260709 12:26:14.341563  6424 connection.cc:256] RPC call timeout handler was delayed by -2.924937884999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 12:26:14.367383  6424 connection.cc:256] RPC call timeout handler was delayed by -2.8435304620002171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.259s for a response
I20260709 12:26:14.343418  9926 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:14.343708  9977 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:14.344070  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:14.369335  6307 raft_consensus.cc:2376] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9. Re-sending same reply.
W20260709 12:26:14.344269  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 154146us
I20260709 12:26:14.370930  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.365160  6456 rpcz_store.cc:276] 0709 12:26:09.633284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.987894 (+1354610us) service_pool.cc:224] Handling call
0709 12:26:10.988200 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.988552 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:11.005191 (+ 16639us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.667870 (+2662679us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.751582 (+ 83712us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b1800492dc0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:14.339992 (+588410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:14.340047 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:14.340095 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.340188 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.340316 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3769125,"prepare.queue_time_us":66418,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":98,"raft.queue_time_us":3169585,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":20552,"spinlock_wait_cycles":17039104,"thread_start_us":231155,"threads_started":4,"wal-append.queue_time_us":267131,"wal-append.run_cpu_time_us":1446,"wal-append.run_wall_time_us":1013107}
I20260709 12:26:14.346310  9931 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:14.346707  9829 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:14.314568  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:14.350468  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.351455  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:14.388002  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73586.1us
I20260709 12:26:14.337710  9963 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.354678  9967 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.355247  6377 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:14.354812  6206 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 5 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:14.357035  6384 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:14.357851  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.354748  9819 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:14.360092  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.360868  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.408501  6379 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:14.373204  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.410418  6459 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:14.373366  9977 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.373824  9985 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:14.386888  6378 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:14.389114  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:14.401012  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50630.6us
W20260709 12:26:14.423349  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 110061us
W20260709 12:26:14.415170  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57379.4us
I20260709 12:26:14.428428  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.429083  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.429255  6450 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:14.426762  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.430195  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.430095  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.431186  6453 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:14.431239  6459 raft_consensus.cc:2376] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7. Re-sending same reply.
I20260709 12:26:14.432480  9929 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:14.434326  9929 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:14.435175  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:14.435384  9937 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:14.436195  9679 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:14.435686  9929 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:14.437300  9929 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:14.437523  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.438819  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.438205  9929 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.436861  9679 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:14.440327  9929 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:14.440699  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.440513  9679 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.443378  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.444183  9931 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:14.444680  9931 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.445415  9931 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.450150  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.455073  9851 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.444700  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:14.387238  9837 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:14.463928  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.396641  9783 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.407701  9668 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:14.470155  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 97038.1us
I20260709 12:26:14.407174  9909 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:14.407356  9848 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:14.418349  9573 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.476327  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.457098  9782 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:14.496500  6424 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.713s (SENT)
I20260709 12:26:14.416687  9978 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:14.498600  6376 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:26:14.470183  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.516927  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:14.520803  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50699.8us
W20260709 12:26:14.523890  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 134915us
I20260709 12:26:14.525343  6383 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:14.527184  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.479452  9992 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:14.498405  9988 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:26:14.530180  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.481539  9668 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.523017  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:14.535586  6456 raft_consensus.cc:2376] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12. Re-sending same reply.
I20260709 12:26:14.415428  9829 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.499476  9851 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.538203  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:14.538146  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 239893us
I20260709 12:26:14.541334  9848 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.539633  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:14.419080  9819 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.526558  9997 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:14.541517  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65297.6us
I20260709 12:26:14.542764  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.552760  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:14.556331  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 115684us
I20260709 12:26:14.554999  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.566078  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.563495  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:14.569944  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.571620  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100065us
I20260709 12:26:14.537191  9992 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.540352  9668 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.553639  9837 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.027s
W20260709 12:26:14.578298  9991 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:14.578701  9837 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:14.575371  6452 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:14.539687  9846 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:14.579867  9989 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:14.579211  9837 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:14.580888  9837 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.582087  9837 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:14.574342  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44068.5us
I20260709 12:26:14.523429  9978 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.597038  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.597038  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.598753  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.544305  9819 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:14.601789  9944 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:14.542275  9404 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:14.598206  9848 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:14.612360  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:14.575783  9997 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.585011  9573 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:14.617020  9930 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:14.626024  9573 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:14.616560 10007 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:14.626551  9573 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:14.626916 10007 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.627354 10007 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.576119  9992 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:14.629779  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 90217.1us
I20260709 12:26:14.630919  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:14.598914  9765 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:14.631636  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:14.632669  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66725.8us
I20260709 12:26:14.611001  9933 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.634526  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:14.633701  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.616726  9937 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.609193  9944 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:14.643169  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:14.643908  6424 connection.cc:256] RPC call timeout handler was delayed by -2.7189767500003654s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20840000000000003s for a response
W20260709 12:26:14.645764  6424 connection.cc:256] RPC call timeout handler was delayed by -2.7041143010001178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21660000000000001s for a response
W20260709 12:26:14.646096  6424 connection.cc:256] RPC call timeout handler was delayed by -1.2670344289999775s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1917s for a response
I20260709 12:26:14.650934  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:14.651899  6308 raft_consensus.cc:2376] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5. Re-sending same reply.
I20260709 12:26:14.653553  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.654335  6453 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:14.654898  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.656522  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.657254  6459 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:14.659981  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.664410  6304 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:14.667327  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.673187  6308 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:26:14.673979  6457 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:14.677613  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:14.681846  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:14.684541  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.686990  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.691110  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.694873  6450 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:14.696281  6304 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:14.667919  9944 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:14.666563  9765 raft_consensus.cc:2764] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:14.708132  6300 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:14.722126  6302 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:14.732024  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.708024  9782 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:14.735266  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88490.7us
I20260709 12:26:14.742317  6300 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:14.742168  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.753983  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:14.771577  9782 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:14.773809  9976 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.000s
I20260709 12:26:14.674798  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:26:14.774269  9976 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:14.774969  6454 raft_consensus.cc:606] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:14.775385  6454 raft_consensus.cc:614] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:14.777221 10001 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:26:14.759640  9884 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:26:14.770615  9968 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:26:14.779335  9994 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:26:14.759450  9929 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:26:14.771703  9931 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.690368  9963 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:14.727866  9679 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 12:26:14.810614  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53795.3us
I20260709 12:26:14.812199  9936 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:14.813047  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51616.3us
I20260709 12:26:14.709914  9980 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.770768  9600 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:14.822844  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 188399us
I20260709 12:26:14.824262  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:14.831634  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 94191.8us
I20260709 12:26:14.754534 10000 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:14.708093 10003 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:14.758827 10009 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:14.853665  6459 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:14.855911  6453 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:14.753041 10005 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:14.899890  6456 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:14.823665  9829 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:14.735875 10002 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:14.830994  9600 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.910339  9600 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:14.910605  9829 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:14.841821  9750 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:26:14.814844  9842 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.094s
I20260709 12:26:14.912514  9829 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:14.903896 10003 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:14.913663 10003 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:14.916939  9978 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:14.926676 10004 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.927678 10004 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:14.929592 10006 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.288s	user 0.000s	sys 0.002s
I20260709 12:26:14.935065  6206 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 5, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:14.937250  6206 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 5, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:14.940377  9668 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:14.941592  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:14.942659  9993 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:26:14.942836  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.943892  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:14.944782  9926 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:14.945418  9930 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:14.945909  9930 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:14.946549  9926 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:14.947474  9926 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:14.948462  9783 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:14.949101  9783 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:14.950186  9783 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:14.910455  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.955469  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:14.957463  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 139045us
W20260709 12:26:14.940433  6201 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 263) took 22796 ms (client timeout 15000 ms). Trace:
W20260709 12:26:14.968606  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 286367us
I20260709 12:26:14.970440  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:14.898701  9997 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.976199  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:14.954597  9926 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:14.908233  9964 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.018s
I20260709 12:26:14.981659  9928 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:15.041569  7319 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:15.030823  9668 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.047175  6302 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:15.047289  6206 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 6, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 6 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:15.049013  6206 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 3 to 6, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:15.055076 10004 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:15.061000  9926 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.064553  6206 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 5, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:14.976653  9985 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.991818  9967 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.073360  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 439688us
W20260709 12:26:15.071822  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 253930us
I20260709 12:26:15.081960  7317 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:26:15.094926  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 335543us
I20260709 12:26:15.095969  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:15.067418  9668 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:15.098470  6206 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 6, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:15.101332 10004 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.027333  9964 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.126006  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:15.117204  6206 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 8 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 10 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:15.093806 10009 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:15.071465 10011 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:15.094517  9600 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.149142 10002 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:15.085983  9967 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.142557 10011 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.134975  9846 raft_consensus.cc:2764] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won
I20260709 12:26:15.158975  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:15.155817  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:15.159870  9937 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:15.162851  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:15.154228  9837 raft_consensus.cc:2764] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:26:15.172170  6307 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:15.172833  9750 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:15.176504  9976 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:15.177577  7329 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:15.176491  7317 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:15.183305  6454 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:15.185068  6206 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:15.108925  9765 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:26:15.073063  9930 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.233958  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:15.205844  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:15.132462  9668 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.156886  9977 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:15.229712  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.241200  6303 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:15.171438 10008 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.529s	user 0.002s	sys 0.000s
I20260709 12:26:15.234885  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:15.249436  7317 raft_consensus.cc:2368] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7.
I20260709 12:26:15.187837 10011 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:15.242094  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.205178  9829 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:15.202564  9851 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:14.964126  6201 rpcz_store.cc:269] 0709 12:25:52.143993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.145465 (+  1472us) service_pool.cc:224] Handling call
0709 12:26:14.940175 (+22794710us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:05.620104 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:05.620909 (+   805us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:05.620981 (+    72us) write_op.cc:435] Acquired schema lock
0709 12:26:05.621026 (+    45us) tablet.cc:662] Decoding operations
0709 12:26:05.784083 (+163057us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:05.784162 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:05.784205 (+    43us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:26:05.786259 (+  2054us) tablet.cc:701] Row locks acquired
0709 12:26:05.786301 (+    42us) write_op.cc:260] PREPARE: finished
0709 12:26:05.801293 (+ 14992us) write_op.cc:270] Start()
0709 12:26:05.801463 (+   170us) write_op.cc:276] Timestamp: P: 1783599965786445 usec, L: 0
0709 12:26:05.801482 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:26:05.833383 (+ 31901us) log.cc:844] Serialized 7332 byte log entry
0709 12:26:06.009402 (+176019us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:06.627420 (+618018us) op_driver.cc:464] REPLICATION: finished
0709 12:26:06.981002 (+353582us) write_op.cc:301] APPLY: starting
0709 12:26:06.994797 (+ 13795us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:06.995375 (+   578us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:06.995428 (+    53us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:06.998292 (+  2864us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:07.010836 (+ 12544us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:26:07.010918 (+    82us) write_op.cc:312] APPLY: finished
0709 12:26:07.011497 (+   579us) log.cc:844] Serialized 132 byte log entry
0709 12:26:07.011693 (+   196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:07.011878 (+   185us) write_op.cc:454] Released schema lock
0709 12:26:07.011956 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:07.012494 (+   538us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:07.012539 (+    45us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3847,"reported_tablets":37,"child_traces":[["op",{"apply.queue_time_us":308090,"apply.run_cpu_time_us":6076,"apply.run_wall_time_us":38738,"num_ops":19,"prepare.queue_time_us":242035,"prepare.run_cpu_time_us":29462,"prepare.run_wall_time_us":312817,"raft.queue_time_us":406004,"raft.run_cpu_time_us":44395,"raft.run_wall_time_us":277354,"replication_time_us":789958,"spinlock_wait_cycles":4687104,"thread_start_us":594954,"threads_started":4,"wal-append.queue_time_us":89656,"wal-append.run_cpu_time_us":17393,"wal-append.run_wall_time_us":2077623}]]}
I20260709 12:26:15.253572  6305 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:15.257493  6206 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:15.226801  9928 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:15.255903  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 164590us
I20260709 12:26:15.158098  9328 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.078s
I20260709 12:26:15.226243  9846 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:15.269338  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:15.270985  6455 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:15.259943  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:15.271705  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:15.266698  9998 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:26:15.274986  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 119266us
I20260709 12:26:15.287307  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:15.236054  9931 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.276077  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.289081  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:15.242726  9848 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.278745  7320 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:15.240931 10005 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:15.272365  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.281188  6455 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:15.284106  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:15.282215  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:15.213083 10003 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.281607  7316 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:15.271311  9679 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.278740 10014 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.163s	user 0.002s	sys 0.000s
I20260709 12:26:15.294859  7318 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:15.278429 10000 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.291893  6425 leader_election.cc:400] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7.
I20260709 12:26:15.301177  9964 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.306212  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:15.289139  9573 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.002s
I20260709 12:26:15.312072  6425 leader_election.cc:403] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:15.260883  9884 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:15.261065 10009 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.295022  6455 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.343608  9600 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:15.344059  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.010s
I20260709 12:26:15.343281  6206 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 6 to 4, leader changed from <none> to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:26:15.345563  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48312.5us
I20260709 12:26:15.345889 10023 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 12:26:15.416139  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:15.421228  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:15.346691 10020 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:15.177217 10012 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 12:26:15.347103 10029 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:15.347450  9668 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.257647  9783 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:15.303366  9931 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:15.269388 10017 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
W20260709 12:26:15.349771  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62568.1us
I20260709 12:26:15.302894  9848 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.350873  9842 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:15.352846 10005 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.074647  9985 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.270298 10010 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:26:15.270825  9851 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.374291  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 729) took 1044 ms. Trace:
W20260709 12:26:15.378909  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 73065.6us
I20260709 12:26:15.414261  6455 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:15.417172  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.001s	sys 0.010s
I20260709 12:26:15.041010  9997 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.419857  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.480516  6305 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:15.420305  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 139673us
I20260709 12:26:15.421968  6458 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:15.481685  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:15.379930 10025 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:15.313619 10004 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:15.300241 10027 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.193s
I20260709 12:26:15.430583  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.431023  6308 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:26:15.472121  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.479871  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 655665us
I20260709 12:26:15.374585  9782 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.522727  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:15.439275  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.484022  7314 rpcz_store.cc:276] 0709 12:26:14.329900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:14.346871 (+ 16971us) service_pool.cc:224] Handling call
0709 12:26:14.347246 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:14.347944 (+   698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:14.347976 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:14.348519 (+   543us) log.cc:844] Serialized 26 byte log entry
0709 12:26:14.617107 (+268588us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:14.617156 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:14.617275 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:15.170733 (+553458us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:15.373525 (+202792us) raft_consensus.cc:1685] finished
0709 12:26:15.373555 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:15.374210 (+   655us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":11862784,"thread_start_us":267895,"threads_started":1,"wal-append.queue_time_us":293095}
I20260709 12:26:15.436255  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.436299  6206 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 7 to 8, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 8 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:15.490561  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:15.437894  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:15.529459  6301 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:15.387490  9573 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.524682  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.483422  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:15.437526 10023 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:15.540302  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:15.493062  9404 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:26:15.541229  6303 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:15.493754  6308 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:15.532465  6455 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:15.523698  9750 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:15.487389  9842 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:15.539516  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56191.9us
I20260709 12:26:15.547732  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:15.430562  9846 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7.
I20260709 12:26:15.437804  9936 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:26:15.551036 10019 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:15.551422  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:15.313515 10002 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.313387  9679 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.556056  6303 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:15.556221  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:15.530581 10004 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:15.560438  6455 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:15.543879  9600 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:15.538859 10028 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.533241  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:15.609328  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76170.6us
W20260709 12:26:15.609354  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.610160  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:15.598282  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59703.9us
I20260709 12:26:15.376681 10000 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:15.604373  6308 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:15.616103  7335 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:15.551590  9842 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.546465  9967 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.551457  9848 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.480067 10029 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.563388  9976 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.551079 10018 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.424247  9928 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.624415  6308 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:15.628573  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:15.438223 10020 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.635617  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:15.622339 10019 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:15.664148  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:15.669026 10003 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.597460  9837 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.075s
I20260709 12:26:15.563799  9446 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:26:15.666742  6458 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:15.543294  9782 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.615176  9668 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:15.669011  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:15.679311 10042 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:26:15.676082  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:15.682111  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.561213 10046 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:15.711982  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.637830  9328 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 12:26:15.641800  9765 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 12:26:15.710822  9992 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.066s	user 0.016s	sys 0.001s
I20260709 12:26:15.744639  6301 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:26:15.746388  6303 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:15.616988 10028 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:15.748526 10039 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.356s	user 0.000s	sys 0.001s
I20260709 12:26:15.769526  6272 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:15.771418  6272 leader_election.cc:403] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:15.776755  9930 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:15.780802  9573 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:15.783303 10040 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:26:15.721923  9976 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.812698  6452 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:15.768119 10019 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:15.838629 10035 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.840272 10050 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:26:15.752148 10004 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:15.811609 10027 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:15.772660  9851 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:15.772876  9980 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.772028  9668 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:15.845072  6458 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:15.798130  9985 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:15.849555  6452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:26:15.841239 10048 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
W20260709 12:26:15.839773 10051 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:26:15.838955 10026 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.265s	user 0.001s	sys 0.000s
I20260709 12:26:15.863148  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:15.878460  6455 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:15.791993  9977 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 12:26:15.769815  9992 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:15.876111  6452 raft_consensus.cc:606] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:15.880223  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:15.886289  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 331540us
W20260709 12:26:15.897027  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.957767  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 77646.6us
I20260709 12:26:15.963114  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:15.964048  6455 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:15.966670  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:15.882164 10045 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
I20260709 12:26:15.838670  9964 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.065s
W20260709 12:26:15.903949 10040 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:26:15.850361 10024 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.120s
I20260709 12:26:15.960719  9679 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.969655 10000 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:15.960606  9977 consensus_queue.cc:1243] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:15.854712 10035 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:15.968941 10017 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.992043  6307 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:15.851966 10019 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:15.987506  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 318593us
I20260709 12:26:16.067451  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:16.033600  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
W20260709 12:26:16.032653  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 229289us
I20260709 12:26:16.022045  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:16.068493  6451 raft_consensus.cc:606] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:15.970453 10032 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:16.068707  6450 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:15.996276  9944 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.267s	user 0.011s	sys 0.002s
W20260709 12:26:15.996629 10055 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 12:26:15.962399  9357 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.114s
I20260709 12:26:16.073520  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:15.997304 10038 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
W20260709 12:26:16.031021 10011 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.001s
I20260709 12:26:16.087255  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:16.088546  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:16.025403  9829 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.005s
I20260709 12:26:16.003199 10028 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.033681  9997 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.000123  9782 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.123371  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.123767  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.025640 10024 consensus_queue.cc:1243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:16.124164  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.158097  6379 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:16.127213  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:16.120428  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:16.127158  6301 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:16.129544  9600 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.001s
I20260709 12:26:15.989360  9980 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:16.131573  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:16.135043  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.165637  6382 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:16.166132  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:16.166956  6379 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:16.166888  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.168329  6382 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:16.169410  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.133834  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.140386  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:15.999354 10020 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.092656 10059 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:16.124191  9846 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:16.091117 10012 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.011s	sys 0.018s
I20260709 12:26:16.149682  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.189157  6384 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:16.156772  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.159423  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.159999  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.160197  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:16.082897 10032 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.154971 10036 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:16.069948  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 322843us
W20260709 12:26:16.162175  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94818.4us
I20260709 12:26:16.204234  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:16.012481  9992 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.180375  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:16.136588  9357 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:16.144013  9829 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:16.179926  6385 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:16.064030 10017 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.130029 10029 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.173236 10018 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:16.213937  6377 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: [6.7-8.8]
I20260709 12:26:16.214975  6458 raft_consensus.cc:1217] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-13.7]   Dedup: 8.5->[11.6-13.7]
I20260709 12:26:16.215581  6458 pending_rounds.cc:85] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 5
I20260709 12:26:16.215811  6458 pending_rounds.cc:107] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:16.217056  6458 raft_consensus.cc:2889] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:16.217523  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57443.8us
I20260709 12:26:16.093169  9944 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.160666  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.187373  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53146508e564f8aae279a61e6696e80"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:26:16.238682  6451 raft_consensus.cc:606] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:16.239148  6451 raft_consensus.cc:614] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:16.212524  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:16.192508 10012 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.669s	user 0.015s	sys 0.018s
I20260709 12:26:16.181007  9600 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:16.179715  9980 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.210031  9753 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.196548  9573 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:26:16.219623  9930 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:16.196947  9765 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 12:26:16.221064 10061 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:26:16.224236  9679 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.225028  9446 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:16.226791 10007 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.599s	user 0.016s	sys 0.000s
I20260709 12:26:16.195734  6450 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:16.242131  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:16.207568 10032 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.245150  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:16.245975  6377 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.143148 10011 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:16.268571  6308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.007s
I20260709 12:26:16.187932  9837 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:26:16.210532  9976 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.093326  9668 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.286751  6381 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:26:16.191504  9846 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.278337  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:26:16.207772  9328 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:26:16.216269  9829 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:26:16.218252 10060 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.158s
I20260709 12:26:16.396126  9936 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.459061  9936 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:16.446121  6308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.005s	sys 0.012s
W20260709 12:26:16.399081  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.412869  9884 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.446616  9600 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:16.447604  9765 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:16.446751 10007 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.285840  9753 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.448155  6383 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:16.448717  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:16.393918 10048 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.446346  9446 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.360780 10018 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:16.450433  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.293138  9944 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.450868 10011 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.286159 10029 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.360693  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 151547us
I20260709 12:26:16.447238  9829 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:26:16.465044  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 260928us
W20260709 12:26:16.295183  9842 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.000s
I20260709 12:26:16.395517 10028 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:16.496904  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.563726  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:16.564637  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.522s (SENT) [suppressed 83 similar messages]
I20260709 12:26:16.465540  9976 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:16.474452  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:16.191239 10059 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.286228  9930 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:16.460570 10072 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.454002  9963 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:16.466123  9600 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.492993  9765 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.277866  9980 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.550958  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.659596  6377 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:16.556056  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:16.556910  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:16.493875  9992 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:16.558240  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:16.606935  9976 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.544502 10048 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.551712 10073 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.564865 10060 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:26:16.608516  9753 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.495501  9679 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.555338  9829 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.608953 10019 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.610535 10017 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.536345 10018 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.616183  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 476) took 1454 ms. Trace:
I20260709 12:26:16.654834 10054 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:16.617506  9750 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.065s	user 0.012s	sys 0.000s
I20260709 12:26:16.566627  9842 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.619103  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:16.629508 10029 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.644567 10072 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.585430 10028 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.716187  6456 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:16.575129 10070 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:26:16.723259  9357 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.564533  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:16.629971  6379 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:16.723845  9357 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.725036  6379 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:16.634776  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:16.493574  9884 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.650066  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:16.650589  6450 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:16.652932  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:16.745229  6459 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:26:16.653020  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:16.658725  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:16.746271  7318 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
W20260709 12:26:16.747123  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 272780us
W20260709 12:26:16.660825  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 105039us
I20260709 12:26:16.748215  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:16.748405  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:16.749804  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:16.667598  6380 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:16.628690  9446 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.635672  9930 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:16.638266  9968 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.642287  9964 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.659458  9782 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:26:16.616680 10007 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.693599  6300 rpcz_store.cc:276] 0709 12:26:15.162154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.177743 (+ 15589us) service_pool.cc:224] Handling call
0709 12:26:15.178068 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.526018 (+347950us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:15.526050 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:15.526559 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:26:15.667907 (+141348us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:15.667951 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.615529 (+947578us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54002d19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:16.615607 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.615679 (+    72us) raft_consensus.cc:1685] finished
0709 12:26:16.615703 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.616054 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76,"raft.queue_time_us":865,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":8787,"spinlock_wait_cycles":794970496,"thread_start_us":118685,"threads_started":1,"wal-append.queue_time_us":238133}
W20260709 12:26:16.595124 10063 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.387s	user 0.000s	sys 0.001s
I20260709 12:26:16.642978  9944 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:26:16.719470 10072 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.703787 10054 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.721127 10028 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.689795 10077 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.693262  9753 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:26:16.754586  6452 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:16.750222  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:16.682780  9992 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.726006  9357 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.770817  6385 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:16.716847  9750 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:16.890116  6454 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:16.789851  9446 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:16.890797  9446 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.891703  9446 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:16.892189  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1806) took 1650 ms (client timeout 1648 ms). Trace:
I20260709 12:26:16.892987  9446 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:16.896085  9884 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.854169  9992 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:16.892771  6307 rpcz_store.cc:269] 0709 12:26:15.242441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.254473 (+ 12032us) service_pool.cc:224] Handling call
0709 12:26:16.891914 (+1637441us) spinlock_profiling.cc:243] Waited 702 ms on lock 0x7b54002e8454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:16.892098 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1614763264}
W20260709 12:26:16.773095  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 214883us
I20260709 12:26:16.719542  9842 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:16.773782  9573 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.786929  9829 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.790920  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:16.772008 10027 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:16.753789 10065 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.797533  9753 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:16.797816 10018 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:16.931968 10044 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.302s	user 0.016s	sys 0.017s
I20260709 12:26:16.932531 10018 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:16.932293 10065 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.787004 10075 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:16.934737 10066 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:16.934839 10075 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.936554 10065 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.937227 10075 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:16.842058 10079 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:26:16.842760  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2085) took 1144 ms. Trace:
I20260709 12:26:16.938181  6457 rpcz_store.cc:276] 0709 12:26:15.698072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.698623 (+   551us) service_pool.cc:224] Handling call
0709 12:26:15.698987 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:15.722173 (+ 23186us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:15.747903 (+ 25730us) log.cc:844] Serialized 14 byte log entry
0709 12:26:16.119912 (+372009us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:16.120482 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:26:16.120670 (+   188us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:16.120718 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.120824 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:16.842455 (+721631us) raft_consensus.cc:1685] finished
0709 12:26:16.842501 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:16.842683 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27978,"prepare.queue_time_us":29390,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":42679,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":28744,"thread_start_us":333903,"threads_started":1,"wal-append.queue_time_us":362244}
I20260709 12:26:16.844280 10028 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.848227 10054 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:16.851830 10093 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:16.950763  9573 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.854943 10048 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:16.952044  9573 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.786934 10062 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:16.954972 10062 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:16.956347 10062 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.860173 10074 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.967317  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 418) took 1411 ms. Trace:
I20260709 12:26:16.689832 10073 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.967784  6454 rpcz_store.cc:276] 0709 12:26:15.555807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.555948 (+   141us) service_pool.cc:224] Handling call
0709 12:26:16.967148 (+1411200us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001617d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:16.967252 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2966420224}
I20260709 12:26:16.861383  9328 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:16.969631 10073 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.832250 10002 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:16.970207 10023 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.427s	user 0.013s	sys 0.002s
I20260709 12:26:16.868813  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:16.775868  9977 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:16.876240 10076 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:26:16.877048  9976 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.878196  6308 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:16.769707  9968 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.742144  9600 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.881578 10059 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.765609  9765 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.798446  9819 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:16.839399 10032 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.839167 10077 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.981900  6302 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:16.982834 10032 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.983042 10077 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:16.908879  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:16.983309  9765 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:16.772387  9964 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.983965  9930 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.985451  9964 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:16.926393  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 135580us
I20260709 12:26:16.868815 10029 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.943436 10075 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:16.931928 10027 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:16.943548 10028 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.944396  9679 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.945407  9848 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:16.931797  9753 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:26:16.962049  9829 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:16.966814 10074 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.764940 10046 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 12:26:16.972347  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:17.044421  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 72150us
I20260709 12:26:16.975337  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:16.975906  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:16.970314 10002 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:16.981107 10023 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:16.977527  9819 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:16.989033  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.059118  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.060650  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.061311  6459 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:17.062595  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.063894  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:17.012627  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 103846us
I20260709 12:26:17.073418  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:17.075608  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:17.011281  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.077517  6453 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:17.017155  9848 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:17.033463 10095 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:17.079057 10095 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:17.079300  9848 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.034022  9964 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.080596 10095 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.081019  9964 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.034647 10032 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.083794 10032 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:17.037496  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.046123  6458 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:17.089138  6305 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:17.092134  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:17.093423  6305 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:17.094749  9679 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.095404  9679 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:16.978669  9328 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:16.997284 10059 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:17.140728 10059 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.011402 10018 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:17.143713 10018 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:16.982856  9968 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:16.987877  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:17.147786 10066 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:17.146155 10018 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:17.173264  9328 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.175160  9328 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:17.179217  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.176385  6459 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:17.035617  9976 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.997488  9977 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.223697  6308 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [4.3-7.4]
I20260709 12:26:17.025002 10075 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.051863 10002 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.051644  9404 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:17.277901  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 290066us
I20260709 12:26:17.287652  6310 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:17.289510  6351 leader_election.cc:400] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:26:17.289978  6351 leader_election.cc:403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:26:17.287791  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.043711 10073 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.287956  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 386956us
I20260709 12:26:17.288278  6376 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:17.297163  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:17.299928  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.022s	sys 0.015s
W20260709 12:26:17.300486  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.022s	sys 0.015s
W20260709 12:26:17.305717  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.218684  9819 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.308559  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:17.319285  6305 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:17.314654  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:17.320729  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:17.319159  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.308718  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.322654  7317 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:17.281180  9600 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:17.364250  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.364519  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:17.364449  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44073.4us
I20260709 12:26:17.365422  7318 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:17.365721  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:17.287199 10059 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:17.286598 10072 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:17.286993 10027 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.364250  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.384604  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.488901  7329 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:17.457458 10031 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:17.468096  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:17.393669  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:17.490897  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 125276us
I20260709 12:26:17.444681  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.500622  7332 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:17.500959  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:17.500180  7319 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.501699  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.291427  9842 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.469302  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:17.443436  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 693649us
I20260709 12:26:17.506829  7318 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
W20260709 12:26:17.496999  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 207584us
I20260709 12:26:17.291707 10032 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:17.507138  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:17.387614  7331 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:17.394557  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:17.510859  6458 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 9
W20260709 12:26:17.410451  6377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.008s	sys 0.025s
W20260709 12:26:17.512075  6377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.010s	sys 0.025s
W20260709 12:26:17.516441  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.415869  9980 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.000s
W20260709 12:26:17.484723  9446 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.591s	user 0.011s	sys 0.002s
I20260709 12:26:17.534556  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:17.517561  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.517720  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:17.365569 10037 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.821s	user 0.010s	sys 0.010s
I20260709 12:26:17.445539  9930 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:17.520146  9977 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:17.464507 10066 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:17.524869  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.525542  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 437) took 1356 ms. Trace:
I20260709 12:26:17.291721  9976 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.526522  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:17.581498  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 64065.6us
I20260709 12:26:17.582679  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:17.527180 10109 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.143s	user 0.002s	sys 0.000s
I20260709 12:26:17.416045 10102 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:17.504340 10077 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:17.416234 10054 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.528362 10035 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:17.324414 10062 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:26:17.367223  9765 raft_consensus.cc:2764] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:17.579712  9977 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.428161  9964 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:17.324826 10094 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:17.479915 10075 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:17.537047  6451 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:17.500094 10028 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:17.537529  6381 rpcz_store.cc:276] 0709 12:26:16.169033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.724469 (+555436us) service_pool.cc:224] Handling call
0709 12:26:16.724854 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:16.725271 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:16.725304 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:16.725874 (+   570us) log.cc:844] Serialized 52 byte log entry
0709 12:26:17.059798 (+333924us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:17.236142 (+176344us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.236423 (+   281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.236521 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:17.525341 (+288820us) raft_consensus.cc:1685] finished
0709 12:26:17.525361 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.525488 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15530,"prepare.queue_time_us":174684,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":20064,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":46228,"spinlock_wait_cycles":46464,"thread_start_us":333436,"threads_started":1,"wal-append.queue_time_us":744927}
I20260709 12:26:17.602677 10005 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:17.579149  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1926) took 1410 ms. Trace:
I20260709 12:26:17.384974  9884 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:17.625835  6377 rpcz_store.cc:276] 0709 12:26:16.168621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.680256 (+511635us) service_pool.cc:224] Handling call
0709 12:26:16.690932 (+ 10676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.691099 (+   167us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:16.908859 (+217760us) log.cc:844] Serialized 14 byte log entry
0709 12:26:17.578398 (+669539us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:17.578500 (+   102us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:17.578584 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.578691 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:17.579049 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4342,"prepare.queue_time_us":89028,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":73504,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":19701,"thread_start_us":817864,"threads_started":2,"wal-append.queue_time_us":710177}
I20260709 12:26:17.584069  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:17.632982  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49014.3us
W20260709 12:26:17.601668  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.609787  6307 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:17.370453 10073 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.636622  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:17.386591 10048 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:17.576584  9404 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:17.509902 10072 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.518081  9842 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.589416 10102 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.620373  6458 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:26:17.621767  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95353.7us
I20260709 12:26:17.756254  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:17.756568  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.757557  6458 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:17.589008 10077 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:17.759958 10077 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:17.617868  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 153015us
I20260709 12:26:17.576506  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.762637  6458 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:17.519157 10059 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.576268  9980 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:17.602568 10023 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.654213  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.747172  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:17.776742  6451 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:17.777469  6455 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:17.748698  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:17.624565 10094 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.639178 10062 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:17.655136  9573 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.018s	sys 0.047s
W20260709 12:26:17.796857  9573 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.018s	sys 0.047s
I20260709 12:26:17.798759 10007 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:17.799607 10007 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:17.800873 10007 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:17.807337 10114 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:17.808600  9980 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:17.810316  9976 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:17.814380 10072 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:17.817695  9829 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.000s
I20260709 12:26:17.825083 10073 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:17.747803 10035 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:17.828614 10035 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:17.761713  9446 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:17.770480  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 263403us
I20260709 12:26:17.642417  9884 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.833323  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:17.610469 10054 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.835330  7314 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:17.835747  9884 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:17.618391 10002 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.836587 10054 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:17.782371  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:17.848109  6304 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:17.647269 10048 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.853098 10048 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:17.788525  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:17.743881 10046 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:26:17.762549 10024 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:17.861469 10024 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.762086 10005 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.101s
I20260709 12:26:17.864042 10005 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:17.864493 10005 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:17.761962 10102 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.866415  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:17.867796  6303 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:17.789429 10105 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.201s	user 0.000s	sys 0.000s
I20260709 12:26:17.789525  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:17.791069 10115 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:26:17.794937  9404 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:17.782269  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:17.913820  6310 raft_consensus.cc:2376] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7. Re-sending same reply.
I20260709 12:26:17.845204  6305 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:17.846094 10093 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.898s	user 0.013s	sys 0.000s
I20260709 12:26:17.869953  6304 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:17.871220  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:17.900108  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.886329  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:17.903467  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.922102  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.885648  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:17.923378  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.924479  6380 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:17.920200  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.925072  6382 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:17.921885  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:17.926126  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.926811  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:17.927126  6379 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:17.928988  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:17.930266  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:17.932550  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61468.3us
I20260709 12:26:17.934333  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:17.937879  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.939208  6380 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:17.942145  6300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.004s	sys 0.023s
I20260709 12:26:17.942780 10107 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:17.942879  6300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.005s	sys 0.023s
I20260709 12:26:17.912359  7319 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:26:17.945721  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 163616us
I20260709 12:26:17.949713  6379 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:17.950695  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:17.952230  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.958376  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.957413  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.959972  6381 raft_consensus.cc:2368] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:26:17.963759  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:17.964371  6377 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:17.965912  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:17.962658  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:17.979126  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.829835 10035 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.979650  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.919469  9573 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:17.936102 10046 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.865526 10005 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.985607  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:17.779775 10059 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:17.981323  6385 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:17.962741  9980 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:26:17.953893 10028 raft_consensus.cc:2764] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:17.943415 10107 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.982088  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:18.000765  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.945073 10095 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:17.992311  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:17.992506  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:17.956643  9829 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:17.996001  6382 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:17.997448  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:18.009622  6377 raft_consensus.cc:2368] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:26:17.997565  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:17.940876 10093 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:18.000988  6380 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:17.951762 10054 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:17.984611  9573 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:18.004921 10107 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.005336 10110 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:18.005728 10106 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.008111  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:18.003284  6379 raft_consensus.cc:2376] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13. Re-sending same reply.
I20260709 12:26:17.943719  9930 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:26:18.004499  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.019376  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:18.020696  6381 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.019383  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.022713  6377 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:18.023840  7329 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:26:18.025033  6459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.025s	sys 0.041s
W20260709 12:26:18.025535  6459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.025s	sys 0.041s
I20260709 12:26:18.026674  6457 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:18.013618 10024 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:18.031433  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.032895  6380 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:18.015859 10054 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.021646  9573 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:18.034873  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:18.069208  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:18.023528 10106 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.023391 10110 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:17.996351 10046 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.030362 10005 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.083554  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.084563  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:18.068739 10054 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.086766  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:18.091306  6300 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:18.081924 10110 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:18.081301 10127 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.094130  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:18.079169 10106 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.099802  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.082948  9783 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:18.083627  9829 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:18.082381 10024 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:18.107419  9328 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.930s	user 0.013s	sys 0.000s
I20260709 12:26:18.110930  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.128878  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:18.115005  9819 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.731s	user 0.013s	sys 0.000s
W20260709 12:26:18.093108  9753 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.059s	user 0.014s	sys 0.000s
I20260709 12:26:18.118613  9829 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.089169 10002 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.137666  6453 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:18.128023  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.138616  6382 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:18.127488  6385 raft_consensus.cc:2410] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.139704  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:18.128531  6383 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:18.129675  6451 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:18.130556  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:18.141391  6274 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.522s (SENT)
I20260709 12:26:18.142046  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.101389  9936 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.140331  6381 raft_consensus.cc:2376] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9. Re-sending same reply.
I20260709 12:26:18.104471 10127 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.343632  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.342936  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:18.312672 10133 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:26:18.220953  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.285245  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:18.352723  6377 raft_consensus.cc:2376] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11. Re-sending same reply.
I20260709 12:26:18.137354  9930 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:18.194610  6424 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.648s (SENT)
I20260709 12:26:18.354746  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:18.212932  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:18.349169  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.211479  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:18.355309  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.355786  6385 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:18.350720  6301 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:18.170310 10027 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.153736  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:18.257570  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87893.1us
I20260709 12:26:18.133052 10024 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.210947  9884 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.375216  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.129112  9783 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.376410  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.384938  6382 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:18.357049  6380 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:18.335119 10094 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.312670 10093 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.229717  9328 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.169082  9976 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:18.140245  9819 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.139355  9753 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.358222  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:18.370157  9600 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.966s	user 0.011s	sys 0.002s
I20260709 12:26:18.356057  6381 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:18.388737  6379 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:26:18.390789  6274 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.604s (SENT)
W20260709 12:26:18.370659 10077 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.600s	user 0.013s	sys 0.000s
I20260709 12:26:18.244619 10059 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.392500  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:18.392613  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 198097us
I20260709 12:26:18.395609  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.286924 10048 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.400312  6450 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:18.401002  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2582) took 1172 ms. Trace:
I20260709 12:26:18.405755  6378 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:26:18.405794  6459 rpcz_store.cc:276] 0709 12:26:17.228316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.282339 (+ 54023us) service_pool.cc:224] Handling call
0709 12:26:17.282689 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.283021 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.283048 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.302377 (+ 19329us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.025977 (+723600us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:18.026042 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.026119 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.400754 (+374635us) raft_consensus.cc:1685] finished
0709 12:26:18.400778 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:18.400931 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":533,"spinlock_wait_cycles":13813120,"thread_start_us":598560,"threads_started":1,"wal-append.queue_time_us":749180}
I20260709 12:26:18.407251  9819 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.370290  9930 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.408659 10127 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.409837  9930 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:18.415598  6274 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.641s (SENT)
I20260709 12:26:18.402745  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.417519  6381 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:18.417613  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.404847  6385 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:18.419934  6380 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:18.437175  6454 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:26:18.385752 10027 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:18.652319  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 212948us
I20260709 12:26:18.653158  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.396764 10094 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:18.445189  6385 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:18.653596  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.394699 10093 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:18.661296  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 244663us
I20260709 12:26:18.416167 10095 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:18.663537  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.673106  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:18.428378 10046 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.396412  9328 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.434918  9600 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.652064 10059 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.688694  6377 raft_consensus.cc:2410] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:26:18.415064 10077 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.400185  9976 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.674165  6380 raft_consensus.cc:2376] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11. Re-sending same reply.
I20260709 12:26:18.663262  9929 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:18.656103 10127 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.675091  9404 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:18.675943  9829 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.673677  9819 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:18.680217 10095 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.718816 10094 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:18.714627  9929 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:18.704695  9985 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:18.702522  9600 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.704083 10059 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:18.759631 10102 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.893s	user 0.010s	sys 0.002s
I20260709 12:26:18.728127 10127 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.793926 10127 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.758880  9446 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.794864  9446 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.797416  9446 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:18.800527 10135 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:26:18.802510 10138 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:18.780275 10059 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.781025  9929 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.733747  9819 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.787675  9976 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.754179 10094 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:18.789624 10054 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.696s	user 0.010s	sys 0.001s
I20260709 12:26:18.791580  9600 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:18.819537  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 150349us
W20260709 12:26:18.770987 10137 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.386s	user 0.000s	sys 0.001s
I20260709 12:26:18.829792 10093 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.830615 10094 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.828191  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:18.828562  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:18.831825 10144 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.833268  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:18.834084 10144 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:18.819635 10102 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.834468  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.834892  9753 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.835304 10144 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:18.835726  7318 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:18.836215 10102 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.836483 10144 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:18.836601  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.837524 10007 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:18.838750 10007 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:18.838922  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.841512 10007 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:18.843370 10077 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:18.845115 10107 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.816s	user 0.011s	sys 0.000s
I20260709 12:26:18.845587 10107 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.847049 10048 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.848129  9328 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.851194  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.851886  6458 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:18.852824  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.853819  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.853067  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.854413  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:26:18.831071 10141 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:18.855293  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.858321  6458 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:18.859123  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:18.859714  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:18.861894  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.855563  9819 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.865986  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:18.867429  9573 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.786s	user 0.010s	sys 0.001s
I20260709 12:26:18.867624  9980 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.868018  9573 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.868222  9980 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.871361  6455 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:18.860143 10106 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.729s	user 0.004s	sys 0.006s
I20260709 12:26:18.878847  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:18.857084  9963 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:18.889565  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:18.866003  9929 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.907261  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:18.912611  9985 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:18.913477  9985 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:18.913890  9985 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.915969  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2572) took 1926 ms. Trace:
I20260709 12:26:18.916389  6450 rpcz_store.cc:276] 0709 12:26:16.989862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.989976 (+   114us) service_pool.cc:224] Handling call
0709 12:26:16.990239 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.405517 (+1415278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.915700 (+510183us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54002c8b54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:18.915904 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3242446720}
I20260709 12:26:18.921559  9944 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:18.922013  9944 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:18.923136 10145 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.922904  9944 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:18.923635 10145 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.924346  9944 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:18.925869 10145 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.927558  9980 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:18.928944 10074 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.914254  9985 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:18.929395  9846 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:18.858042  9976 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:26:18.827219 10054 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:18.931432  6455 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:18.947021  7319 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:18.887146 10106 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:18.953083  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 93447.9us
I20260709 12:26:18.954022  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.955257 10107 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:18.955950  9573 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:18.956218 10121 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.964s	user 0.000s	sys 0.010s
W20260709 12:26:18.958076  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1831) took 1866 ms (client timeout 1795 ms). Trace:
I20260709 12:26:18.959759 10054 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:18.950538 10005 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.861s	user 0.009s	sys 0.005s
I20260709 12:26:18.968317 10106 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:18.980564  6307 rpcz_store.cc:269] 0709 12:26:17.091536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.091718 (+   182us) service_pool.cc:224] Handling call
0709 12:26:18.957779 (+1866061us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b540025add4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:18.957981 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186468096}
W20260709 12:26:18.986850  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124995us
I20260709 12:26:18.991604  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:18.987413  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:18.986850  9931 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:18.995368  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:18.998979 10151 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.010954  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 743) took 1712 ms. Trace:
I20260709 12:26:19.011480  7320 rpcz_store.cc:276] 0709 12:26:17.298603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.298748 (+   145us) service_pool.cc:224] Handling call
0709 12:26:17.366870 (+ 68122us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.398573 (+ 31703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.398611 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.399310 (+   699us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.613079 (+213769us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:17.613122 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.613197 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:18.955454 (+1342257us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:19.010591 (+ 55137us) raft_consensus.cc:1685] finished
0709 12:26:19.010626 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.010876 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31142,"raft.queue_time_us":49083,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":16026,"spinlock_wait_cycles":19034240,"thread_start_us":194830,"threads_started":1,"wal-append.queue_time_us":500812}
I20260709 12:26:18.993570  9976 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:19.014925  9976 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.017102  9976 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:19.016957  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.017772 10054 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:19.018520  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:19.018867 10054 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:19.019091  6450 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.019270 10054 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:19.019482  9679 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.020371  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.020140 10054 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.021399  6455 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:19.023483  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1918) took 2885 ms. Trace:
W20260709 12:26:18.994242 10110 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.890s	user 0.009s	sys 0.003s
I20260709 12:26:19.023900  6376 rpcz_store.cc:276] 0709 12:26:16.137987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.191260 (+ 53273us) service_pool.cc:224] Handling call
0709 12:26:16.191564 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:17.296734 (+1105170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.023240 (+1726506us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:19.023422 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521830144}
I20260709 12:26:19.027252  6455 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:19.028023  6450 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
W20260709 12:26:19.035316  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 484) took 2947 ms (client timeout 1805 ms). Trace:
W20260709 12:26:19.039283  6302 rpcz_store.cc:269] 0709 12:26:16.088193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.088324 (+   131us) service_pool.cc:224] Handling call
0709 12:26:19.035093 (+2946769us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54002de6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:19.035233 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965827840}
I20260709 12:26:19.022172 10054 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.001037 10151 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.045398 10151 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.025134  9928 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:19.047197  9928 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.047575  9928 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.025002  9884 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:19.050740  9944 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:19.051539  9944 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.052002  9985 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.053874  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:19.058660  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:19.063702  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 72100.2us
I20260709 12:26:19.064577  9978 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:19.065315  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.065658 10150 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:19.075469 10150 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:19.075899 10150 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.076201  9985 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.076558 10152 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:19.077018  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:19.077139 10152 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:19.077299  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.067546  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.067806  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.078351  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.069487  9842 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:19.069500 10031 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:19.078580 10152 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:19.071179  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2583) took 1763 ms (client timeout 2146 ms). Trace:
W20260709 12:26:19.086771  6454 rpcz_store.cc:269] 0709 12:26:17.308270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.308441 (+   171us) service_pool.cc:224] Handling call
0709 12:26:19.070992 (+1762551us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54002c8654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:19.071118 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2562311424}
I20260709 12:26:19.088490  6451 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:19.088855  9978 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:19.050892  9884 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.074759  7332 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:19.091032  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1830) took 1999 ms. Trace:
I20260709 12:26:19.091665  6308 rpcz_store.cc:276] 0709 12:26:17.091212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.092343 (+  1131us) service_pool.cc:224] Handling call
0709 12:26:17.092680 (+   337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:17.284970 (+192290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.090697 (+1805727us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400206cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:19.090949 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295537920}
I20260709 12:26:19.074463  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:19.068262  7331 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:19.080083  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.080618  6455 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:26:19.081693 10057 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:19.070392  9931 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:19.082952  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.085104 10150 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.085829  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.098869  9985 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.106281 10108 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:19.100111  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:19.103614  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.104158  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.140136  6455 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:26:19.141023  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.141976  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:19.090027  9978 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.144224  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.144482  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.145159  6385 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:19.145965  6274 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:19.097525  9884 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:19.097322  9963 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:19.059150 10110 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.143164  6381 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:26:19.142525  6376 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:19.147465  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:19.151885  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.187539  6379 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.153301  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.155364  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:19.158463 10125 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.092s	user 0.001s	sys 0.004s
I20260709 12:26:19.159211  6380 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:19.159483  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.160836  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 521) took 1275 ms. Trace:
I20260709 12:26:19.211354  7318 rpcz_store.cc:276] 0709 12:26:17.885094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.885265 (+   171us) service_pool.cc:224] Handling call
0709 12:26:19.160633 (+1275368us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:19.160771 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2102558080}
W20260709 12:26:19.161859  6457 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 421) took 2189 ms (client timeout 1966 ms). Trace:
I20260709 12:26:19.158476  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.166504  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:19.109969 10150 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.189110  6382 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:19.199112  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:19.212787  6457 rpcz_store.cc:269] 0709 12:26:16.972736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.975698 (+  2962us) service_pool.cc:224] Handling call
0709 12:26:19.161661 (+2185963us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:19.161769 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254786944}
W20260709 12:26:19.268289  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101928us
W20260709 12:26:19.214540  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67190.1us
W20260709 12:26:19.259310  6453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2587) took 1768 ms (client timeout 1626 ms). Trace:
I20260709 12:26:19.265354  6385 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:26:19.296964  6453 rpcz_store.cc:269] 0709 12:26:17.490777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.501422 (+ 10645us) service_pool.cc:224] Handling call
0709 12:26:19.259014 (+1757592us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54001c1454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:19.259200 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460150016}
I20260709 12:26:19.298367  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.153462 10152 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:19.269970  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:19.300541  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.145287 10151 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.302516  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.306515  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.307524  6455 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:19.308588  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:19.309247  6455 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.310580 10110 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:19.211017  9978 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:19.313421  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:19.318184  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:19.295002  9976 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:19.323762  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.324290  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.273567  9963 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:19.137806 10057 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.298906  9782 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.324705  6382 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:19.326128  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.326825  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.317523 10150 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:19.327634  9404 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:19.327950  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.328145  6455 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:19.335536  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 422) took 2359 ms. Trace:
I20260709 12:26:19.335536  6380 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:19.335992  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.336639  6455 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:19.336212  6452 rpcz_store.cc:276] 0709 12:26:16.976445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.980751 (+  4306us) service_pool.cc:224] Handling call
0709 12:26:17.012340 (+ 31589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:17.012722 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:17.012752 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:17.277974 (+265222us) log.cc:844] Serialized 26 byte log entry
0709 12:26:17.755070 (+477096us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:17.755169 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.755307 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.157864 (+1402557us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:19.330324 (+172460us) raft_consensus.cc:1685] finished
0709 12:26:19.330343 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.335445 (+  5102us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":303,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":19302,"spinlock_wait_cycles":4086528,"thread_start_us":476545,"threads_started":1,"wal-append.queue_time_us":738048}
I20260709 12:26:19.301560 10152 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:19.326978  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.329022  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:19.308977  9884 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.325630  9976 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.322760  9985 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:19.341428  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2562) took 3541 ms. Trace:
I20260709 12:26:19.350757  6456 rpcz_store.cc:276] 0709 12:26:15.800243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.800376 (+   133us) service_pool.cc:224] Handling call
0709 12:26:15.835595 (+ 35219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.716693 (+881098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.341184 (+2624491us) spinlock_profiling.cc:243] Waited 880 ms on lock 0x7b54001e2dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:19.341368 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2024711040}
I20260709 12:26:19.331759 10074 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:26:19.353211 10074 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:19.353582 10074 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:19.333768 10150 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.345605  9976 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.336876  9944 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.365697  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.370817  6378 raft_consensus.cc:2376] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10. Re-sending same reply.
I20260709 12:26:19.361718  6452 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:19.366336  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.342975 10152 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:19.368443  6456 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:19.371878  6385 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:19.347249  9980 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:19.331007  9928 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.376179  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.351643  9985 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.377799  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.379069  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.379608  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.380403  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.383924  6379 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:19.366636 10150 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.419925  9985 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.420773  6380 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:19.420069  6381 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:19.422624  9944 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:19.423195  9944 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.425563  9944 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.427040  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.384672  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.429497  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.430439  6378 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:19.430601  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.430965  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.433422  6379 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:19.417443  9980 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:19.441234  9783 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:19.447121 10157 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 12:26:19.452925 10159 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:26:19.455204  6455 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:19.457660 10127 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.514984  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:19.458626 10129 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.430s	user 0.000s	sys 0.012s
I20260709 12:26:19.465528  6384 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:19.466290  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:19.473120 10165 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:19.474745  9928 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:19.352227  9963 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.476898 10095 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:19.477246 10074 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:19.555989 10164 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:26:19.556191 10074 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.478144  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:19.559250 10057 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:19.559378  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 81324.3us
I20260709 12:26:19.560500  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:19.485983  9976 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.488564 10027 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.494928  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:19.578126  6457 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.497817  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:19.579766  6459 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:19.499734  6458 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:19.580569  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.581501  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:19.531139  6379 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:19.554384  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.558310 10024 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:19.562590  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.564347  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.585563  6378 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.570176 10004 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:19.559904 10057 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.566234  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:19.573055  6456 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:19.573227  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:19.588723  6452 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:19.517501  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.584156  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.590103  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:19.590756  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:19.593040  6301 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:19.591915  6307 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:19.595022  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.595579  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.595800  7319 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:19.596976  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:19.597560  6384 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.598713  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.599294  7319 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:19.601153  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.601637  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.602391  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.602571  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.602922  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:19.603060  6301 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:19.603672  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.604019  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.604245 10027 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:19.604398  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2434) took 1247 ms. Trace:
I20260709 12:26:19.604748  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:19.605471  9980 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:19.606739  6307 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:19.607366  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 457651us
I20260709 12:26:19.607261 10152 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.608235  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:19.603942  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.604883  6377 rpcz_store.cc:276] 0709 12:26:18.357001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.357917 (+   916us) service_pool.cc:224] Handling call
0709 12:26:19.604211 (+1246294us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54003d3bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:19.604337 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664459008}
I20260709 12:26:19.610533  6307 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:26:19.611713  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.612404  7319 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:19.605504  7329 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:19.615713  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:19.617622  6457 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:19.624876  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:19.543473 10127 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.629496  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.630779  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.633841  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:19.634912  7314 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:19.628818  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:19.639402  7318 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:19.633391  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:26:19.669564  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.675222  7320 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:19.677789  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44474.6us
I20260709 12:26:19.678606  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:19.687671  6378 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:19.691910 10110 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:19.695056 10110 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:19.695547 10110 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:19.677414 10157 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:26:19.708413 10150 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.709307  9848 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:19.710646 10150 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:19.692324  9884 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.702423 10145 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:19.742633 10152 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:19.750053 10127 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:19.748355  9976 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.758059  9985 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:19.604701  6301 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:19.861918  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.862259  9985 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:19.864025 10004 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.811527  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 245178us
I20260709 12:26:19.815109  6307 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:19.815248  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:19.824088  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 145451us
W20260709 12:26:19.836629  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.826280  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:19.844262  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.916791  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.927326  6309 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:19.924765  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 331001us
I20260709 12:26:19.846216  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.946522  6305 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:19.952744  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:19.955363  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.958267  6310 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
W20260709 12:26:19.958823  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 133155us
I20260709 12:26:19.865232 10004 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:19.959578  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:19.960636  6310 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:19.960561  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:19.963171 10057 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.963209  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:19.967237 10169 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:19.967857 10169 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.968250 10169 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:19.962049 10004 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:19.979508  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:19.990638  9782 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:19.991895  9782 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:26:19.999393  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.999783  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.001049  6305 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.003103  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.004750  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.005934  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.006047  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 736) took 3375 ms. Trace:
I20260709 12:26:20.008405  6309 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 5: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.009460  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.011263  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.002199  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:20.012189  6300 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 5.
I20260709 12:26:20.015022  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.015666  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.016419  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.017087  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.017815  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.017995  6310 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
W20260709 12:26:20.024362  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54756.2us
I20260709 12:26:20.010352  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:20.006525  7335 rpcz_store.cc:276] 0709 12:26:16.630580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.630789 (+   209us) service_pool.cc:224] Handling call
0709 12:26:16.662361 (+ 31572us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:16.723066 (+ 60705us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:16.723420 (+   354us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.856701 (+2133281us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b18004b8ce0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a
0709 12:26:18.856893 (+   192us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.865512 (+  8619us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.985078 (+119566us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b18004b8ce0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:19.105819 (+120741us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:19.105864 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.005407 (+899543us) spinlock_profiling.cc:243] Waited 60.1 ms on lock 0x7b540016dd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:20.005484 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.005558 (+    74us) raft_consensus.cc:1685] finished
0709 12:26:20.005568 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.005811 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3076,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":11168,"raft.queue_time_us":1021,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":5060,"spinlock_wait_cycles":155012736,"thread_start_us":165261,"threads_started":2,"wal-append.queue_time_us":352073,"wal-append.run_cpu_time_us":1411,"wal-append.run_wall_time_us":1021239}
I20260709 12:26:20.045406  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:20.050916  9980 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.048213  6300 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
W20260709 12:26:20.054993  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.032105  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:20.067602  6303 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:20.068696  6301 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:20.089210  7329 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
W20260709 12:26:20.103704  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 131717us
I20260709 12:26:20.102830  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:20.108840  6309 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.118822  6307 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:20.128754 10110 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.129508  7335 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:20.130784 10127 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:20.131487 10110 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.132793  7320 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:26:20.135018  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 89792.5us
I20260709 12:26:20.135438  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:20.137189  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 528905us
I20260709 12:26:20.137336  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:20.138219  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:20.142782  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 163400us
I20260709 12:26:20.143643  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:20.146927 10110 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:20.147380 10110 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.147787 10110 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:20.150971  9573 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:20.154117  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79036.6us
I20260709 12:26:20.157490  9679 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.158071  9679 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.159811  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:20.160467  7317 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:20.161648 10151 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:20.162127 10151 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:20.162546 10151 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.168128  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:20.158423  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:20.169811  9980 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:20.170374 10074 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:20.171455  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:20.173373  7318 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:20.175450  7320 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:20.176506  7335 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:20.177201 10168 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:20.136180  6307 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.146252  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:20.180817  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:20.199704  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53605.5us
I20260709 12:26:20.200840 10127 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:20.201588 10127 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.201741 10004 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:20.202042 10127 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.202374  6304 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:20.202651 10004 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.205533 10004 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.189837 10168 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:20.207446 10168 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:20.212456  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 252184us
W20260709 12:26:20.217286  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 90210.2us
I20260709 12:26:20.216797  6302 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:20.219710  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48487.5us
I20260709 12:26:20.221596  9997 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:20.222102  9997 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.222548  9997 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:20.222095  9976 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:20.227563 10074 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:20.231889  9976 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:20.232355  9976 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:20.233067  6452 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.638s	user 0.014s	sys 0.002s
I20260709 12:26:20.233639  6452 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:20.236510 10095 raft_consensus.cc:2764] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:20.239869 10151 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:20.241353  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.242581  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.242242 10151 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.252096  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:20.256625  7318 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:20.229681 10074 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.261705  6348 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:20.254211  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.238826  9976 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.267035  7314 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.269574 10018 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.272095 10018 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.275014 10181 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:26:20.280947  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:20.294346  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 113773us
I20260709 12:26:20.297214  7314 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:20.297013 10181 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:20.302403 10004 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:20.303689 10151 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:20.304162 10151 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.304540 10151 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.282956 10018 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.302450 10180 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:20.317550 10180 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:20.318116  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:20.314054  9976 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.319567  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:20.324669  6308 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:26:20.320536  9679 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.339004 10179 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 12:26:20.340960  9928 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:20.347729  9980 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:20.348026  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:20.349550 10170 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:20.350232 10170 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:20.318768 10180 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.359385  9963 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:20.356168 10095 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.361832 10095 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.383486  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.384473  7316 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:20.386754  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.387763  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2101) took 1351 ms. Trace:
I20260709 12:26:20.388196  6455 rpcz_store.cc:276] 0709 12:26:19.036018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.337528 (+301510us) service_pool.cc:224] Handling call
0709 12:26:19.362881 (+ 25353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.504494 (+141613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.387283 (+882789us) spinlock_profiling.cc:243] Waited 91.8 ms on lock 0x7b54001626d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:20.387552 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211248512}
I20260709 12:26:20.377480 10170 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.397693  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 754) took 1332 ms. Trace:
I20260709 12:26:20.384306 10095 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:20.398890  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 152474us
I20260709 12:26:20.400341 10057 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:20.401842  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:20.403002 10181 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:20.403743 10181 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:20.405638 10181 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.406284 10180 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.380539  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:20.410640  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55395.1us
I20260709 12:26:20.410327 10004 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:20.412017 10004 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.412436 10004 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:20.414646 10005 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:20.415249 10005 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:20.371590 10169 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.416353 10005 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.417130  6305 raft_consensus.cc:2368] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14.
I20260709 12:26:20.417797 10005 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.419991  9963 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:20.420549  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.420657  9963 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:20.421284  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.421649  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.422823  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51458.6us
I20260709 12:26:20.398145  7317 rpcz_store.cc:276] 0709 12:26:19.065227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.067239 (+  2012us) service_pool.cc:224] Handling call
0709 12:26:20.397444 (+1330205us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400439c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:20.397593 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487134464}
I20260709 12:26:20.437029  9985 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:20.438148  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.353858 10110 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.445575 10127 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.450399  9573 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:20.451121  9573 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.459262  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:20.459998  6457 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:20.461863  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:20.462260 10018 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.463318  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.463146 10018 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.464061  6455 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:20.441926  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 188939us
I20260709 12:26:20.466413 10018 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.467818  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:20.468431  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.472430  6457 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:20.473516  9985 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:20.474092  9783 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:20.474668 10018 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:20.475750 10018 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:20.479270  9782 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:20.480757  9985 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:20.482184  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.484192  6302 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:20.486567 10095 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:20.474174  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:20.489573  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.491829  6452 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:20.510212  6455 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:20.492166  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.493058  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:20.494449  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:20.496048  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:20.503595  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.504873  7320 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:20.512010  7316 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:20.512977  6300 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:20.511480  7332 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
W20260709 12:26:20.513103  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 111419us
I20260709 12:26:20.517952  7317 raft_consensus.cc:2368] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9.
I20260709 12:26:20.515419  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.519915  7335 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:20.520761  6305 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:20.519747  6204 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:20.536356  6454 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.540104  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.520193 10151 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.508759  9985 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:20.543175  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:20.515992 10095 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.540844  6455 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:20.529032 10110 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.568524 10004 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.570753 10074 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:20.571725 10074 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:20.572794 10074 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:20.574291  7287 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.548s (SENT)
I20260709 12:26:20.439986 10168 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:20.547714  6274 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.216s (SENT)
I20260709 12:26:20.521132  9997 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.551875 10169 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.530761 10018 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.556882 10095 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:20.557250  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.551072  9985 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.533663  9928 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:20.585862  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:20.579206  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:20.562145 10127 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.590396  7335 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:20.589424  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:20.615331  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.603782  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.622911  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:20.600538  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85085.3us
I20260709 12:26:20.623087  6305 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:20.627877  6454 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:26:20.629952  6455 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:20.636318  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:20.639513  6457 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:20.640722  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 93061.9us
I20260709 12:26:20.593179 10004 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.635622  9928 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:20.644965  9928 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.647082  9848 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.647652  9848 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:20.648725  6351 connection.cc:256] RPC call timeout handler was delayed by -1.1609637939995991s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
I20260709 12:26:20.650012  9848 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.650784  6204 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 7, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 7 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:20.651794  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.651979  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.652596  7316 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.652518  6204 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 6 to 7, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:20.653528  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.654114  7317 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:20.654839  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:20.655261  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.663659  9997 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.664261 10168 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.664222  6454 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:20.668359  9573 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:20.670756  9782 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:20.670851 10004 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:20.671543  9782 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.671798 10004 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.673764  9782 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.674011 10004 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.683107  7335 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:26:20.611066 10095 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:20.589932  9976 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:20.631493 10169 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:20.633589 10005 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:26:20.695751  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 117016us
I20260709 12:26:20.697124  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:20.699220  7331 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:20.703085  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63392.9us
I20260709 12:26:20.705075  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:20.705269  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.705853  6309 raft_consensus.cc:2368] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9.
I20260709 12:26:20.706992  6310 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:20.706911  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:20.670444  9573 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:20.714906  6303 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:26:20.715063  6457 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
I20260709 12:26:20.697976 10110 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.725227  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:20.726838  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.692651 10095 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:20.767025  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55343.3us
I20260709 12:26:20.699528 10181 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.711160 10004 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.721263  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:20.790886  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.793313  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.793716  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.794023  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:20.793366  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:20.804070  6456 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:20.805002  6305 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:20.804174  7317 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:20.715826  9976 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:20.814710  6454 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:20.826665  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:20.979887  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.877625  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.815609  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 99280.6us
I20260709 12:26:20.954809  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:20.884653  6451 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:21.167564  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 460541us
W20260709 12:26:20.897259  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.884913  6453 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:20.869567  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70945us
W20260709 12:26:21.205914  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.799981 10005 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.885272  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:21.012279  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:20.894944  7287 leader_election.cc:336] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.785s (SENT)
I20260709 12:26:20.972353  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:20.828514  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:20.885720  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:20.827853  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:21.199932  9985 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:20.886453  9782 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.734s
W20260709 12:26:21.620529  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 299498us
I20260709 12:26:20.897015 10095 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:21.324874  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.364868  9976 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:20.827267  6384 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:21.323870 10027 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:21.222824  7331 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.521s	user 0.006s	sys 0.006s
I20260709 12:26:21.623456  7331 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:21.623291 10027 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.592388  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:20.972831  9963 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.002s
I20260709 12:26:20.897470 10151 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:21.625026  9963 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:21.475468 10207 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.233s	user 0.001s	sys 0.001s
I20260709 12:26:21.627205  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:20.974663 10127 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.628386  6459 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:20.821712  6204 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:20.870411  6450 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:21.619227  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 349329us
W20260709 12:26:21.177542  6206 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 270) took 29726 ms (client timeout 15000 ms). Trace:
I20260709 12:26:20.947876 10181 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.620128  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.624876  6377 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:21.246608  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.478479  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:21.650833  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.627815  6382 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:20.885120 10180 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:20.887528 10057 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:21.625592 10027 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:21.622408  6383 raft_consensus.cc:2410] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:20.798903 10110 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:21.644343  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.327359 10035 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:21.652019  6378 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:21.537472  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:21.331029 10205 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:26:21.652498  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:21.654613  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:21.136593 10200 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
W20260709 12:26:20.975697 10197 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 12:26:21.663833  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:21.665323  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 957608us
I20260709 12:26:20.853175  9679 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:21.661175  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.662056  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:21.678279 10027 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:21.628923 10127 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.678444  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.256003  9573 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.627837  9963 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:21.689848  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68315us
I20260709 12:26:21.658305 10095 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.815339 10004 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.697391  6452 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:21.692446  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:21.699276  6454 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:21.685871  6424 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.205s (SENT) [suppressed 39 similar messages]
I20260709 12:26:21.694641  6377 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:21.696561  6384 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.702078  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.705160  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:21.684973  9679 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.706619  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.706799  6456 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:21.708027  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 180950us
I20260709 12:26:21.658910 10005 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:21.661659 10180 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.713382  6380 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:21.715546  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.718904  6385 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 6.
I20260709 12:26:21.685339  9928 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:21.725746  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46425.5us
I20260709 12:26:21.710754  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 6 to 7, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:21.728299  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45452.5us
I20260709 12:26:21.705895  9963 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.708045  9679 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:21.732226  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:21.681417 10035 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:21.694274  9573 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.737062  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:21.694419 10057 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.695482 10027 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:21.747849  6378 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:26:21.721597 10215 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:21.704196 10095 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:21.757726  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 862820us
W20260709 12:26:21.771406  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.739758  6204 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 4 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 6 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:21.776528  6455 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:21.777020 10212 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:21.777196 10127 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:21.777619 10212 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.779374 10057 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:21.779824 10212 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:21.780644  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:21.781383  6458 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:21.782083 10004 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:21.782464  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 444) took 1077 ms. Trace:
I20260709 12:26:21.782938  6452 rpcz_store.cc:276] 0709 12:26:20.705273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.716015 (+ 10742us) service_pool.cc:224] Handling call
0709 12:26:21.782266 (+1066251us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54000c17d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:21.782389 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095458688}
I20260709 12:26:21.784386  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:21.785888 10214 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:21.775002  6383 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:21.794423  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:21.816171  6459 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:21.816684  6450 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:26:21.816841  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2615) took 1013 ms. Trace:
I20260709 12:26:21.817157  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2612) took 1105 ms. Trace:
I20260709 12:26:21.814528  6456 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:26:21.814752  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75565.5us
I20260709 12:26:21.815537  6457 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:21.817299  6459 rpcz_store.cc:276] 0709 12:26:20.803413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.827355 (+ 23942us) service_pool.cc:224] Handling call
0709 12:26:21.816770 (+989415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.817508  6450 rpcz_store.cc:276] 0709 12:26:20.717796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.720974 (+  3178us) service_pool.cc:224] Handling call
0709 12:26:21.817100 (+1096126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:21.826479  6204 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 8 to 10, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 10 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:21.736640 10209 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:21.829356 10213 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:21.829461 10127 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:21.775578  9782 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:21.737556  9963 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:21.852527  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101846us
W20260709 12:26:21.855304  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 158507us
I20260709 12:26:21.852080  6454 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:21.745921  9928 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:21.863179  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 111576us
I20260709 12:26:21.872066  6457 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:21.833148 10209 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:21.635381  6206 rpcz_store.cc:269] 0709 12:25:51.451182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:51.452081 (+   899us) service_pool.cc:224] Handling call
0709 12:25:53.518529 (+2066448us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:21.177264 (+27658735us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:16.360973 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:16.642562 (+281589us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:16.663505 (+ 20943us) write_op.cc:435] Acquired schema lock
0709 12:26:16.679296 (+ 15791us) tablet.cc:662] Decoding operations
0709 12:26:16.847398 (+168102us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:16.847500 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:16.847532 (+    32us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:26:16.848968 (+  1436us) tablet.cc:701] Row locks acquired
0709 12:26:16.849027 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:26:16.849274 (+   247us) write_op.cc:270] Start()
0709 12:26:16.849456 (+   182us) write_op.cc:276] Timestamp: P: 1783599976849192 usec, L: 0
0709 12:26:16.849481 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:26:16.939456 (+ 89975us) log.cc:844] Serialized 4723 byte log entry
0709 12:26:18.105450 (+1165994us) op_driver.cc:464] REPLICATION: finished
0709 12:26:18.427770 (+322320us) write_op.cc:301] APPLY: starting
0709 12:26:18.655919 (+228149us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:18.725343 (+ 69424us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:18.728977 (+  3634us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:18.912821 (+183844us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:18.918927 (+  6106us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:26:18.919033 (+   106us) write_op.cc:312] APPLY: finished
0709 12:26:18.919655 (+   622us) log.cc:844] Serialized 89 byte log entry
0709 12:26:18.920910 (+  1255us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:18.921146 (+   236us) write_op.cc:454] Released schema lock
0709 12:26:18.921263 (+   117us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:19.066007 (+144744us) spinlock_profiling.cc:243] Waited 50.9 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a2e28 00007f6fa6cdd581 00007f6fa6cdcd58 00007f6fa6ce19f1 00007f6fa6ce19a9 00007f6fa6ce1939 00007f6fa6ce1901 00007f6fa6ce0b2d 0000000000388634 0000000000388479
0709 12:26:19.066288 (+   281us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:19.066321 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":788105,"reported_tablets":33,"spinlock_wait_cycles":72712064,"child_traces":[["op",{"apply.queue_time_us":289622,"apply.run_cpu_time_us":10838,"apply.run_wall_time_us":522294,"num_ops":12,"prepare.queue_time_us":365419,"prepare.run_cpu_time_us":13691,"prepare.run_wall_time_us":2847231,"raft.queue_time_us":316500,"raft.run_cpu_time_us":93545,"raft.run_wall_time_us":261121,"replication_time_us":1255777,"spinlock_wait_cycles":117141888,"thread_start_us":822221,"threads_started":4,"wal-append.queue_time_us":443880,"wal-append.run_cpu_time_us":65022,"wal-append.run_wall_time_us":2567494}]]}
I20260709 12:26:21.852756  9976 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.892236  6380 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:21.853106 10213 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:21.877693  9750 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.878461 10095 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:21.880720 10216 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:21.854290 10212 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:21.907752  6384 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:21.908380  6380 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:21.813813 10217 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:26:21.913578  6454 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:21.915179  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:21.832324 10004 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
W20260709 12:26:21.928733  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 293372us
I20260709 12:26:21.895641 10215 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:21.933848  6378 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:21.896071 10209 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:21.931360  6204 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 6, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 6 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:21.940181 10233 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.942214 10233 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.944041 10235 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.935546  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2447) took 1108 ms. Trace:
I20260709 12:26:21.944561 10235 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.944778 10225 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.944906  6379 rpcz_store.cc:276] 0709 12:26:20.820950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.829227 (+  8277us) service_pool.cc:224] Handling call
0709 12:26:21.929037 (+1099810us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540010a4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:21.929136 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245146368}
W20260709 12:26:21.944768  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.946874  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:21.953471  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55930us
I20260709 12:26:21.945642 10225 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.940869 10110 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:21.908339  9976 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:21.965663 10230 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:21.968458  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:21.929586 10218 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:26:21.979959  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:21.951333 10095 raft_consensus.cc:2764] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:21.971202 10027 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:21.964584  6204 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:21.914296  9750 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.012465 10225 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.015709 10213 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:22.015620  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.016656  6457 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:22.016789  9750 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:21.951323 10215 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:22.019485 10215 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.021131 10235 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:22.022224  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 565468us
W20260709 12:26:21.986660  6424 leader_election.cc:336] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.163s (SENT)
I20260709 12:26:21.992064 10230 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:21.993122  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2446) took 1172 ms. Trace:
I20260709 12:26:21.993554 10227 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:26:21.993587 10110 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.995246  9928 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:21.964644 10226 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:22.000368 10181 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.001312  9573 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:22.039335 10229 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:22.011513  6204 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 7, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:22.027879 10233 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.028861 10180 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.031687  9963 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:21.975008 10005 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:22.038743  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83980.9us
I20260709 12:26:22.066722  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:21.935281 10212 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.055933  6377 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:22.039562  6376 rpcz_store.cc:276] 0709 12:26:20.820527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.826467 (+  5940us) service_pool.cc:224] Handling call
0709 12:26:21.276914 (+450447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.313697 (+ 36783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.313747 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.365571 (+ 51824us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.666016 (+300445us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:21.759826 (+ 93810us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.760118 (+   292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.760228 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.992778 (+232550us) raft_consensus.cc:1685] finished
0709 12:26:21.992797 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.992924 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15175,"prepare.queue_time_us":93728,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":108300,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":43996,"thread_start_us":350279,"threads_started":2,"wal-append.queue_time_us":313594}
I20260709 12:26:22.003170  9976 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:22.062980  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 102894us
I20260709 12:26:22.065052  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:22.055409 10214 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:26:22.065845  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 207606us
W20260709 12:26:22.074501  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 106144us
I20260709 12:26:22.068150  6204 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 3 to 8, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 8 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:22.077603  6382 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:22.054872 10227 consensus_queue.cc:1243] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:22.073892 10221 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.085590  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:22.042392 10234 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:26:22.080154  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:22.061795 10110 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.078876  6204 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:22.088476  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.088953 10219 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:22.064169 10226 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:22.089660  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103070us
W20260709 12:26:22.144543  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 79563.3us
I20260709 12:26:22.295646  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.077771 10005 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.219s
I20260709 12:26:22.065832 10230 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:22.286013  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:22.291184  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 74224.2us
I20260709 12:26:22.135617  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2449) took 1313 ms. Trace:
W20260709 12:26:22.217192  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 131786us
W20260709 12:26:22.303444  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.303841  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:22.232172  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.070514 10035 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:22.302881  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79002.2us
I20260709 12:26:22.310459  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2621) took 1418 ms. Trace:
W20260709 12:26:22.310875  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 225359us
I20260709 12:26:22.144477 10228 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:22.307984  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.312836  6204 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 2 to 4, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:22.224800  9573 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:22.067528 10168 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.241s
I20260709 12:26:22.091341 10221 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.317485  6204 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 5 to 7, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:22.318867 10231 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:22.320005  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:22.291679 10180 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:26:22.322539 10180 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:22.322958 10180 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:22.324091 10180 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.325719 10180 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:22.329702  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:22.330683  6456 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:22.331549  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 251483us
I20260709 12:26:22.335364  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.318109 10221 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.137547 10236 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:22.340667  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.069753 10145 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.246s
I20260709 12:26:22.341243  6459 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:22.341740  6451 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:22.342411  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.139462 10212 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.299410 10219 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:22.344342  6458 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:22.301082  6381 rpcz_store.cc:276] 0709 12:26:20.821871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.866645 (+ 44774us) service_pool.cc:224] Handling call
0709 12:26:20.898092 (+ 31447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.898731 (+   639us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:21.242660 (+343929us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.686510 (+443850us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.690994 (+  4484us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.700983 (+  9989us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.701296 (+   313us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:21.725356 (+ 24060us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.734601 (+  9245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.734708 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.984606 (+249898us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b18002ce420. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:22.135145 (+150539us) raft_consensus.cc:1685] finished
0709 12:26:22.135180 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.135388 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13407,"prepare.queue_time_us":1040022,"prepare.run_cpu_time_us":128,"prepare.run_wall_time_us":2221,"raft.queue_time_us":375044,"raft.run_cpu_time_us":636,"raft.run_wall_time_us":58584,"spinlock_wait_cycles":77785984,"thread_start_us":688310,"threads_started":3,"wal-append.queue_time_us":352993}
I20260709 12:26:22.309971 10151 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:22.348742 10220 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:22.349177  6452 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:22.352928  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.371199  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.134733 10227 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.209s
I20260709 12:26:22.311056  6453 rpcz_store.cc:276] 0709 12:26:20.892413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.361714 (+469301us) service_pool.cc:224] Handling call
0709 12:26:21.659775 (+298061us) spinlock_profiling.cc:243] Waited 8.7 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:26:21.659977 (+   202us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.860352 (+200375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.860394 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.886842 (+ 26448us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.056039 (+169197us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:22.056086 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.309261 (+253175us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b54001635d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:22.309306 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:22.309384 (+    78us) raft_consensus.cc:1685] finished
0709 12:26:22.309400 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.310400 (+  1000us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":18095,"spinlock_wait_cycles":452595200,"thread_start_us":168712,"threads_started":1,"wal-append.queue_time_us":172850}
I20260709 12:26:22.374630  6455 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:26:22.365548  7287 leader_election.cc:336] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.903s (SENT)
I20260709 12:26:22.361835  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.377502 10004 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:22.379932 10226 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.352389 10035 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:22.381596 10226 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:22.342869  9985 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:22.338222 10230 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:22.401261  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.402995  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.405272  6457 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:22.388090  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85896.6us
I20260709 12:26:22.388294  6456 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:22.353365 10215 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:22.362157 10237 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:22.366171  9573 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:22.412981  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.372620 10212 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.419101  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.423825  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.423118  6305 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:22.436179  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:22.437296  6300 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:22.375921 10151 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:22.417102 10215 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.442695 10215 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.449589 10233 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:22.450150  9976 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:22.451943 10233 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:22.452675  9976 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:22.422843  9573 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:22.389398 10220 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:22.465117  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100711us
W20260709 12:26:22.466295  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100779us
I20260709 12:26:22.468298  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.469051  6457 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:22.371876 10181 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:26:22.477710  6454 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:22.413734 10168 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:22.478926  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.485038  6456 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:22.479404  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.438217  9928 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:22.463559  9573 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.454963 10237 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:22.456880 10232 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:22.454773 10212 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.499406  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.500679  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.463907  9963 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:22.470939  9985 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:22.423487 10209 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:22.508996  7287 leader_election.cc:336] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.948s (SENT)
I20260709 12:26:22.514462  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.488538 10228 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:22.518747  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.485028 10236 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:22.514586  6309 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:22.526230  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.527053  6302 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
W20260709 12:26:22.526535  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.526520  6303 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:22.480119  9750 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
W20260709 12:26:22.531111  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 224608us
I20260709 12:26:22.497560 10232 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:22.477648 10230 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.545205  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.488039 10168 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.497748  9928 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.559701  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.534142 10245 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:26:22.564345  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55389.5us
I20260709 12:26:22.563306  6309 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:26:22.564285  6274 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.831s (SENT)
I20260709 12:26:22.566430  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.568172  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.554502 10095 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:22.554030 10005 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.545003 10004 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:22.518152  9963 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.532727 10228 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.557554 10110 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:22.582214  7287 leader_election.cc:336] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.966s (SENT)
I20260709 12:26:22.553368  6300 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:22.583488  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:22.565649 10223 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.587165  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80136.8us
I20260709 12:26:22.566409 10245 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:22.570962  9846 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:22.610945  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 162585us
I20260709 12:26:22.592233  6309 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:22.611804  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.574167 10095 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.618556 10209 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:22.619146  9928 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.619287 10209 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:22.564471 10151 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.540786 10220 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:22.617252  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 67337.8us
W20260709 12:26:22.625844  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92303.2us
I20260709 12:26:22.626199  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:22.628185  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63945.8us
I20260709 12:26:22.571486 10181 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:26:22.630605  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.631155  6455 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:22.617025 10095 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:22.611140  9846 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.626967  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:22.634658  6274 leader_election.cc:336] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.018s (SENT)
I20260709 12:26:22.635411  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:22.636225  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.639516  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.639657 10228 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.642560  6454 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:26:22.659315  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:22.661983  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115538us
I20260709 12:26:22.591094 10004 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.667852  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.669538  6459 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:26:22.670397  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60064.3us
I20260709 12:26:22.663637  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:22.674736 10227 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:22.674207  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:22.675130  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:22.675622  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:22.676777  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.677174  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.678709  6379 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:22.678961  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.679358  6378 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:26:22.679626  6383 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:22.679857  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.675163 10227 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:22.680986 10227 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.678053  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:22.681499  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.682404  6382 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:22.682994  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:22.683977  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.684557  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:22.686774  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.688889  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:22.689584  6383 raft_consensus.cc:2368] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6.
I20260709 12:26:22.691182  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.634582 10231 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:22.609086 10005 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:26:22.694541  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68391.3us
I20260709 12:26:22.696990  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.608655  9985 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.660843  9846 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:22.707571  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.614013 10223 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:22.660290  9928 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:22.709787  6381 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:22.581780 10237 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.696650  9750 consensus_queue.cc:1243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:26:22.698752  9679 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.937s	user 0.009s	sys 0.004s
W20260709 12:26:22.774304  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 91365us
I20260709 12:26:22.765102  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.610764 10232 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.693809 10257 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:22.726869 10240 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.269s	user 0.001s	sys 0.001s
I20260709 12:26:22.698531 10035 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.780705  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50477.9us
W20260709 12:26:22.779810  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 150731us
I20260709 12:26:22.696482 10145 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:22.782258  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.832741  6456 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:22.785480  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.833727  6383 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:22.696236 10231 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:22.642445 10236 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.693185 10228 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 12:26:22.792483  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.812420  6459 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:22.835606  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.696211 10250 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:22.740811 10243 raft_consensus.cc:2764] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:26:22.740401  9928 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:22.792277  7287 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.078s (SENT)
I20260709 12:26:22.847252  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.765789 10255 raft_consensus.cc:2764] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:26:22.841881  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.851126  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:22.788261  9750 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:26:22.852273  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.881047  6379 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:22.853605  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 179508us
I20260709 12:26:22.857702 10004 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:22.859086 10005 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.862109  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.786908 10244 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:22.849341  6378 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:22.789391 10257 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.792676 10254 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:22.883211 10250 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:22.883893  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:22.890556  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 232235us
I20260709 12:26:22.894680  6456 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:26:22.895143  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 115209us
I20260709 12:26:22.891716  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:22.847826 10232 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.850322 10228 consensus_queue.cc:1243] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:22.851722  9928 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.907847  9963 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.788877  9679 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.700987 10245 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:22.912384  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 126175us
I20260709 12:26:22.912998  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:22.918979  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68460.6us
I20260709 12:26:22.919247  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.838665 10035 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:22.918568  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:22.887604 10263 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:22.897513 10254 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.922413 10230 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:26:22.926468 10231 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:22.939710  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.926793  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:22.928902 10259 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
W20260709 12:26:22.931360 10253 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
I20260709 12:26:22.923905  6379 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9. Re-sending same reply.
I20260709 12:26:22.898406 10257 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:22.941249  6425 leader_election.cc:400] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9.
I20260709 12:26:22.889055 10227 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:22.945365  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 404118us
I20260709 12:26:22.922271  9963 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:22.951205  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.932264  9679 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:22.953698  6425 leader_election.cc:403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:26:22.941391  7288 leader_election.cc:400] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14.
I20260709 12:26:22.961977  7288 leader_election.cc:403] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:26:22.962128  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:22.956573  6382 raft_consensus.cc:2368] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10.
W20260709 12:26:22.960449  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.910502 10262 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:22.967468  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.938071 10263 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:22.974778  6383 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:22.952775 10259 consensus_queue.cc:1243] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:26:22.942986 10254 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:22.975404  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:22.955416 10245 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:22.978349  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.983152  6381 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:22.965281 10231 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:22.984876  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44499.2us
I20260709 12:26:22.986878  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:22.986387  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.991328 10259 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:22.991596 10261 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:22.991838 10259 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.993489  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:22.996863  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:22.992235 10261 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.974291 10263 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.016626 10261 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:22.998999 10244 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14.
I20260709 12:26:23.018152  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:22.992352 10259 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.055960  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:23.045707  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.097342  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:22.996306  9679 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:23.105038  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114777us
I20260709 12:26:22.980540 10151 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
W20260709 12:26:23.070273 10260 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 12:26:23.111526 10270 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 12:26:23.111999 10256 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
I20260709 12:26:23.064122 10262 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:23.142863  6424 leader_election.cc:336] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.632s (SENT)
I20260709 12:26:22.953910 10219 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.095775  9750 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9.
I20260709 12:26:23.155866 10262 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.161899 10269 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.988250 10250 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.163285 10271 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:23.164012 10215 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.721s	user 0.007s	sys 0.004s
W20260709 12:26:23.165081 10267 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
I20260709 12:26:23.154561  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:23.101487 10095 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.160089 10035 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:23.166275  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.211326  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:23.194427  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.223618  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.221621  6378 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
W20260709 12:26:23.211865 10180 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.886s	user 0.012s	sys 0.005s
I20260709 12:26:23.168212  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.224356  6377 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
W20260709 12:26:23.302148  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 246344us
I20260709 12:26:23.303229  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:23.304600  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 287435us
I20260709 12:26:23.226689  6452 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:23.309247  6456 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:23.311097  6382 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:23.194396 10223 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.301926 10278 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:26:23.179610  9573 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.002s
I20260709 12:26:23.322618  6385 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:23.326349  6384 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:23.328439  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 185618us
W20260709 12:26:23.330325  6424 leader_election.cc:336] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.656s (SENT)
W20260709 12:26:23.331250  6424 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.690s (SENT)
I20260709 12:26:23.325435  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.333204  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.334875  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.335074  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:23.335597  6378 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 5.
I20260709 12:26:23.336555  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.337144  6378 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
W20260709 12:26:23.334035  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 409664us
I20260709 12:26:23.124680  9679 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.342052 10288 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.342526  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.342670 10288 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.343547  9679 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:23.344249 10275 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.275s	user 0.000s	sys 0.002s
I20260709 12:26:23.344985 10265 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:26:23.345564 10265 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:23.223968 10276 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:23.347995 10276 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:23.348420 10276 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:23.348218 10265 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.348757 10274 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:23.349313 10274 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:23.253971 10269 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.349707 10274 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:23.341682  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.339860  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.300125 10271 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:23.168375  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:26:23.351413 10269 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.455518  6450 raft_consensus.cc:606] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:23.455996  6450 raft_consensus.cc:614] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:23.367241  7331 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:23.378702  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:23.459451  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 128225us
I20260709 12:26:23.455188  6452 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:23.224180 10262 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.462299  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:23.323438 10035 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:23.461272  7317 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:23.351125  6382 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:26:23.463184  6424 leader_election.cc:336] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.836s (SENT)
I20260709 12:26:23.476714  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2646) took 1123 ms. Trace:
I20260709 12:26:23.476898  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:23.475502  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:23.483076  6425 leader_election.cc:400] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9.
I20260709 12:26:23.415727  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.484047  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:23.483805  6378 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:26:23.484227  6454 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.003s	sys 0.018s
W20260709 12:26:23.487254  6454 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.003s	sys 0.018s
I20260709 12:26:23.489552  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.485924  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:23.493476  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.490963 10255 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:23.495404 10255 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.495756 10255 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:23.363440 10285 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.490675  9750 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:23.502939 10285 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.339794  9573 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.505249  9573 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:23.505287 10285 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:23.501830 10276 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.506724 10277 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:23.507634 10277 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.506784  9985 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:23.339236 10219 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.451808 10288 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.318030 10215 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.415650 10145 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.508104  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:23.509016  9846 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.723s	user 0.008s	sys 0.000s
I20260709 12:26:23.485110 10095 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:23.566298 10181 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.558360  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:23.567394  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 81531.1us
I20260709 12:26:23.531668  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:23.564652  9928 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.642s	user 0.006s	sys 0.003s
W20260709 12:26:23.570431  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 77043.9us
I20260709 12:26:23.552186 10278 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:23.477418 10035 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.508059 10277 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.338820 10227 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.580363  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:23.569790  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 106647us
W20260709 12:26:23.621063  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.621287  6424 leader_election.cc:336] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.895s (SENT)
I20260709 12:26:23.621908  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:23.505456 10231 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.000s
I20260709 12:26:23.587807  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:23.477401  6458 rpcz_store.cc:276] 0709 12:26:22.353002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.423144 (+ 70142us) service_pool.cc:224] Handling call
0709 12:26:22.423404 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.464184 (+ 40780us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.464222 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.507170 (+ 42948us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.890177 (+383007us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:22.890241 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.890352 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.476518 (+586166us) raft_consensus.cc:1685] finished
0709 12:26:23.476537 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.476662 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6250,"raft.queue_time_us":6379,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":14989,"thread_start_us":344659,"threads_started":1,"wal-append.queue_time_us":404921}
I20260709 12:26:23.624500  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.624600  9976 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:23.625273  9976 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:23.625747  9976 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:23.628077 10292 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:23.585819  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:23.589259  7331 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:23.588901  6378 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:23.603233 10257 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.635s	user 0.005s	sys 0.005s
I20260709 12:26:23.605513 10215 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:23.630996  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45259.6us
I20260709 12:26:23.631852 10290 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.632306  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:23.632365 10290 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.633033  6378 raft_consensus.cc:2376] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7. Re-sending same reply.
I20260709 12:26:23.634039  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:23.606643 10287 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 12:26:23.635025  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:23.607299 10254 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.610s	user 0.009s	sys 0.000s
I20260709 12:26:23.635792  6378 raft_consensus.cc:2376] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11. Re-sending same reply.
I20260709 12:26:23.635982 10254 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.637717  6377 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:23.635105  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.607909  7317 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:23.638346 10257 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.639914  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:23.640668  7317 raft_consensus.cc:2376] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14. Re-sending same reply.
I20260709 12:26:23.640928 10257 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:23.636896  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.564110 10276 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:23.643013 10276 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:23.643817 10276 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:23.581444 10145 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:23.611349  7287 leader_election.cc:336] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.990s (SENT)
I20260709 12:26:23.361812 10261 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.649668 10261 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.531895  9783 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:23.616473  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.650875  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:23.652056  7319 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:23.630826 10292 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:23.653246 10292 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:23.653673 10292 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:23.654789 10292 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.605897  9928 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.658082  9928 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.644009 10215 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:23.658732 10288 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:23.661729  9783 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:23.663308  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.665107  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:23.581800  9846 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.666451  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.667095  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:23.667050  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 174137us
I20260709 12:26:23.634807 10278 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:23.665992  7314 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEARNER]: Advancing to term 11
I20260709 12:26:23.678839  6310 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.672565  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.618578 10035 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:23.675590  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.561539 10274 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:23.668741 10232 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.760s	user 0.010s	sys 0.000s
I20260709 12:26:23.680353  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.691037  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:23.691470  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.689240  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:23.694931  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73662.4us
W20260709 12:26:23.696879  6424 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.038s (SENT)
W20260709 12:26:23.700515  6424 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.098s (SENT)
I20260709 12:26:23.588881 10181 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.706334  6453 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:23.702258  6302 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:23.707711  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:23.707896  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:23.706921  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66523.1us
I20260709 12:26:23.678093  9783 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:23.717545  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.719360  9846 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:23.722390  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.723559  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.723765  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:23.724355  6303 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.724888  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:23.670382 10220 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:26:23.675547 10288 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:23.694087 10257 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:23.726150  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:23.757442  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.758770  6309 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.759831  7314 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:23.726955  9985 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.701969 10278 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:23.716903 10263 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.004s
I20260709 12:26:23.761554  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:23.683821 10215 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:23.763211  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.716096 10231 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:23.763760  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.768682  6382 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.761730  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:23.763980  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:23.773170  6310 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:23.727149  9573 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:23.716303 10145 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:23.774761  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74286.9us
W20260709 12:26:23.776784  6424 leader_election.cc:336] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.138s (SENT)
I20260709 12:26:23.777527  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:23.760599 10232 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:23.780571  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:23.783571  6304 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:23.782027 10274 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.786856 10284 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:23.791344  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:23.791630 10290 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:23.792165  7316 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:23.795499 10293 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:26:23.795814 10215 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:23.796840  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:23.797693 10181 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:23.798638  7320 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:23.799614  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2647) took 1092 ms. Trace:
I20260709 12:26:23.800028  6454 rpcz_store.cc:276] 0709 12:26:22.707306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.860042 (+152736us) service_pool.cc:224] Handling call
0709 12:26:22.860429 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.488300 (+627871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.488330 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.488827 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.621659 (+132832us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:23.621698 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.621770 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.799397 (+177627us) raft_consensus.cc:1685] finished
0709 12:26:23.799420 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.799553 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111780,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":5617,"thread_start_us":758511,"threads_started":2,"wal-append.queue_time_us":204775}
I20260709 12:26:23.806092  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:23.807559  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:23.807978 10005 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:23.808208  6458 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:23.808491 10005 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:23.809289  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:23.809389 10005 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.810189  6458 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:23.810547 10145 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:23.810354  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:23.811164 10145 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:23.811549  6457 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:23.812230 10145 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:23.813318 10280 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.462s	user 0.001s	sys 0.001s
I20260709 12:26:23.813397  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:23.814699  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2648) took 1105 ms. Trace:
I20260709 12:26:23.815049 10110 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:23.815614 10110 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:23.815097  6451 rpcz_store.cc:276] 0709 12:26:22.709495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.860928 (+151433us) service_pool.cc:224] Handling call
0709 12:26:22.861427 (+   499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.946163 (+ 84736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:22.946205 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:22.968973 (+ 22768us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.304201 (+335228us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:23.304265 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.304367 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.812827 (+508460us) spinlock_profiling.cc:243] Waited 49.8 ms on lock 0x7b24001ea9b8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:26:23.814367 (+  1540us) raft_consensus.cc:1685] finished
0709 12:26:23.814388 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.814626 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99462,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":24491,"spinlock_wait_cycles":114497280,"thread_start_us":351012,"threads_started":2,"wal-append.queue_time_us":369486}
I20260709 12:26:23.816776 10110 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:23.780854  6424 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.146s (SENT)
I20260709 12:26:23.817907  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:23.766690  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56272.2us
I20260709 12:26:23.798259  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:23.820926  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.781742  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:23.822865  6310 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.824513  6305 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:23.825140  7320 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:23.727917 10254 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:23.769126 10259 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.644s	user 0.016s	sys 0.002s
I20260709 12:26:23.810359  9573 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.821792  6383 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9. Re-sending same reply.
I20260709 12:26:23.689108 10271 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.797919 10035 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.087949  6303 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:26:23.863831  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.934835  6382 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:23.830566  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:24.099725  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.030539  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.946220  6310 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:24.156137  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.151793  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:24.152029  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 331192us
I20260709 12:26:23.774122 10231 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.862210  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.158344  6302 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:23.832357  6451 pending_rounds.cc:85] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 3
I20260709 12:26:23.883864  6458 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:24.146802  6304 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:24.156880  7314 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:23.935974 10295 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
I20260709 12:26:24.164769  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:24.166380  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 304329us
I20260709 12:26:24.166414  6458 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:24.167577  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.168152  6458 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.163627  6379 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:24.157685  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:24.169595  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 388740us
W20260709 12:26:24.170361  6424 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.179s (SENT)
I20260709 12:26:24.159188  6451 pending_rounds.cc:107] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:26:24.172075  6451 raft_consensus.cc:2889] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:23.790306 10277 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.175194  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.176087 10277 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.177202  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.177959  6455 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.178107 10274 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.163105 10231 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.178653 10274 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.180917 10274 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.181495 10274 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.178838  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:23.915107  9846 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.182566  6455 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.180617  9783 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.853307 10257 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:23.876978 10232 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:24.183274  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.836360 10005 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:23.916045 10110 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:24.185883  6309 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:24.186219  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:23.774060 10263 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.197446 10221 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:24.199878 10221 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:24.200847 10221 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:23.854813 10181 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.188808  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.165894  9928 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:24.206763  6310 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
I20260709 12:26:23.973407  9985 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:24.184007  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.189594 10227 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:24.198434 10255 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.000s
I20260709 12:26:24.169698 10248 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:23.976821 10262 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.208709  6455 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:23.818622  9976 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.220687 10292 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:24.175977  6383 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.223310  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.210978 10254 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:26:24.196996  6302 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:24.212478  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.181933 10274 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.214707 10168 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.157298 10259 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.215319  6301 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:24.215667  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:23.851202 10288 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:24.223489  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.163216  9573 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.234398  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2658) took 1322 ms. Trace:
I20260709 12:26:24.235211  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.235347  6456 rpcz_store.cc:276] 0709 12:26:22.911757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.947456 (+ 35699us) service_pool.cc:224] Handling call
0709 12:26:24.234205 (+1286749us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b540009b354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.234342 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":795917184}
I20260709 12:26:24.237572  6309 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:24.238121  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:23.830701 10278 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.239199  6378 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.232333  6307 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:24.239487  6452 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:26:24.238766  6309 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:24.190650  9846 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:24.215356 10277 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:24.240842  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70503us
I20260709 12:26:24.243248  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.242405 10227 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:24.243781 10254 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:24.213663 10219 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:26:24.245345 10254 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:24.245759 10254 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.246477  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:24.246722 10262 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.250675 10004 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:24.252159 10263 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.252919 10232 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.256924  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.257862  6302 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.258718  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:24.259374  6302 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.216893 10255 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.265806  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.266199  6308 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:24.260491  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.260921  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.268075  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.268261 10274 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.268455 10281 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:24.268776  6376 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:24.269064 10281 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.269774  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:24.269402  6453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2645) took 1922 ms (client timeout 2105 ms). Trace:
W20260709 12:26:24.270941  6453 rpcz_store.cc:269] 0709 12:26:22.347616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.372739 (+ 25123us) service_pool.cc:224] Handling call
0709 12:26:24.269197 (+1896458us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540009a6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.269338 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2969901824}
I20260709 12:26:24.271524  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.272203  6377 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:24.273064  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.273753  6377 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:24.274967  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.262842  6384 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.275713  6376 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.275449  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.281385  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.291502  6383 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:24.292073  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.292798  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.288686  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.295182  6458 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 12 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:24.295236  6455 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:24.295734  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.293540  6304 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:24.296788  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.293546  6383 raft_consensus.cc:2376] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9. Re-sending same reply.
I20260709 12:26:24.299885  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.300810  6376 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.300736  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:24.302089  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.302589  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.294281 10274 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:24.311041 10274 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.311522 10274 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.307825  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.304744  6454 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:24.305374  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.306146  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.309782  7331 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:26:24.316293  6377 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.321475  6450 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.277143 10212 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.322549  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.323369  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:24.328819  7331 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:24.305644 10277 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:24.314744 10272 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:24.308741  9783 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.337769  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.317380  9976 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:24.339320  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1950) took 1664 ms (client timeout 1564 ms). Trace:
I20260709 12:26:24.341434  6309 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:24.342346  6450 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:26:24.344869  6381 rpcz_store.cc:269] 0709 12:26:22.674678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.674843 (+   165us) service_pool.cc:224] Handling call
0709 12:26:24.339090 (+1664247us) spinlock_profiling.cc:243] Waited 33.9 ms on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.339230 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77892480}
I20260709 12:26:24.321249 10004 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.294514 10209 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 12:26:24.353664 10234 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:24.355207  6300 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:24.355540 10234 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:24.357712 10271 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:24.352739  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:24.358033 10262 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.359112 10262 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.349627  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.318256 10095 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260709 12:26:24.368834  6302 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:24.370545  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.350677 10212 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:24.369644 10151 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:24.371867  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.372249 10151 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:24.372459  6377 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:24.366658  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.372020 10212 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.384411  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.385288  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:24.398608  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.399528  6302 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:24.402841 10318 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:24.402244 10277 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:24.405084 10277 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.406020 10318 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:24.406312  6302 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:26:24.407527 10318 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.412081  6302 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:26:24.412509  6302 pending_rounds.cc:85] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 5
I20260709 12:26:24.412801  6302 pending_rounds.cc:107] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:26:24.413033  6302 raft_consensus.cc:2889] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.417848  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:24.425648  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 129983us
W20260709 12:26:24.426229  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 59639us
I20260709 12:26:24.426472  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.432389 10318 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:24.434504  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:24.427245  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:24.439669  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 135907us
I20260709 12:26:24.439800  6301 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:24.440680  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:24.444116  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81981.9us
I20260709 12:26:24.365310 10004 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:24.445887  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.445978  7287 leader_election.cc:336] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.811s (SENT)
I20260709 12:26:24.447397  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.483572  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.465102  6301 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:24.457580  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.435396 10281 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.433280 10318 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:24.494454  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48581us
I20260709 12:26:24.423982 10284 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.067s
I20260709 12:26:24.495532  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.484405 10245 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:24.502458  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.500289  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.504484  6377 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.489627 10272 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:24.508494  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.509256  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 469) took 1036 ms. Trace:
I20260709 12:26:24.509548  6377 rpcz_store.cc:276] 0709 12:26:23.473093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.508200 (+1035107us) service_pool.cc:224] Handling call
0709 12:26:24.509202 (+  1002us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.509768 10004 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.493644 10215 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:24.510471  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.521081  6376 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:24.521878  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 475) took 1021 ms. Trace:
I20260709 12:26:24.524242  6376 rpcz_store.cc:276] 0709 12:26:23.500395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.299335 (+798940us) service_pool.cc:224] Handling call
0709 12:26:24.521823 (+222488us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:24.504664  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58729.8us
I20260709 12:26:24.516320  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.503403 10245 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.527731  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.502861  9573 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.504415 10259 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.507090  9846 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:24.506628 10244 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:24.530607  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:24.543646  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:24.512389 10251 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:24.550818  6382 raft_consensus.cc:2410] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:26:24.552057  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.552644  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.530140  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2476) took 1217 ms. Trace:
I20260709 12:26:24.545746  6377 raft_consensus.cc:606] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.546006  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.553524  6376 rpcz_store.cc:276] 0709 12:26:23.312780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.527518 (+1214738us) service_pool.cc:224] Handling call
0709 12:26:24.530072 (+  2554us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.556463  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.557121  6450 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:24.557873 10281 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.558357 10281 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.559252 10212 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.560773 10281 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:24.561574  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.561401 10254 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.456096 10151 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.572607  6376 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:26:24.554188  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:24.574081  6376 pending_rounds.cc:85] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 3
I20260709 12:26:24.574402  6376 pending_rounds.cc:107] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:26:24.574798  6376 raft_consensus.cc:2889] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.555209  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:26:24.568486  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.529364 10245 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:24.538128 10319 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:24.578477  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2468) took 1267 ms. Trace:
I20260709 12:26:24.531874 10325 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:24.549162 10251 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:24.603015  7329 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:24.603724  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 777) took 1182 ms. Trace:
I20260709 12:26:24.603552 10251 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.604061  7329 rpcz_store.cc:276] 0709 12:26:23.421623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.421778 (+   155us) service_pool.cc:224] Handling call
0709 12:26:24.602531 (+1180753us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:26:24.602798 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.603414 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.603625 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8336256}
I20260709 12:26:24.604694 10251 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:24.605805  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2478) took 1266 ms. Trace:
I20260709 12:26:24.606168  6384 rpcz_store.cc:276] 0709 12:26:23.340484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.342235 (+  1751us) service_pool.cc:224] Handling call
0709 12:26:24.605597 (+1263362us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.605745 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114680192}
I20260709 12:26:24.607621  6384 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:24.608196  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2470) took 1296 ms. Trace:
I20260709 12:26:24.578590 10281 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:24.525624 10231 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.596092  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.597965  6376 rpcz_store.cc:276] 0709 12:26:23.310827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.554959 (+1244132us) service_pool.cc:224] Handling call
0709 12:26:24.555282 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.578186 (+ 22904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.578429 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.575434 10110 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:24.601372  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.608501  6384 rpcz_store.cc:276] 0709 12:26:23.311326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.606994 (+1295668us) service_pool.cc:224] Handling call
0709 12:26:24.607297 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.607968 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.608143 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.594862 10318 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.612355  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.690922 10262 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.691479  6451 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:24.696897  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.698622  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2484) took 1246 ms. Trace:
W20260709 12:26:24.697513  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.623870  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.596603  9846 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:24.663895  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.701632  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:24.702587  6377 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:24.711637  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.664317  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 775) took 1270 ms. Trace:
I20260709 12:26:24.712939  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:24.699074  6378 rpcz_store.cc:276] 0709 12:26:23.452627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.636586 (+183959us) service_pool.cc:224] Handling call
0709 12:26:24.698412 (+1061826us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b540022ef54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.698564 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1361524992}
I20260709 12:26:24.712873  7335 rpcz_store.cc:276] 0709 12:26:23.393750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.393900 (+   150us) service_pool.cc:224] Handling call
0709 12:26:24.663523 (+1269623us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:26:24.663740 (+   217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.663954 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.663984 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.664040 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:24.664081 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.664138 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.664254 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7112064}
I20260709 12:26:24.715363  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:24.715705  7331 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:24.717024  7331 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:24.717155  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.719247  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.720201  6305 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:24.599210 10212 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:24.724570  7332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 537) took 1315 ms (client timeout 1689 ms). Trace:
I20260709 12:26:24.725397  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:24.690047 10324 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:26:24.727309  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:24.728539  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 132553us
I20260709 12:26:24.733772  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.735009  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2473) took 1423 ms. Trace:
I20260709 12:26:24.735986  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.736400  6376 rpcz_store.cc:276] 0709 12:26:23.312080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.721409 (+1409329us) service_pool.cc:224] Handling call
0709 12:26:24.721679 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.734704 (+ 13025us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.734730 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.734773 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:24.734808 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.734858 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.734956 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.738330  6376 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260709 12:26:24.739138  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1989) took 1369 ms. Trace:
I20260709 12:26:24.739552  6376 rpcz_store.cc:276] 0709 12:26:23.369911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.737482 (+1367571us) service_pool.cc:224] Handling call
0709 12:26:24.737928 (+   446us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:24.738830 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.739071 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:24.729619  7332 rpcz_store.cc:269] 0709 12:26:23.409661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.409840 (+   179us) service_pool.cc:224] Handling call
0709 12:26:24.663697 (+1253857us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:26:24.724491 (+ 60794us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7805696}
I20260709 12:26:24.735806  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:24.608639 10035 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.758872  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:24.759238  6378 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:26:24.758948  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 468) took 1371 ms. Trace:
I20260709 12:26:24.760506  6376 rpcz_store.cc:276] 0709 12:26:23.370421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.740475 (+1370054us) service_pool.cc:224] Handling call
0709 12:26:24.740817 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.741059 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.741098 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:24.741159 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:24.741217 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.741281 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.741374 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.749492  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:24.761955  6384 raft_consensus.cc:606] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.762454  6384 raft_consensus.cc:614] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:24.763008  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.763832  6384 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:24.764707  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 477) took 1263 ms. Trace:
I20260709 12:26:24.765094  6384 rpcz_store.cc:276] 0709 12:26:23.500986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.763302 (+1262316us) service_pool.cc:224] Handling call
0709 12:26:24.763659 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.764390 (+   731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.764639 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.765777  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:24.762144  6376 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:24.714964 10271 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:24.767588  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 474) took 1267 ms. Trace:
I20260709 12:26:24.711869  9846 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:24.767889  6274 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.730s (SENT)
I20260709 12:26:24.782352  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.751931  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.767889  7329 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:26:24.759583  6378 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 4
I20260709 12:26:24.783963  6378 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:26:24.784228  6378 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.784619  6378 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:26:24.784893  6378 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.769755  6384 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:24.769932  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.636778 10281 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:24.774999  6376 rpcz_store.cc:276] 0709 12:26:23.500081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.761492 (+1261411us) service_pool.cc:224] Handling call
0709 12:26:24.761824 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.767319 (+  5495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.767529 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.665210 10231 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.776110  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.785777  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 478) took 1284 ms. Trace:
I20260709 12:26:24.806171  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.806919  6384 rpcz_store.cc:276] 0709 12:26:23.501298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.765981 (+1264683us) service_pool.cc:224] Handling call
0709 12:26:24.766316 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.785485 (+ 19169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.785713 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.808039  6450 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.809083  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2475) took 1496 ms. Trace:
I20260709 12:26:24.809643  6378 rpcz_store.cc:276] 0709 12:26:23.312557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.720771 (+1408214us) service_pool.cc:224] Handling call
0709 12:26:24.758876 (+ 38105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.808532 (+ 49656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.808984 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.735131 10325 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:24.807626  6455 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:24.819379  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.787544  6376 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:24.812629  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.749403 10328 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:24.815981  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.749419  9573 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:24.760032 10278 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:24.721175 10274 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.823696  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 479) took 1322 ms. Trace:
I20260709 12:26:24.735874 10284 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.078s
I20260709 12:26:24.750447 10212 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.813122 10277 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:24.776065 10315 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.786554 10095 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:24.599803 10272 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:24.759660 10220 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:26:24.790345  9846 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:24.842222  6451 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:24.853554  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.853864 10110 raft_consensus.cc:2764] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:26:24.845467 10277 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.776649  9783 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:26:24.843912  6376 rpcz_store.cc:276] 0709 12:26:23.501592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.787028 (+1285436us) service_pool.cc:224] Handling call
0709 12:26:24.787362 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.823107 (+ 35745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.823567 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:24.838657  6450 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:26:24.788465  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.825999 10254 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:24.861829  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1881) took 1007 ms. Trace:
W20260709 12:26:24.851202  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 122580us
W20260709 12:26:24.851585  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55360.2us
I20260709 12:26:24.852239  6383 raft_consensus.cc:2368] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11.
I20260709 12:26:24.827999 10250 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.862417  6310 rpcz_store.cc:276] 0709 12:26:23.854542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.165583 (+311041us) service_pool.cc:224] Handling call
0709 12:26:24.166651 (+  1068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.207506 (+ 40855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.861511 (+654005us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:24.861732 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91818752}
I20260709 12:26:24.840845 10284 consensus_queue.cc:1243] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:24.839815 10237 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.839923 10278 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.854285  6451 raft_consensus.cc:2410] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:24.865516  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:24.876132  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.874040  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:24.915763  6450 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.916523  6376 raft_consensus.cc:606] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.917088  6376 raft_consensus.cc:614] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:24.880739  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 112901us
I20260709 12:26:24.893972  6455 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 6.
W20260709 12:26:24.892016  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.859915 10315 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:24.890982  6385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1957) took 2200 ms (client timeout 1758 ms). Trace:
I20260709 12:26:24.891747 10278 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.890949  6378 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:26:24.909405  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.923388  6378 pending_rounds.cc:85] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 10
I20260709 12:26:24.923841  6378 pending_rounds.cc:107] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:26:24.924122  6378 raft_consensus.cc:2889] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:24.922732  6385 rpcz_store.cc:269] 0709 12:26:22.677054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.679537 (+  2483us) service_pool.cc:224] Handling call
0709 12:26:24.871055 (+2191518us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54001955d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.877167 (+  6112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1467660672}
I20260709 12:26:24.924789  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.904443 10251 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.926358  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:24.926095  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60685.4us
I20260709 12:26:24.860131 10281 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.937938  6450 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:24.938908  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.902001 10237 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.932705 10335 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:24.943271 10212 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:24.950397  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2123) took 1145 ms. Trace:
I20260709 12:26:24.950870  6452 rpcz_store.cc:276] 0709 12:26:23.805349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.810703 (+  5354us) service_pool.cc:224] Handling call
0709 12:26:24.234597 (+423894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.239834 (+  5237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.950214 (+710380us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540009b350. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:24.950342 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974024064}
I20260709 12:26:24.951906  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.952764  6450 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:24.954402  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 184539us
I20260709 12:26:24.954643  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.963536  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:24.937561 10316 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:26:24.964885  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:24.968662  6452 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:24.965281  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:24.930480 10251 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:24.974429  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:24.975979  6376 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:24.975555 10284 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:24.942162 10250 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:26:24.978440  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:24.979507  6376 raft_consensus.cc:606] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.979866  6376 raft_consensus.cc:614] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:24.978591  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:24.971710 10315 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.985333  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2649) took 2272 ms. Trace:
I20260709 12:26:24.985950  6457 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:26:24.990033  6452 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:24.989384  6459 rpcz_store.cc:276] 0709 12:26:22.712439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.898144 (+185705us) service_pool.cc:224] Handling call
0709 12:26:22.917598 (+ 19454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.096417 (+178819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:23.096445 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:23.126444 (+ 29999us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.305355 (+178911us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:23.305415 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.305516 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.954461 (+1648945us) spinlock_profiling.cc:243] Waited 48.3 ms on lock 0x7b24001ea9b8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f8fb0 00007f6fa2e6f488 00007f6fa2e6fbe8 00007f6fa2e47398 00007f6fa2e4276a 00007f6fa2e41bb6
0709 12:26:24.985029 (+ 30568us) raft_consensus.cc:1685] finished
0709 12:26:24.985050 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.985261 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":142391,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":26354,"spinlock_wait_cycles":111072512,"thread_start_us":267157,"threads_started":2,"wal-append.queue_time_us":184337}
I20260709 12:26:24.972345 10340 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:24.994035  7332 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:24.994457  7332 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:24.996713  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:24.995638  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.998147  6381 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:24.998870  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:24.999749  7332 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.004297  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.013002  6450 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:25.014169  7332 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:25.013530  6376 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:25.014961  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.014448  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.017102  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.018002  6459 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:24.993338 10274 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.020447  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.016541  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:24.995103 10328 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:25.021054  6376 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:25.001691 10347 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:25.000367 10277 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.034358  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:24.977610 10339 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:26:25.043756  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.038760  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.039737  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.036230  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:25.044947  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:25.045511  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:25.037664  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.047358  6376 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:25.047254  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:25.048236  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.048403  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:25.048949  7329 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:25.050233  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:24.994325 10315 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.054692  6310 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:25.055478  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:25.048852  6376 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:25.044225 10334 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:25.058873  6459 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:25.059154  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:25.059569  6376 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:26:25.060475  6459 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:25.035427 10335 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:25.063845  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.065963  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.069540  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:25.073841  6381 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:26:25.074527  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.076030  6459 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:25.080879  6308 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:26:25.087869 10345 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:25.087527 10335 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.055701  9783 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:25.092049  6459 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:25.068482 10234 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.042932 10328 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:25.108347  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 43756.1us
I20260709 12:26:25.111995  6450 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:25.112323  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.113144 10254 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.113664 10254 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.114110 10328 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.076967 10220 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:25.118948  7318 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:25.119223  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:25.119827  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2133) took 1061 ms. Trace:
I20260709 12:26:25.120268  6458 rpcz_store.cc:276] 0709 12:26:24.058219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.183739 (+125520us) service_pool.cc:224] Handling call
0709 12:26:25.119665 (+935926us) spinlock_profiling.cc:243] Waited 84.7 ms on lock 0x7b54001c1e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.119772 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194721536}
I20260709 12:26:25.121688  6458 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:26:25.124593  6458 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:25.130350  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.131234  6376 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:25.132058  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.132617  6376 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:25.081755 10338 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:26:25.134610  9928 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:25.135102  9928 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.092588  9783 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.115337  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.089814 10250 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:26:25.082401 10212 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:26:25.153331 10315 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.164032  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:25.166659 10262 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:25.167809 10262 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:25.168188 10262 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.168536 10262 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.168920 10288 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.170676 10276 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:25.171103 10276 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:25.171926 10276 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.173110 10276 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.175949  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.176605  6378 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:25.142882  9783 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:25.181205  6424 leader_election.cc:336] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.032s (SENT)
I20260709 12:26:25.181141  6459 raft_consensus.cc:1217] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[6.4-11.6]   Dedup: 6.4->[8.5-11.6]
W20260709 12:26:25.182924  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 1980) took 1991 ms (client timeout 2032 ms). Trace:
W20260709 12:26:25.186008  6379 rpcz_store.cc:269] 0709 12:26:23.192116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.192270 (+   154us) service_pool.cc:224] Handling call
0709 12:26:25.182663 (+1990393us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b540023dcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.182846 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372672896}
I20260709 12:26:25.187412  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.188655  6376 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:25.188868  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.189626  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.190696  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.191205  6376 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10. Re-sending same reply.
I20260709 12:26:25.191948  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:25.192138 10245 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.581s	user 0.005s	sys 0.006s
I20260709 12:26:25.192591  6376 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:25.182917  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.192538 10245 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.144780 10349 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:25.193859  7319 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:25.193545 10349 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.195678 10349 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:25.195865 10254 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:25.204540  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1873) took 1484 ms (client timeout 1726 ms). Trace:
W20260709 12:26:25.205045  6307 rpcz_store.cc:269] 0709 12:26:23.719796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.757131 (+ 37335us) service_pool.cc:224] Handling call
0709 12:26:25.204273 (+1447142us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b540010f254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.204423 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062977280}
I20260709 12:26:25.205466 10348 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.206108 10348 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.207266 10209 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:25.207701 10209 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:25.208251 10348 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:25.208590 10209 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.210052 10209 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:25.210915  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2128) took 1388 ms. Trace:
I20260709 12:26:25.211323  6454 rpcz_store.cc:276] 0709 12:26:23.822707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.822884 (+   177us) service_pool.cc:224] Handling call
0709 12:26:24.165248 (+342364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.315954 (+150706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.210646 (+894692us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54001c1954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:25.210844 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320219904}
I20260709 12:26:25.211956  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.190326  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:25.185513  6459 pending_rounds.cc:85] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 4
I20260709 12:26:25.230504  6459 pending_rounds.cc:107] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:26:25.230822  6459 raft_consensus.cc:2889] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:25.231103  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.231290  6452 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:25.232209  7332 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:26:25.307641  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.361713  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 514) took 1599 ms. Trace:
I20260709 12:26:25.362224  6300 rpcz_store.cc:276] 0709 12:26:23.615189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.620381 (+  5192us) service_pool.cc:224] Handling call
0709 12:26:23.673188 (+ 52807us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:24.355741 (+682553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.206332 (+850591us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54002df5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:25.215015 (+  8683us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1568067840}
I20260709 12:26:25.361892  6452 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:26:25.392454  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 211262us
I20260709 12:26:25.394352  6452 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:26:25.435317  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.185143 10237 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:25.230392 10328 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.450235  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:25.363734 10110 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:25.448973 10245 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:25.473073  6452 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:25.476421  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:25.479923  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:25.485234  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 319931us
W20260709 12:26:25.466398 10278 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.001s
I20260709 12:26:25.454708 10328 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:25.492261  6382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 470) took 1993 ms (client timeout 1811 ms). Trace:
I20260709 12:26:25.466235 10035 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:25.494943  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.498432  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:25.493773  6382 rpcz_store.cc:269] 0709 12:26:23.498777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.155889 (+657112us) service_pool.cc:224] Handling call
0709 12:26:25.492055 (+1336166us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b54003e7954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.492191 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":905484160}
I20260709 12:26:25.501682  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.501852 10250 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:25.511065 10151 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:25.511705 10151 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:25.512687 10151 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.513201  6382 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:25.514532  6382 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:25.516634 10251 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:25.508605  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:25.511400 10250 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.458520 10110 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.520140 10250 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.520277 10110 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:25.525134 10336 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:25.525743 10336 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:25.526175 10336 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.535115 10215 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:25.539240  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.542001  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.506737 10035 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:25.551932  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.556845  6378 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:25.566654  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:25.567531  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 209126us
I20260709 12:26:25.568955  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:25.570194  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 120079us
I20260709 12:26:25.593039  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:25.567551  9846 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.001s
I20260709 12:26:25.606688  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:26:25.570518 10271 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.000s
I20260709 12:26:25.612242  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:25.565485 10035 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:25.559283 10220 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:26:25.526307 10278 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.563110 10237 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:26:25.566259 10251 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:26:25.617105 10095 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:25.661027  6384 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:25.615767 10357 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:26:25.616161 10347 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:25.568082 10281 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:25.633292 10244 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:25.675959  6450 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:26:25.647454 10110 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:25.647929  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 476) took 2147 ms (client timeout 2789 ms). Trace:
I20260709 12:26:25.656567  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2127) took 1850 ms. Trace:
I20260709 12:26:25.658987 10221 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:25.630277 10271 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:25.678433  6383 rpcz_store.cc:269] 0709 12:26:23.500688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.300457 (+799769us) service_pool.cc:224] Handling call
0709 12:26:25.647664 (+1347207us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003e4ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.647851 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265239424}
W20260709 12:26:25.662182 10325 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.008s	sys 0.013s
I20260709 12:26:25.665717  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.681679  6454 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:25.621816  9846 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.678610  6457 rpcz_store.cc:276] 0709 12:26:23.806399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.812317 (+  5918us) service_pool.cc:224] Handling call
0709 12:26:24.985698 (+1173381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.990271 (+  4573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.656286 (+666015us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400265f50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.656485 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697660672}
W20260709 12:26:25.665026  6381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.013s	sys 0.023s
W20260709 12:26:25.670207  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 165502us
W20260709 12:26:25.686776  6381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.013s	sys 0.023s
I20260709 12:26:25.633942 10250 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:26:25.677548 10244 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:25.694923 10244 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:25.700675 10276 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:26:25.702075 10276 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.703894 10278 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:25.682590 10325 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.008s	sys 0.014s
I20260709 12:26:25.707969 10325 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:25.709009 10325 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:25.711457 10288 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
I20260709 12:26:25.712157 10274 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.712656 10274 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:25.713411  9928 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.578s	user 0.005s	sys 0.004s
I20260709 12:26:25.713824  6379 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:26:25.714231  6379 pending_rounds.cc:85] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 5
I20260709 12:26:25.714444  6379 pending_rounds.cc:107] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:25.714686  6379 raft_consensus.cc:2889] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:25.716301 10234 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.002s
I20260709 12:26:25.716753 10234 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:25.664760 10212 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:26:25.717552 10336 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.688509  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:25.731926  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:25.733227  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:25.734944 10328 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:25.735833 10357 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:25.736289 10357 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.736680 10357 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:25.738286 10288 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:26:25.760027  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 129429us
I20260709 12:26:25.761159 10336 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:25.777140  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 211346us
I20260709 12:26:25.713866  9928 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.776603 10364 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:26:25.760668 10288 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:25.795058  6300 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 7. Current term is 8. Ops: []
W20260709 12:26:25.739616 10262 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.571s	user 0.012s	sys 0.000s
I20260709 12:26:25.799471  6300 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:25.801211  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:25.806463  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.819990  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:25.743553 10254 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:25.825568  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.825801  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:25.826512  6302 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.772008 10264 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:25.832095 10339 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:25.836145  9928 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:25.835893 10035 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.838979  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:25.819895 10281 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:26:25.843109  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2660) took 1014 ms. Trace:
I20260709 12:26:25.843544  6459 rpcz_store.cc:276] 0709 12:26:24.829102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.143895 (+314793us) service_pool.cc:224] Handling call
0709 12:26:25.180855 (+ 36960us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:25.447439 (+266584us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:25.459383 (+ 11944us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.685764 (+226381us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:25.686810 (+  1046us) log.cc:844] Serialized 52 byte log entry
0709 12:26:25.686980 (+   170us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:25.687023 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.687090 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.842842 (+155752us) raft_consensus.cc:1685] finished
0709 12:26:25.842869 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.843023 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6345,"prepare.queue_time_us":8278,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":10889,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":6920,"thread_start_us":188987,"threads_started":1,"wal-append.queue_time_us":298505}
I20260709 12:26:25.827371  7332 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:25.849685  6459 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:25.779804  9783 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.851790  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 548) took 1340 ms. Trace:
I20260709 12:26:25.852387  9783 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:25.852260  7319 rpcz_store.cc:276] 0709 12:26:24.511147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.529955 (+ 18808us) service_pool.cc:224] Handling call
0709 12:26:25.851572 (+1321617us) spinlock_profiling.cc:243] Waited 639 ms on lock 0x7b5400479ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.851722 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1469895424}
I20260709 12:26:25.773202 10367 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:26:25.854079 10035 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:25.855032  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.855752  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 449) took 1070 ms. Trace:
I20260709 12:26:25.863241 10215 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:25.854449  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 482) took 1536 ms. Trace:
I20260709 12:26:25.858286  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2659) took 1038 ms. Trace:
I20260709 12:26:25.866622 10215 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.854508 10367 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:25.867622 10367 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:25.866631  6384 rpcz_store.cc:276] 0709 12:26:24.318555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.809368 (+490813us) service_pool.cc:224] Handling call
0709 12:26:25.660763 (+851395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.661618 (+   855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.854251 (+192633us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54003e4ed0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.854389 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922454400}
I20260709 12:26:25.868786 10215 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:25.866890 10341 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.894s	user 0.010s	sys 0.008s
I20260709 12:26:25.860751  9846 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:25.828068 10237 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:26:25.870676  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 791) took 1168 ms. Trace:
I20260709 12:26:25.871454 10237 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:25.861552  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2661) took 1032 ms. Trace:
I20260709 12:26:25.872001 10237 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:25.872093  6450 rpcz_store.cc:276] 0709 12:26:24.829399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.186344 (+356945us) service_pool.cc:224] Handling call
0709 12:26:25.216482 (+ 30138us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.676452 (+459970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.861217 (+184765us) spinlock_profiling.cc:243] Waited 459 ms on lock 0x7b54000eb354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:25.861465 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1056069504}
I20260709 12:26:25.873095 10237 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.820024 10262 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.876667 10262 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.877281 10262 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:25.877624 10262 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:25.865773  6455 rpcz_store.cc:276] 0709 12:26:24.784932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.819062 (+ 34130us) service_pool.cc:224] Handling call
0709 12:26:25.855515 (+1036453us) spinlock_profiling.cc:243] Waited 73.8 ms on lock 0x7b54001e19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.855677 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":169726208}
I20260709 12:26:25.872231 10274 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:25.856626  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 1982) took 2632 ms. Trace:
I20260709 12:26:25.886952  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.894752  6301 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:25.887192  6303 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:25.871102  7316 rpcz_store.cc:276] 0709 12:26:24.701896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.702010 (+   114us) service_pool.cc:224] Handling call
0709 12:26:24.702279 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.702814 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.702851 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.749267 (+ 46416us) log.cc:844] Serialized 26 byte log entry
0709 12:26:24.908798 (+159531us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:24.908843 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.908931 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.866213 (+957282us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:25.870362 (+  4149us) raft_consensus.cc:1685] finished
0709 12:26:25.870382 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.870616 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":326,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":11368,"spinlock_wait_cycles":5427584,"thread_start_us":92224,"threads_started":1,"wal-append.queue_time_us":163115}
I20260709 12:26:25.863837 10281 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:25.897025 10281 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:25.897760  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.898033 10281 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.899430 10281 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:25.890308  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.887482  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:25.894788  6308 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:25.786219 10244 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.873458  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:25.905797  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:25.920387  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47016us
I20260709 12:26:25.920866  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 523) took 1200 ms. Trace:
I20260709 12:26:25.921213  6305 rpcz_store.cc:276] 0709 12:26:24.720276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.905539 (+1185263us) service_pool.cc:224] Handling call
0709 12:26:25.920811 (+ 15272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:25.867128  6458 rpcz_store.cc:276] 0709 12:26:24.820288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.125517 (+305229us) service_pool.cc:224] Handling call
0709 12:26:25.856981 (+731464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.857221 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.857258 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.857318 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:25.857377 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.857450 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.858103 (+   653us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54001e19d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.858225 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1681580928}
I20260709 12:26:25.892248  6380 rpcz_store.cc:276] 0709 12:26:23.224039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.224413 (+   374us) service_pool.cc:224] Handling call
0709 12:26:25.182867 (+1958454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.183184 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.183212 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.183250 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:25.183295 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.183365 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.856384 (+673019us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540023dcd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:25.856516 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":462,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":12511,"spinlock_wait_cycles":4308755200}
I20260709 12:26:25.903023  6450 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:25.924135  6310 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:25.924424  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:25.926126  6450 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.928118  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:25.931831  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:25.932462  6458 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:25.932785  6302 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:25.882880 10255 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:25.874410 10237 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:25.881794  9985 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:25.942775  9985 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:25.944356  6301 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:25.930184  9846 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:26:25.946718  9846 consensus_queue.cc:1243] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:26:25.949487  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 190151us
W20260709 12:26:25.950845  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.882725 10220 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:26:25.983867  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:25.941905  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 160577us
I20260709 12:26:25.958863  6450 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:25.987063  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:25.988127  6450 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:25.878674 10212 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:26:25.989243  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:25.868963 10367 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:25.998986  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:26:26.002833  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.934401  6302 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:26.010772  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:25.908977 10288 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:25.878468 10262 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:26.026831  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 414706us
I20260709 12:26:26.033493  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:26.036015  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:26.028326  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:26.014454 10264 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:25.960465 10349 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:26:26.026113 10209 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:26.059363  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.027881 10367 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.061654  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.043848 10215 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:26.045305 10321 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:26.049901 10255 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:26.046432 10264 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:26.058368 10262 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:26.072474  6381 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:26.058107  9846 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.059440 10209 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.081231  6381 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:26.084815  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:26.075978  6195 maintenance_manager.cc:419] P 08c731ab128f4ef881e4c3c73ff48bd8: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033369
I20260709 12:26:26.089562  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:26.085443  6381 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:26.056106 10254 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:26.060284 10278 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:26:26.096158  6380 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:26.096318 10278 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:26.096702 10278 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:26.097633 10278 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.067528 10321 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:26.099390 10321 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.102300 10278 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:26.071116 10035 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:26.149837  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.152424 10318 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:26:26.152845 10318 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:26.152963 10348 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:26.153599 10348 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:26.154690 10348 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.155980 10348 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:26.156551 10335 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:26.157127 10335 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.158224 10372 raft_consensus.cc:1064] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:26:26.159327 10335 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:26.160051 10335 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.160441 10372 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 5, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.071101 10380 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:26.093812  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:26.094887 10095 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:26.095286  6384 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:26.104027  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:26.069471 10255 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.064972 10274 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
W20260709 12:26:26.165864  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61940.6us
I20260709 12:26:26.076690  9846 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.085292 10209 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:26.117772 10254 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.126842  6458 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
W20260709 12:26:26.116219  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.121035 10276 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:26.121699 10244 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:26.137594 10388 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.137997 10264 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.141837  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2142) took 1844 ms. Trace:
I20260709 12:26:26.189230  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:26.192381  6456 rpcz_store.cc:276] 0709 12:26:24.297198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.319967 (+ 22769us) service_pool.cc:224] Handling call
0709 12:26:24.330197 (+ 10230us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.198689 (+868492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.198720 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.199225 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.478978 (+279753us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:25.479023 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.141422 (+662399us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:26.141478 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.141521 (+    43us) raft_consensus.cc:1685] finished
0709 12:26:26.141534 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.141777 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21631,"raft.queue_time_us":18323,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":156216,"spinlock_wait_cycles":1996536576,"thread_start_us":279326,"threads_started":1,"wal-append.queue_time_us":492632}
I20260709 12:26:26.194346  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:26.194743  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:26.195467  6456 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:26.057003  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:26.142554  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:26.144649  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 595) took 1422 ms. Trace:
I20260709 12:26:26.147245 10374 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.148002 10357 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.160797 10335 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.187148  9846 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:26.079516 10319 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.229s	user 0.009s	sys 0.029s
I20260709 12:26:26.164464  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:26.248822  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84459.1us
W20260709 12:26:26.171187  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.186408  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:26.251757  6458 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:26.165459 10255 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:26.198215  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86636.8us
I20260709 12:26:26.197055  7331 rpcz_store.cc:276] 0709 12:26:24.721952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.725097 (+  3145us) service_pool.cc:224] Handling call
0709 12:26:26.144444 (+1419347us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b540070e4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:26.144587 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754759936}
I20260709 12:26:26.153745 10318 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.106848 10257 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.092s
W20260709 12:26:26.212922 10331 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.350s	user 0.003s	sys 0.009s
I20260709 12:26:26.215725 10264 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:26.189471 10110 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:26.217115  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2504) took 1088 ms. Trace:
I20260709 12:26:26.267288  6379 rpcz_store.cc:276] 0709 12:26:25.128980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.490431 (+361451us) service_pool.cc:224] Handling call
0709 12:26:25.490908 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.715150 (+224242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.216802 (+501652us) spinlock_profiling.cc:243] Waited 223 ms on lock 0x7b54001946d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:26.217023 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":511792000}
I20260709 12:26:26.219064  6457 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:26.248430 10319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.009s	sys 0.029s
I20260709 12:26:26.221194 10335 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:26.222842 10383 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:26.222963  6456 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:26.228036 10357 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:26.259649  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.260565  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 170747us
I20260709 12:26:26.262193  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:26.270042  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:26.277642  7287 leader_election.cc:400] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11.
I20260709 12:26:26.189647 10271 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
W20260709 12:26:26.219300  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 339671us
I20260709 12:26:26.279263 10276 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:26.279702 10276 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.280102 10276 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:26.281385  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:26.282050  7331 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.264453 10281 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.310587  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:26.316754  6457 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:26.242866 10367 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:26.291057  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.197778 10388 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.322885  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:26.328006  7316 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:26.271490 10110 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.230649 10374 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.331350  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:26.335055  6384 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:26.273881 10383 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:26.336107  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:26.336871  6380 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:26.332337  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56960.9us
I20260709 12:26:26.338292  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:26.339427  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:26.274305 10390 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:26:26.383111 10390 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:26.383574 10390 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:26.384462 10390 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.341302  7332 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:26:26.342178  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:26.388424  7329 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:26.342859 10357 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.343138 10245 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:26.343669 10364 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:26.346319  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:26.347074  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:26.349874  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 455) took 1531 ms (client timeout 1899 ms). Trace:
W20260709 12:26:26.391067  6451 rpcz_store.cc:269] 0709 12:26:24.818932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.926019 (+107087us) service_pool.cc:224] Handling call
0709 12:26:26.349670 (+1423651us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:26.349809 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2700770688}
I20260709 12:26:26.392107  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.350080  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:26.392968  6376 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:26.271849 10244 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:26.352946  9985 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.354921  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:26.414093  7329 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:26.355214  6383 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:26:26.356238  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 194118us
I20260709 12:26:26.357421  7331 raft_consensus.cc:2410] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:26:26.415904  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69680.8us
I20260709 12:26:26.416215  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:26:26.360307  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.320811 10281 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:26.366361  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.357210  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:26.276188 10336 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:26:26.326869 10095 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:26.337282 10110 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.379341  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:26.353806 10318 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:26.388007  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:26.389328 10357 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.385808 10390 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:26.400017  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.411612  9846 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:26:26.384366 10358 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:26:26.417538  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:26.418931  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:26.325719 10367 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:26.423033  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68202.4us
I20260709 12:26:26.516099  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:26.489445  6384 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.508140  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:26.415266 10168 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:26.520632  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:26.421272 10232 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.426337 10281 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.421331 10254 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:26.453758 10237 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.529340  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:26.511108 10357 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.549809  6451 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:26.425664 10394 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.549233  6308 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:26.512171  9976 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.498270 10271 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:26.423926 10257 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:26:26.424547 10319 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.111s
I20260709 12:26:26.422710 10325 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.105s
W20260709 12:26:26.565200  6453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2137) took 2272 ms (client timeout 1539 ms). Trace:
W20260709 12:26:26.570585  6453 rpcz_store.cc:269] 0709 12:26:24.281603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.281780 (+   177us) service_pool.cc:224] Handling call
0709 12:26:26.553797 (+2272017us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b54001e23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:26.554002 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4298853120}
I20260709 12:26:26.561175  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:26.572228  6459 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:26.511606 10396 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 12:26:26.578791  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.564079 10262 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:26.577955  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:26.611536  6384 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
W20260709 12:26:26.612311  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 262736us
I20260709 12:26:26.532864 10168 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.624883  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:26.548661 10232 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.584755 10254 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:26.566469 10250 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:26:26.607084 10262 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.584463 10394 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:26.570129  9976 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.516548 10383 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.569509 10271 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:26.664865  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 241302us
W20260709 12:26:26.669579  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 250743us
I20260709 12:26:26.672382  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2678) took 1200 ms. Trace:
I20260709 12:26:26.677032  6452 rpcz_store.cc:276] 0709 12:26:25.472204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.600006 (+127802us) service_pool.cc:224] Handling call
0709 12:26:25.600297 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.600512 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:26:25.631878 (+ 31366us) log.cc:844] Serialized 14 byte log entry
0709 12:26:26.396962 (+765084us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b18002593c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:26:26.672023 (+275061us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:26.672081 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:26.672136 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.672208 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.672326 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":294,"prepare.queue_time_us":31187,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":3357,"spinlock_wait_cycles":4085120,"thread_start_us":274342,"threads_started":1}
I20260709 12:26:26.666257  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:26.644341 10325 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:26.643378 10262 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:26.647567 10336 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:26.653631 10254 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:26.685371 10393 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
W20260709 12:26:26.685811  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 165296us
I20260709 12:26:26.759402  6378 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:26:26.759874  6378 pending_rounds.cc:85] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 7
I20260709 12:26:26.760095  6378 pending_rounds.cc:107] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:26:26.760327  6378 raft_consensus.cc:2889] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.760684  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:26.672070 10388 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:26.763334  6376 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:26.763607  6380 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:26.652311 10394 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.671459 10380 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:26:26.708227 10374 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:26.684443 10257 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:26:26.760988  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 252380us
I20260709 12:26:26.734854  6452 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:26.684293 10328 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:26.766388  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 188517us
I20260709 12:26:26.734479 10245 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:26.818470  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:26.755812 10319 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.756016 10237 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:26.825932  7329 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:26.833644  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 248636us
I20260709 12:26:26.835037  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:26.838587  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.840021  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:26.823956 10392 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:26:26.799866  9846 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:26.824486 10400 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.863797  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.818979 10367 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:26.816869 10413 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:26.852703 10398 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:26:26.835431 10288 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.870437  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:26.888818  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.867420 10407 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:26.880265 10374 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:26.836742 10319 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:26.887940 10403 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:26.881433  9846 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:26.883208 10400 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:26.883780 10417 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.918717 10339 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:26.898882 10168 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:26.917325 10407 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.971400  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 467) took 1079 ms. Trace:
I20260709 12:26:26.970930 10388 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:26.951431 10237 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:26.922806 10418 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:26.983355 10407 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:26.883519 10288 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:26.989437  6450 rpcz_store.cc:276] 0709 12:26:25.892061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.993632 (+101571us) service_pool.cc:224] Handling call
0709 12:26:25.994222 (+   590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.994771 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.994815 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.012699 (+ 17884us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.183672 (+170973us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:26.183733 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.183841 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:26.971145 (+787304us) raft_consensus.cc:1685] finished
0709 12:26:26.971167 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:26.971327 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1581,"raft.queue_time_us":14998,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":2044,"thread_start_us":126654,"threads_started":1,"wal-append.queue_time_us":382735}
W20260709 12:26:26.997219  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 236638us
I20260709 12:26:26.942926 10232 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:26.927928  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 261644us
W20260709 12:26:26.998965  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 581483us
I20260709 12:26:27.011794  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:27.013450  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:27.014860  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:27.016278  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:26.987157 10418 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.988438  9846 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:26.950387 10374 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:26.875692 10254 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:27.037179  6450 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:26.979935 10271 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:26.998584 10364 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.000s	sys 0.008s
I20260709 12:26:26.988327  9976 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:26.983902 10388 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:27.061200  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.014s	sys 0.005s
W20260709 12:26:27.107014  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.014s	sys 0.005s
W20260709 12:26:27.112651  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 96469.9us
I20260709 12:26:26.998361 10416 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:27.100683 10364 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:26.992350 10168 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:26:27.101334 10215 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:27.032887 10418 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:27.111358 10271 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:27.106647 10367 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:27.011960 10417 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:27.146152  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.001s	sys 0.012s
W20260709 12:26:27.129338 10209 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.934s	user 0.012s	sys 0.001s
W20260709 12:26:27.144307  6459 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.002s	sys 0.010s
I20260709 12:26:27.142905 10215 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:27.176625  6459 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.003s	sys 0.010s
I20260709 12:26:27.145642 10232 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.193329 10439 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:27.153650 10271 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.143970 10416 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:27.112128  9846 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
W20260709 12:26:27.177047  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.002s	sys 0.012s
W20260709 12:26:27.202905 10404 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
W20260709 12:26:27.203953 10405 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
W20260709 12:26:27.204272 10402 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.347s	user 0.000s	sys 0.001s
W20260709 12:26:27.207207 10437 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:26:27.178326 10434 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.281s
W20260709 12:26:27.214918 10419 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
I20260709 12:26:27.215086 10232 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:27.217257 10377 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.148s	user 0.004s	sys 0.001s
I20260709 12:26:27.191589 10209 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:27.219147 10427 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 12:26:27.220207 10410 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.340s	user 0.000s	sys 0.001s
I20260709 12:26:27.220583 10444 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:27.221143 10440 raft_consensus.cc:2764] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:27.223367 10373 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:27.224575 10245 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:27.170922 10334 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:26:27.172789 10394 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:27.109536 10400 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:27.624907 10386 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:26:27.687228 10271 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:27.614575 10288 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:27.688673  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2014) took 1410 ms. Trace:
W20260709 12:26:27.220104 10438 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:26:27.229333  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 473) took 1066 ms. Trace:
W20260709 12:26:27.311240  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 290063us
W20260709 12:26:27.683885  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.804s	user 0.004s	sys 0.020s
W20260709 12:26:27.697347  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.004s	sys 0.020s
W20260709 12:26:27.698222  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 820319us
W20260709 12:26:27.703267  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.692481  6456 rpcz_store.cc:276] 0709 12:26:26.162953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.855685 (+692732us) service_pool.cc:224] Handling call
0709 12:26:26.872162 (+ 16477us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.872488 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.872518 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.879805 (+  7287us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.034787 (+154982us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:27.034841 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.034933 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.229065 (+194132us) raft_consensus.cc:1685] finished
0709 12:26:27.229089 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.229253 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":181612,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":41889,"thread_start_us":86878,"threads_started":1,"wal-append.queue_time_us":224490}
W20260709 12:26:27.238359 10255 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.969s	user 0.011s	sys 0.002s
I20260709 12:26:27.640663  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 599) took 1343 ms. Trace:
I20260709 12:26:27.689204  6379 rpcz_store.cc:276] 0709 12:26:26.278341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.294606 (+ 16265us) service_pool.cc:224] Handling call
0709 12:26:26.294969 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.295166 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.295183 (+    17us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.295598 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.627721 (+332123us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:26.627756 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.628234 (+   478us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.688350 (+1060116us) raft_consensus.cc:1685] finished
0709 12:26:27.688379 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.688577 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":261051,"threads_started":1,"wal-append.queue_time_us":350466}
I20260709 12:26:27.731900  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 810) took 1181 ms. Trace:
I20260709 12:26:27.689180 10288 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
W20260709 12:26:27.651494 10110 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.139s	user 0.012s	sys 0.001s
I20260709 12:26:27.178380 10423 raft_consensus.cc:2764] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:26:27.694336 10281 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.130s	user 0.010s	sys 0.002s
W20260709 12:26:27.695551 10372 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.833s	user 0.001s	sys 0.023s
I20260709 12:26:27.145354 10374 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:27.624740 10406 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 12:26:27.743317 10429 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.448s	user 0.000s	sys 0.001s
I20260709 12:26:27.730720  7320 rpcz_store.cc:276] 0709 12:26:26.297349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.297823 (+   474us) service_pool.cc:224] Handling call
0709 12:26:26.320842 (+ 23019us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:26.321345 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.321390 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:26.322170 (+   780us) log.cc:844] Serialized 52 byte log entry
0709 12:26:26.802942 (+480772us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:26.803000 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.803105 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.640378 (+837273us) raft_consensus.cc:1685] finished
0709 12:26:27.640399 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.640585 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1110,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":422649,"threads_started":1,"wal-append.queue_time_us":1174586}
I20260709 12:26:27.288170  9783 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:27.225318 10349 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:27.710337 10373 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:27.745123  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.030s	sys 0.040s
W20260709 12:26:27.530804 10264 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.261s	user 0.013s	sys 0.001s
W20260709 12:26:27.227339 10433 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:26:27.281402 10390 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.000s
I20260709 12:26:27.682359 10244 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.760440  6272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.884s	user 0.019s	sys 0.039s
W20260709 12:26:27.744077  7286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.792s	user 0.003s	sys 0.038s
I20260709 12:26:27.583837 10430 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.775600  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.004s	sys 0.008s
I20260709 12:26:27.776621  6453 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:27.732522  7319 rpcz_store.cc:276] 0709 12:26:26.550506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.550700 (+   194us) service_pool.cc:224] Handling call
0709 12:26:26.569785 (+ 19085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.795547 (+225762us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.795576 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.796056 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.922307 (+126251us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:26.923226 (+   919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.923333 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.219254 (+295921us) spinlock_profiling.cc:243] Waited 87.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:27.701779 (+482525us) raft_consensus.cc:1685] finished
0709 12:26:27.701804 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.731808 (+ 30004us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":184557,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8546,"spinlock_wait_cycles":202121984,"thread_start_us":280578,"threads_started":2,"wal-append.queue_time_us":159932}
I20260709 12:26:27.707966 10245 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:27.977661  6272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.024s	sys 0.043s
W20260709 12:26:27.978046  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.062s	user 0.037s	sys 0.041s
W20260709 12:26:27.978174  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.10232e+06us
W20260709 12:26:27.973920  7287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.601s (SENT) [suppressed 50 similar messages]
I20260709 12:26:27.859279  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2016) took 1580 ms. Trace:
I20260709 12:26:27.649617 10215 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:27.980409  7286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.006s	sys 0.045s
I20260709 12:26:27.843243  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2688) took 1931 ms. Trace:
W20260709 12:26:27.977304  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.007s	sys 0.008s
I20260709 12:26:27.992839  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:27.899286  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2700) took 1500 ms. Trace:
W20260709 12:26:27.777019 10372 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.001s	sys 0.025s
I20260709 12:26:27.747411 10334 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.001487  6310 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:28.035754  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:28.001777  6457 rpcz_store.cc:276] 0709 12:26:25.917403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.318283 (+400880us) service_pool.cc:224] Handling call
0709 12:26:26.318726 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.322341 (+  3615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.322378 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.322938 (+   560us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.825926 (+502988us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:26.825968 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.826040 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.842461 (+1016421us) raft_consensus.cc:1685] finished
0709 12:26:27.842531 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.842844 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1452,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":7979,"thread_start_us":445902,"threads_started":1,"wal-append.queue_time_us":1475565}
I20260709 12:26:28.002736  6300 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 5. Current term is 6. Ops: []
W20260709 12:26:27.917654 10262 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.152s	user 0.010s	sys 0.001s
W20260709 12:26:27.841882 10357 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.271s	user 0.007s	sys 0.004s
W20260709 12:26:27.849304 10336 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.092s	user 0.008s	sys 0.001s
W20260709 12:26:27.981719 10325 consensus_meta.cc:287] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.218s	user 0.007s	sys 0.006s
I20260709 12:26:27.829650 10423 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.001183  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 875793us
I20260709 12:26:28.003280  6309 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:28.003492  6450 rpcz_store.cc:276] 0709 12:26:26.398570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.051583 (+653013us) service_pool.cc:224] Handling call
0709 12:26:27.052008 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.054715 (+  2707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.054763 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.096011 (+ 41248us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.653314 (+557303us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:27.653364 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.653457 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.898974 (+245517us) raft_consensus.cc:1685] finished
0709 12:26:27.898998 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.899203 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2153,"raft.queue_time_us":631419,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":37046,"thread_start_us":169785,"threads_started":1,"wal-append.queue_time_us":767889}
W20260709 12:26:27.867444 10276 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.587s	user 0.014s	sys 0.001s
W20260709 12:26:27.981019 10278 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.803s	user 0.015s	sys 0.000s
I20260709 12:26:28.003885  7335 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:28.045735  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:27.866361 10445 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.887979 10416 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:27.830379 10448 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.048664  6304 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:27.845077 10388 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.049983  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2689) took 2132 ms. Trace:
I20260709 12:26:28.061823  6454 rpcz_store.cc:276] 0709 12:26:25.917688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.345681 (+427993us) service_pool.cc:224] Handling call
0709 12:26:26.346015 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.346413 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.346447 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.346904 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.614642 (+267738us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:26.614683 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.614754 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.049773 (+1435019us) raft_consensus.cc:1685] finished
0709 12:26:28.049795 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.049921 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3845,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":267300,"threads_started":1,"wal-append.queue_time_us":385152}
I20260709 12:26:27.780313 10110 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.001720 10319 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:27.999279 10450 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.074436  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:27.802691 10281 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.048990 10336 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:27.759275 10255 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:27.414901 10432 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.898s
I20260709 12:26:27.855304 10417 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:27.977056 10349 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:28.054574  6308 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
W20260709 12:26:28.037765 10427 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 12:26:28.003840 10237 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.985666  9783 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:28.136838  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2153) took 2100 ms. Trace:
I20260709 12:26:28.137349  6459 rpcz_store.cc:276] 0709 12:26:26.036626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.606784 (+570158us) service_pool.cc:224] Handling call
0709 12:26:26.628799 (+ 22015us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:27.191885 (+563086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:27.191953 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:27.192616 (+   663us) log.cc:844] Serialized 26 byte log entry
0709 12:26:27.421976 (+229360us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:27.422054 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.422145 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.136575 (+714430us) raft_consensus.cc:1685] finished
0709 12:26:28.136602 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.136769 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8381,"raft.queue_time_us":70453,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8285,"thread_start_us":726873,"threads_started":2,"wal-append.queue_time_us":865000}
I20260709 12:26:27.839442 10447 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.830174 10436 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.319s
I20260709 12:26:28.049121 10325 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:28.005160 10244 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.059402 10276 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.003474  6381 rpcz_store.cc:276] 0709 12:26:26.278965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.301521 (+ 22556us) service_pool.cc:224] Handling call
0709 12:26:26.301885 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.302259 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.302292 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.302811 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.549105 (+246294us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:26.549152 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.549237 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:27.858877 (+1309640us) raft_consensus.cc:1685] finished
0709 12:26:27.858917 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:27.859170 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10169,"raft.queue_time_us":13749,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4958,"thread_start_us":207228,"threads_started":1,"wal-append.queue_time_us":348557}
I20260709 12:26:28.052063  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:27.729375 10444 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.108072 10349 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.117990  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 806) took 1722 ms. Trace:
I20260709 12:26:28.118718 10417 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.119402 10416 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.120430 10451 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.129081 10255 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.129295 10321 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:28.129765  6458 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:28.133162 10418 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.982s	user 0.009s	sys 0.000s
I20260709 12:26:28.101503  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2691) took 2178 ms. Trace:
I20260709 12:26:28.212275  6455 rpcz_store.cc:276] 0709 12:26:25.922772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.347690 (+424918us) service_pool.cc:224] Handling call
0709 12:26:26.347979 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.348315 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.348342 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.348766 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.572869 (+224103us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:26.572931 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.573030 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.101214 (+1528184us) raft_consensus.cc:1685] finished
0709 12:26:28.101254 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.101417 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2207,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":47,"thread_start_us":223685,"threads_started":1,"wal-append.queue_time_us":414666}
I20260709 12:26:28.066884 10278 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:28.045591 10262 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.143900  6383 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:26:28.145596  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:28.048645 10357 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.083577 10319 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.151141  7316 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:28.154827  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:28.287274  6385 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:28.288012  6385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 493) took 1939 ms (client timeout 1601 ms). Trace:
W20260709 12:26:28.288622  6385 rpcz_store.cc:269] 0709 12:26:26.348440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.350839 (+  2399us) service_pool.cc:224] Handling call
0709 12:26:28.154654 (+1803815us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:26:28.287911 (+133257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10711936}
I20260709 12:26:28.075515 10448 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.066350 10445 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.163751  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:28.300056  6304 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:28.163940  7335 rpcz_store.cc:276] 0709 12:26:26.395268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.395664 (+   396us) service_pool.cc:224] Handling call
0709 12:26:26.480493 (+ 84829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.054038 (+1573545us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.117656 (+ 63618us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b540016dd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:28.117901 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3502982016}
I20260709 12:26:28.168586  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:28.001892 10430 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.126076 10237 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:28.172956  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 199075us
I20260709 12:26:28.175724  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2018) took 1438 ms. Trace:
W20260709 12:26:28.175740  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 196876us
I20260709 12:26:28.306715 10237 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:28.308207  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.174248  6425 leader_election.cc:336] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.726s (SENT)
W20260709 12:26:28.310355  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136129us
I20260709 12:26:28.160609 10388 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.311738 10388 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:28.314035 10388 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.044651 10334 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.209579 10453 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:28.305128  6384 rpcz_store.cc:276] 0709 12:26:26.737275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.737440 (+   165us) service_pool.cc:224] Handling call
0709 12:26:26.737949 (+   509us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:26.738557 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.738597 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:26.753024 (+ 14427us) log.cc:844] Serialized 78 byte log entry
0709 12:26:26.859553 (+106529us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:26.859616 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.859712 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.175488 (+1315776us) raft_consensus.cc:1685] finished
0709 12:26:28.175514 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.175666 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134071,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":9561,"thread_start_us":41357,"threads_started":1,"wal-append.queue_time_us":232438}
I20260709 12:26:28.210258  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:28.315428  6305 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:28.210829 10415 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:26:28.054004 10423 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:28.266796  6348 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.539s (SENT)
I20260709 12:26:28.099619 10450 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:28.206037 10318 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.701s	user 0.016s	sys 0.003s
I20260709 12:26:28.325564  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:28.326658  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59942.9us
I20260709 12:26:28.172914 10325 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.071936 10373 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:28.286640 10407 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.266s	user 0.012s	sys 0.000s
I20260709 12:26:28.292291 10215 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.206813 10349 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.292943 10456 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:28.245947 10452 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:28.218182 10209 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:28.161366  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109403us
I20260709 12:26:28.368698  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:28.371086  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:28.304409 10430 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:28.353029  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 176580us
I20260709 12:26:28.202256 10416 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.176733 10417 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.151628 10447 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.373418  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.276115 10278 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.418947  6457 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:28.408411  6385 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:28.409933  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.421128  6450 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:28.422566  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.424168  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.331247 10237 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.334741 10388 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:28.415792  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.425825  6456 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:28.355512 10035 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.442629  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.443413  6457 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:28.358004 10461 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.251957 10418 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.318312 10319 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.399173 10255 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.254062  9783 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.418524 10374 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.211480 10451 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.353767 10325 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.482595  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:28.355758 10407 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.451735 10430 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:26:28.506111  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 195112us
I20260709 12:26:28.352258 10453 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.480573 10110 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.446256 10237 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.525178  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:28.530185  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 127113us
I20260709 12:26:28.532267  6385 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:28.529968  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 161423us
I20260709 12:26:28.453958 10035 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.494957 10462 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:28.533286  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:28.469703 10432 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:28.498206 10460 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:28.537338  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:28.499207 10281 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.476925 10255 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.417567 10452 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.483852 10374 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.559104  6456 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:26:28.555634  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.530011 10209 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.535068 10276 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.565868 10445 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.566793 10334 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.558869 10272 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:28.567481 10272 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:28.567945 10272 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:28.568126  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.568797  6274 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.946s (SENT)
I20260709 12:26:28.555897 10465 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
W20260709 12:26:28.569644  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.556203 10460 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:28.584507 10469 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:28.554323 10430 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:28.460642 10461 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.595707  6457 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:28.599500  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2694) took 2645 ms. Trace:
I20260709 12:26:28.599964  6451 rpcz_store.cc:276] 0709 12:26:25.943881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.392958 (+449077us) service_pool.cc:224] Handling call
0709 12:26:26.393214 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.563425 (+170211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.589238 (+2025813us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001c1e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:28.589441 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":359528832}
W20260709 12:26:28.602281  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.553509 10110 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:26:28.571012 10315 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.607890  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.563675 10254 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:26:28.617172  6456 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:28.627458  6452 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:28.578712 10209 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.633992  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:28.595292 10462 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:26:28.621701 10315 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.629688 10318 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.748819  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.718204  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:28.709594  6274 connection.cc:256] RPC call timeout handler was delayed by -1.9041239160001169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26480000000000004s for a response
W20260709 12:26:28.760119  6274 connection.cc:256] RPC call timeout handler was delayed by -1.8983337440004107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.267s for a response
I20260709 12:26:28.758248  6456 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:28.626881 10460 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.757807  6451 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
W20260709 12:26:28.680430 10411 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.713s	user 0.005s	sys 0.016s
W20260709 12:26:28.769837  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.638180 10450 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:28.779099  6274 connection.cc:256] RPC call timeout handler was delayed by -1.718275210000229s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
I20260709 12:26:28.683077 10445 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.684017 10415 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:28.782687  6274 connection.cc:256] RPC call timeout handler was delayed by -1.111348652000288s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
W20260709 12:26:28.788427  6274 connection.cc:256] RPC call timeout handler was delayed by -1.0627159310001844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
I20260709 12:26:28.788300  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:28.790164  6274 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.516s (SENT)
I20260709 12:26:28.750182 10237 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.636835 10416 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.680927 10035 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.635699 10417 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.646056 10278 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.680255 10321 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:28.638156 10334 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:28.763608 10318 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.728477 10319 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.719453 10418 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.790618  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:28.633217 10423 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.764833 10476 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.793190  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.793346 10477 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.193s
I20260709 12:26:28.793514 10281 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:28.796492  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 804) took 2414 ms. Trace:
I20260709 12:26:28.881580  6451 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:28.798117 10328 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:28.803126 10408 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.985s	user 0.017s	sys 0.013s
I20260709 12:26:28.807524 10452 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:28.808569 10412 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.023s	user 0.007s	sys 0.007s
I20260709 12:26:28.808517 10461 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.811901 10255 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.814023 10448 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.818292  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:28.871196 10424 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:28.823644 10447 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.825947 10209 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.826725 10453 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:28.827455 10374 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.847453  6456 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:28.778959 10460 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:28.734135 10325 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:28.880149  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 90016.9us
I20260709 12:26:28.791980 10445 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.881661  7314 rpcz_store.cc:276] 0709 12:26:26.384499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.385206 (+   707us) service_pool.cc:224] Handling call
0709 12:26:26.385648 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:26.386089 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.386130 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:26.386734 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:26.706173 (+319439us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:26.706213 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.706281 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.638280 (+1931999us) spinlock_profiling.cc:243] Waited 923 us on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:28.796095 (+157815us) raft_consensus.cc:1685] finished
0709 12:26:28.796130 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.796411 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1756,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":125,"spinlock_wait_cycles":2122112,"thread_start_us":318720,"threads_started":1,"wal-append.queue_time_us":532847}
W20260709 12:26:28.891535 10484 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:26:28.884074 10476 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.855701 10334 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.872484 10450 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.898181  6272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.007s	sys 0.022s
W20260709 12:26:28.920539  6272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.007s	sys 0.022s
W20260709 12:26:28.920848  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 604377us
I20260709 12:26:28.920953  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2699) took 2522 ms. Trace:
I20260709 12:26:28.921303  6452 rpcz_store.cc:276] 0709 12:26:26.398289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.047504 (+649215us) service_pool.cc:224] Handling call
0709 12:26:27.049067 (+  1563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.627981 (+1578914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.920687 (+292706us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54001e1ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:28.920890 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3615181568}
I20260709 12:26:28.877321 10278 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.899968  6385 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:28.822139  6383 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:26:28.915022 10245 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.910s	user 0.009s	sys 0.005s
I20260709 12:26:28.886392 10281 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:28.910851  6274 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.644s (SENT)
I20260709 12:26:28.937820  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:28.890126 10328 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:28.884199 10485 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:28.831149 10415 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.870399 10321 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:28.948850 10486 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:26:28.949899 10321 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.914649  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:28.916286  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95760.7us
I20260709 12:26:28.953347  6451 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:28.954591  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.954737 10035 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:28.923378 10334 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.918164  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.956745  6456 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:28.922741  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.957845  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.893178 10272 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.959257  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.959971 10476 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:28.960124 10272 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:28.905994  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:28.962345  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:28.980492  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 74637.4us
I20260709 12:26:28.965199  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.961136  6456 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:26:28.982760  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.983322  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:28.983803  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:28.984279  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:26:28.963069  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.986255  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:28.985324  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:28.992978  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.986653  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:28.988274  6452 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 6.
I20260709 12:26:28.932183 10432 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:28.957787 10349 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.000s
I20260709 12:26:28.915673 10460 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.992594  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:28.944479 10281 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:28.943215 10245 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:28.932567 10450 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.936553 10278 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:28.911320 10424 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.094806 10321 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:29.007164 10416 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:29.097519 10416 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.097265 10321 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.097954 10416 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.012871 10237 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:29.098875 10415 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.098994 10237 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:29.100512 10237 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:28.998585 10272 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.007047  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:29.022562  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:29.065935  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:29.103463  6456 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:29.104391  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:28.998613 10462 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:29.001461 10491 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.125s
I20260709 12:26:28.991168 10394 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:29.025342 10432 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.112700 10394 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.113523 10394 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:29.012990  6383 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:29.009622 10476 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:26:29.036954  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:28.991407 10418 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:29.036180  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:29.116460  6384 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:29.036600  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:29.051663 10461 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:29.054440 10319 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:29.052533 10257 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.035961  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:29.117589  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 195581us
I20260709 12:26:28.962373 10315 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.119073  7319 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:29.118468  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.099671 10348 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:29.130640 10348 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:29.126121  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:29.118723 10272 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.127875  6383 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
W20260709 12:26:29.133618 10489 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:26:29.133725 10348 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.135072 10348 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:29.137395  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:29.115958 10209 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.143582  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.115567 10418 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:29.167073  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:29.091827 10436 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:26:29.168891  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:29.187330  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 276503us
I20260709 12:26:29.120767 10448 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:26:29.192631  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:29.212494  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.214701  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 160605us
I20260709 12:26:29.126976 10319 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.122459 10461 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.225026  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:29.225705  6383 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.131413 10257 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.243374  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.244585  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.244868  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.245695  6380 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:29.246201  6456 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:26:29.246101  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.246802  6382 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:29.247123  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.245711  6376 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:29.247944  6380 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 5.
I20260709 12:26:29.251205  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.251572  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:29.251856  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.254117  6385 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:29.254201  6384 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:29.257932  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:29.258745  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:29.259333  7332 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:29.260444  7332 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:29.261804  7332 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:26:29.262805  6204 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 141) took 29743 ms (client timeout 15000 ms). Trace:
I20260709 12:26:29.263919 10491 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:29.264573 10491 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:29.265059 10491 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:29.266810 10318 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.267532 10318 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:29.267887 10461 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.268141 10318 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.269081 10445 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.272289 10432 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.273648  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.274369  6384 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
W20260709 12:26:29.275490 10496 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:26:29.275949 10319 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.276649 10272 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.278640 10321 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.280721  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.283057  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.284070  6377 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:29.285106 10424 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.285740 10424 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:26:29.286720  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:29.286646 10418 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.288275  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.289252  6382 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:29.290686 10450 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.291287 10450 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:29.291649 10450 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:29.292024 10450 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:29.292758 10460 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.291929  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.294266 10321 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:29.294776 10321 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.295202 10321 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:29.297715 10334 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.299715  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.301609  6378 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:29.301542  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.170794 10209 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.310113  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.313627  6385 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:29.314050 10460 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:29.316123  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.317337  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.300864 10448 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:29.320785  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:29.324430  6383 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:29.326293  7331 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:29.328293  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.329690  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.338609  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.339932  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:29.263232  6204 rpcz_store.cc:269] 0709 12:25:59.520145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.520504 (+   359us) service_pool.cc:224] Handling call
0709 12:26:29.262707 (+29742203us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:22.379437 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:22.380098 (+   661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:22.380157 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:26:22.380182 (+    25us) tablet.cc:662] Decoding operations
0709 12:26:22.381379 (+  1197us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:22.381459 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:22.381483 (+    24us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:26:22.532227 (+150744us) tablet.cc:701] Row locks acquired
0709 12:26:22.553844 (+ 21617us) write_op.cc:260] PREPARE: finished
0709 12:26:22.578569 (+ 24725us) write_op.cc:270] Start()
0709 12:26:22.615594 (+ 37025us) write_op.cc:276] Timestamp: P: 1783599982571995 usec, L: 0
0709 12:26:22.615622 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:26:22.847211 (+231589us) log.cc:844] Serialized 5386 byte log entry
0709 12:26:23.714591 (+867380us) op_driver.cc:464] REPLICATION: finished
0709 12:26:24.216130 (+501539us) write_op.cc:301] APPLY: starting
0709 12:26:24.229444 (+ 13314us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:24.326806 (+ 97362us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:24.373181 (+ 46375us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:24.664971 (+291790us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:24.665422 (+   451us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:26:24.665470 (+    48us) write_op.cc:312] APPLY: finished
0709 12:26:24.665918 (+   448us) log.cc:844] Serialized 101 byte log entry
0709 12:26:24.828540 (+162622us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:24.841009 (+ 12469us) write_op.cc:454] Released schema lock
0709 12:26:24.842829 (+  1820us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:24.845873 (+  3044us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:24.845914 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":187133,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":428450,"apply.run_cpu_time_us":54774,"apply.run_wall_time_us":639931,"num_ops":14,"prepare.queue_time_us":54360,"prepare.run_cpu_time_us":8182,"prepare.run_wall_time_us":589535,"raft.queue_time_us":325246,"raft.run_cpu_time_us":31772,"raft.run_wall_time_us":454388,"replication_time_us":1066605,"thread_start_us":640646,"threads_started":5,"wal-append.queue_time_us":292063,"wal-append.run_cpu_time_us":11852,"wal-append.run_wall_time_us":2530416}]]}
I20260709 12:26:29.333561  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.334580  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:29.356540  6376 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:29.359817  6383 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:29.312239 10325 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:26:29.365222  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 389833us
I20260709 12:26:29.321806 10380 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.322570 10448 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.375417  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.379613  6378 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
W20260709 12:26:29.381345  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 212578us
W20260709 12:26:29.387972  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 197370us
I20260709 12:26:29.329608 10364 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:26:29.400928  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.411785  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:29.400262  6274 leader_election.cc:336] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.710s (SENT)
I20260709 12:26:29.413029  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:29.414048  6274 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.898s (SENT)
I20260709 12:26:29.414533  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:29.364928 10272 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:29.363358 10460 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:29.422138  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:29.369092 10325 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:29.433151  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.387245 10255 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:29.436523  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:29.430409  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 476) took 1065 ms. Trace:
I20260709 12:26:29.451783  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:29.450929  6459 rpcz_store.cc:276] 0709 12:26:28.365272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.365427 (+   155us) service_pool.cc:224] Handling call
0709 12:26:28.418329 (+ 52902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.461795 (+ 43466us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:28.593902 (+132107us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.882642 (+288740us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:28.882707 (+    65us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:28.882753 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.428467 (+545714us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b18003dcd20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419
0709 12:26:29.430082 (+  1615us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b5400345154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:29.430137 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.430336 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29527,"prepare.queue_time_us":297292,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":4294,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8985,"spinlock_wait_cycles":109902976,"thread_start_us":395786,"threads_started":2,"wal-append.queue_time_us":318936}
I20260709 12:26:29.427843  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:29.411062 10364 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:29.411057 10380 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.740347 10451 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.754652 10396 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:29.411137 10448 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:29.755208 10396 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:29.755613 10396 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:29.678998 10328 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.009s
I20260709 12:26:29.409899 10461 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.425904 10424 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:29.759577 10424 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:29.760573 10424 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:29.625470  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 209250us
I20260709 12:26:29.511168  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:29.720686  7331 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:29.691037  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 481) took 1129 ms. Trace:
I20260709 12:26:29.453330  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:29.765484  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 337733us
I20260709 12:26:29.765926  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:29.597616  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.695312 10450 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.764858  6455 rpcz_store.cc:276] 0709 12:26:28.562037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.562312 (+   275us) service_pool.cc:224] Handling call
0709 12:26:28.571194 (+  8882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.585988 (+ 14794us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:28.740856 (+154868us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.122039 (+381183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.122108 (+    69us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:29.122412 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.690020 (+567608us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b180028c960. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419
0709 12:26:29.690660 (+   640us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:29.690747 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.690943 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":238080,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":97,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":17470,"spinlock_wait_cycles":44218496,"thread_start_us":517209,"threads_started":2,"wal-append.queue_time_us":511916}
I20260709 12:26:29.822155  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:29.823865  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:29.586951  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 359698us
I20260709 12:26:29.437345 10325 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:29.751739  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2695) took 3801 ms. Trace:
I20260709 12:26:29.832801  6453 rpcz_store.cc:276] 0709 12:26:25.950425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.571757 (+621332us) service_pool.cc:224] Handling call
0709 12:26:26.586106 (+ 14349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.979717 (+1393611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.751395 (+1771678us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:29.751659 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712022912}
I20260709 12:26:29.715047  6381 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:29.714679  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 803) took 3374 ms. Trace:
I20260709 12:26:29.680724  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:29.424304 10272 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:29.641244 10281 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.006s
I20260709 12:26:29.835562  6450 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:29.479053  9846 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.620035 10416 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.516s	user 0.003s	sys 0.009s
W20260709 12:26:29.658303 10498 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:26:29.430889 10455 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:29.650274 10452 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.008s
W20260709 12:26:29.725865 10237 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.624s	user 0.010s	sys 0.003s
I20260709 12:26:29.433578 10490 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:29.684110 10357 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.759473 10407 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.644321 10334 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.480044 10245 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.506835 10318 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.470901 10278 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.752388 10271 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.463445 10460 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:29.723701 10491 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.835140  7316 rpcz_store.cc:276] 0709 12:26:26.340341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.340509 (+   168us) service_pool.cc:224] Handling call
0709 12:26:26.340794 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.286886 (+1946092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.714360 (+1427474us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54006801d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:29.714601 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4153035520}
I20260709 12:26:29.481112 10394 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.304047 10394 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.472934 10255 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.506150 10505 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.442s
I20260709 12:26:29.539129 10506 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.438s
I20260709 12:26:29.787623 10364 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:26:29.796054 10328 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.306509 10255 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:31.308300 10328 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.643779 10209 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.631464 10415 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:26:30.178398 10520 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
W20260709 12:26:30.222020 10517 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:29.866928 10272 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:30.337576 10452 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.207s	user 0.033s	sys 0.033s
I20260709 12:26:31.316397 10364 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.318980 10364 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.320827 10272 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:30.044584 10495 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.010s	sys 0.010s
W20260709 12:26:31.321875 10495 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.021s	user 0.010s	sys 0.016s
I20260709 12:26:30.339582 10450 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:29.758030 10461 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:30.776008 10237 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.325585 10237 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:30.981221 10448 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.226s	user 0.009s	sys 0.002s
I20260709 12:26:31.334642 10448 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.337800 10448 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:30.921053 10325 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.050s	user 0.012s	sys 0.000s
I20260709 12:26:31.341359 10325 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.344199 10325 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:31.080875 10424 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.001s	sys 0.002s
W20260709 12:26:31.345237 10424 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.002s	sys 0.002s
I20260709 12:26:29.900354 10396 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.347793 10396 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:31.004195 10491 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.354068 10257 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.739858 10383 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.360423 10272 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:31.362742 10439 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.169s	user 0.009s	sys 0.024s
W20260709 12:26:31.363284 10439 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.169s	user 0.009s	sys 0.025s
I20260709 12:26:31.363665 10439 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:31.364890 10439 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.366655 10288 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.367504 10288 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:29.639189 10250 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:31.369755 10250 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.663194 10232 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.371160 10455 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:31.372462 10455 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.373172 10288 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:29.902354 10380 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:31.375682 10416 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.378775 10490 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:29.614105 10509 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:29.790110 10444 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:29.827178 10451 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.831599 10319 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:31.379890 10221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.718s	user 0.007s	sys 0.016s
W20260709 12:26:31.381677 10335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.675s	user 0.002s	sys 0.018s
I20260709 12:26:31.370390 10250 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:31.392529 10455 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:29.752467 10321 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:29.893381 10281 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:31.395248 10221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.733s	user 0.008s	sys 0.016s
I20260709 12:26:31.395360 10451 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.396090 10319 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:31.384784  9846 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:31.399395 10335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.695s	user 0.002s	sys 0.019s
I20260709 12:26:29.847597 10477 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.388932 10416 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:31.409512 10319 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.406298 10321 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:31.406334 10451 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.408147 10281 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.398936 10490 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.411936 10262 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:31.402669 10447 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.354s	user 0.010s	sys 0.021s
W20260709 12:26:31.413393 10447 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.369s	user 0.010s	sys 0.022s
I20260709 12:26:31.414052 10244 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:31.412300  9846 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:31.417006 10477 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.428887 10250 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:31.412783 10400 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.874s	user 0.024s	sys 0.010s
W20260709 12:26:31.430852 10400 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.904s	user 0.024s	sys 0.010s
W20260709 12:26:31.429838  7314 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.766s	user 0.004s	sys 0.019s
W20260709 12:26:31.432180  7314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.769s	user 0.005s	sys 0.019s
I20260709 12:26:31.434729  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 613) took 2346 ms. Trace:
I20260709 12:26:31.435031  7314 rpcz_store.cc:276] 0709 12:26:29.088360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.662042 (+573682us) service_pool.cc:224] Handling call
0709 12:26:29.662386 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.662652 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:29.663018 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.434205 (+1771187us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.434391 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.434435 (+    44us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:31.434510 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.434575 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.434686 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2118,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":74,"raft.queue_time_us":193,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"spinlock_wait_cycles":230912,"thread_start_us":1770444,"threads_started":2,"wal-append.queue_time_us":1307120}
I20260709 12:26:31.437239 10250 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.439761 10400 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.440258 10400 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.443575 10400 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.466308 10470 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:31.467020 10470 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:31.467962 10470 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.468899 10470 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:31.536511  6377 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.013s	sys 0.021s
W20260709 12:26:31.537410  6377 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.978s	user 0.013s	sys 0.021s
I20260709 12:26:31.538890  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2041) took 2263 ms. Trace:
I20260709 12:26:31.539410  6377 rpcz_store.cc:276] 0709 12:26:29.274969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.284740 (+  9771us) service_pool.cc:224] Handling call
0709 12:26:29.285143 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.285454 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:29.285875 (+   421us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.537965 (+2252090us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.538105 (+   140us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:31.538253 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.538370 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.538700 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12623,"prepare.queue_time_us":209,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":97468,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":12125,"thread_start_us":2251451,"threads_started":2,"wal-append.queue_time_us":355301,"wal-append.run_cpu_time_us":1611,"wal-append.run_wall_time_us":1128343}
I20260709 12:26:31.541643  6379 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:26:31.542506  6379 pending_rounds.cc:85] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 8
I20260709 12:26:31.542860  6379 pending_rounds.cc:107] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:26:31.543346  6379 raft_consensus.cc:2889] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:31.610133 10417 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:31.610870 10417 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:31.611351  9985 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.614324 10417 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.618407 10417 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:31.628973 10434 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.629686 10434 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.645553 10417 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.646225 10416 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.646588 10417 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.647500 10416 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.651257 10416 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.658862  6380 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:31.662006  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 496) took 3540 ms. Trace:
I20260709 12:26:31.663101  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.663709 10417 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.664458  7314 raft_consensus.cc:2368] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11.
I20260709 12:26:31.651151 10262 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.667470 10262 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.671654 10262 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:31.663159  6379 rpcz_store.cc:276] 0709 12:26:28.121955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.137140 (+ 15185us) service_pool.cc:224] Handling call
0709 12:26:28.137466 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.544260 (+3406794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.661532 (+117272us) spinlock_profiling.cc:243] Waited 3.4 s on lock 0x7b54003e94d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:31.661910 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7825861376}
I20260709 12:26:31.681977 10319 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.683038 10319 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:31.683828  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.685982 10319 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:31.688149  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.688235 10490 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.689505 10490 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.692615 10490 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:31.692677  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.693910  7314 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:31.700284  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:31.687784 10319 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:31.703070  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:31.704394  7316 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:31.704236 10319 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.706956 10319 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:29.614707  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 200678us
W20260709 12:26:31.713936  6275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.276s	user 0.011s	sys 0.014s
W20260709 12:26:31.714442  6275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.277s	user 0.011s	sys 0.014s
I20260709 12:26:31.723122  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:31.725205  7316 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:31.726507  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:31.727365  6452 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:31.727732  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.724020  6274 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.903s (SENT)
W20260709 12:26:31.729020  6452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2174) took 2840 ms (client timeout 2484 ms). Trace:
I20260709 12:26:31.729257  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:31.729507  6452 rpcz_store.cc:269] 0709 12:26:28.888253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.137066 (+248813us) service_pool.cc:224] Handling call
0709 12:26:31.728764 (+2591698us) spinlock_profiling.cc:243] Waited 2.59 s on lock 0x7b54001e32d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:31.728944 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5953433472}
I20260709 12:26:31.735832  6377 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:26:31.737123  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 477) took 3371 ms. Trace:
I20260709 12:26:31.737180  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:31.737788  6454 rpcz_store.cc:276] 0709 12:26:28.365558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.369796 (+  4238us) service_pool.cc:224] Handling call
0709 12:26:28.370127 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.583439 (+213312us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:28.686982 (+103543us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.784157 (+ 97175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:28.784218 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:28.784271 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.736511 (+2952240us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54002c7754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:31.736586 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.736851 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11265,"prepare.queue_time_us":209052,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":36,"raft.queue_time_us":4599,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":39228,"spinlock_wait_cycles":489980288,"thread_start_us":197992,"threads_started":2,"wal-append.queue_time_us":210809,"wal-append.run_cpu_time_us":2249,"wal-append.run_wall_time_us":1235342}
I20260709 12:26:31.741631  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2707) took 3288 ms. Trace:
I20260709 12:26:31.744560  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:31.745012  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2701) took 5345 ms. Trace:
I20260709 12:26:31.745285  6454 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:31.745436  6458 rpcz_store.cc:276] 0709 12:26:26.398953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.730476 (+1331523us) service_pool.cc:224] Handling call
0709 12:26:27.730935 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.205409 (+474474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.744353 (+3538944us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54000c12d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:31.744773 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":916593408}
I20260709 12:26:31.742218  6457 rpcz_store.cc:276] 0709 12:26:28.452982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.453141 (+   159us) service_pool.cc:224] Handling call
0709 12:26:28.589628 (+136487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.596307 (+  6679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.741086 (+3144779us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54001c1e50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:31.741378 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312918400}
I20260709 12:26:31.748140  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:31.749997  6457 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:31.752511  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:31.753672  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:31.753932  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:31.754652  6453 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.755005 10400 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:31.755553 10400 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.755919 10400 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.758935  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:31.759859  6377 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10. Re-sending same reply.
I20260709 12:26:31.760444 10455 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:31.761013 10455 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.761180  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:31.761478 10455 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.759053  6455 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:26:31.763090  7288 connection.cc:256] RPC call timeout handler was delayed by -1.3859170320001795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1564s for a response
I20260709 12:26:31.763288 10451 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:31.763568  7288 connection.cc:256] RPC call timeout handler was delayed by -1.3274170320000849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
I20260709 12:26:31.763911 10451 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:31.763917  7288 connection.cc:256] RPC call timeout handler was delayed by -1.2617170319999786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 12:26:31.723740  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2166) took 2849 ms (client timeout 1751 ms). Trace:
I20260709 12:26:31.727413  7316 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:31.773509  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2163) took 2901 ms. Trace:
W20260709 12:26:31.766670  6451 rpcz_store.cc:269] 0709 12:26:28.873750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.954289 (+ 80539us) service_pool.cc:224] Handling call
0709 12:26:31.723269 (+2768980us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54000eae54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:31.723466 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392878336}
I20260709 12:26:31.774106  6455 rpcz_store.cc:276] 0709 12:26:28.872582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.871650 (+999068us) service_pool.cc:224] Handling call
0709 12:26:31.750142 (+1878492us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.762694 (+ 12552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.773433 (+ 10739us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.777426  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:31.777570  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.779500  6455 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:31.762059  6377 raft_consensus.cc:2376] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10. Re-sending same reply.
W20260709 12:26:31.783993  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.784538  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.784972  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.784796  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:31.774012 10455 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.787766 10244 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.788821  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 609) took 2793 ms. Trace:
I20260709 12:26:31.789032 10455 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.788537 10244 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.792164 10335 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:31.792645 10335 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.792989 10335 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:31.794637 10215 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:31.795038 10544 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.795507 10215 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.795763 10544 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.797819 10544 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.789273  7329 rpcz_store.cc:276] 0709 12:26:28.995049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.995377 (+   328us) service_pool.cc:224] Handling call
0709 12:26:28.995728 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:28.999429 (+  3701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:28.999478 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.013525 (+ 14047us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.315035 (+301510us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:29.315078 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.315155 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.788310 (+2473155us) raft_consensus.cc:1685] finished
0709 12:26:31.788351 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.788596 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3211,"raft.queue_time_us":3337,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15894,"thread_start_us":301117,"threads_started":1,"wal-append.queue_time_us":425969}
I20260709 12:26:31.799734  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:31.801856  7335 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:31.802700 10373 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:31.803191  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.804260  7335 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:31.805405  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.806150  7335 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:31.772619  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 487) took 2719 ms. Trace:
I20260709 12:26:31.808360  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:31.807672  6452 rpcz_store.cc:276] 0709 12:26:29.053497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.771256 (+2717759us) service_pool.cc:224] Handling call
0709 12:26:31.771759 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.772207 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.772232 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.772290 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:31.772352 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.772410 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.772537 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.770618  6454 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:31.767918  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.817224  9783 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.819154 10477 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.810819 10461 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.766476 10451 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.815348  7316 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:31.825603 10215 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:31.825928  7316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 615) took 1977 ms (client timeout 2117 ms). Trace:
W20260709 12:26:31.826403  7316 rpcz_store.cc:269] 0709 12:26:31.761852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.767488 (+  5636us) service_pool.cc:224] Handling call
0709 12:26:31.825851 (+ 58363us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.827777  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.829075  7316 raft_consensus.cc:2410] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:31.791641 10244 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.844290  7314 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:31.844908  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2177) took 2870 ms. Trace:
I20260709 12:26:31.845377  6454 rpcz_store.cc:276] 0709 12:26:28.962253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.769522 (+2807269us) service_pool.cc:224] Handling call
0709 12:26:31.770150 (+   628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.832457 (+ 62307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.832677 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:31.847577  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.850908  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.851608  7314 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:31.850266  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:31.853613  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.855549  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:31.857561  6454 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:31.815167 10544 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:31.860355 10417 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.856794 10477 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:31.861223 10417 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.861644 10477 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.864223  7335 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:31.865733  7335 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:31.865257 10400 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.870249  7329 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:31.871683  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 491) took 2173 ms. Trace:
I20260709 12:26:31.875294  6377 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 6: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:31.875401  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:31.879483 10544 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.881934  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:31.848981 10461 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.876895  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:31.929651  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85607.7us
I20260709 12:26:31.937515  6454 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:31.872129  6451 rpcz_store.cc:276] 0709 12:26:29.686328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.775056 (+2088728us) service_pool.cc:224] Handling call
0709 12:26:31.775517 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.775898 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:31.776309 (+   411us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.859538 (+ 83229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.859587 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:31.859677 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.859728 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.859824 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16663,"prepare.queue_time_us":552,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":11688,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":259,"spinlock_wait_cycles":23296,"thread_start_us":71065,"threads_started":1,"wal-append.queue_time_us":98399}
I20260709 12:26:31.945115 10400 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.946419 10417 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.947381 10477 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.949572 10335 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.951830 10095 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.952504 10095 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.953424 10335 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:31.957250  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.499s	user 0.010s	sys 0.068s
I20260709 12:26:31.958949  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2040) took 2684 ms. Trace:
I20260709 12:26:31.959168  6453 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:31.960589 10544 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:31.959440  6384 rpcz_store.cc:276] 0709 12:26:29.274679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.274994 (+   315us) service_pool.cc:224] Handling call
0709 12:26:29.275295 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.275521 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:29.276180 (+   659us) log.cc:844] Serialized 14 byte log entry
0709 12:26:29.556889 (+280709us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.665286 (+2108397us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b18004b49c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:26:31.707517 (+ 42231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.858619 (+151102us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.858694 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:31.871107 (+ 12413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.927757 (+ 56650us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.958726 (+ 30969us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36287,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":138,"raft.queue_time_us":197862,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":1033,"spinlock_wait_cycles":8703232,"thread_start_us":465401,"threads_started":4,"wal-append.queue_time_us":436125,"wal-append.run_cpu_time_us":1523,"wal-append.run_wall_time_us":1001565}
I20260709 12:26:31.961267 10461 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:31.962234  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:31.961194  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:31.963737 10461 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:31.964309 10288 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:31.964804  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:31.964967 10288 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:31.967456 10288 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:31.968088  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2025) took 2842 ms. Trace:
I20260709 12:26:31.970063  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2049) took 2633 ms. Trace:
I20260709 12:26:31.970721  7314 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:31.971616  7316 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:31.964989  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:31.978767  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44959.9us
W20260709 12:26:31.981528  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.524s	user 0.012s	sys 0.069s
I20260709 12:26:31.968571  6381 rpcz_store.cc:276] 0709 12:26:29.125233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.125413 (+   180us) service_pool.cc:224] Handling call
0709 12:26:29.125724 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.834737 (+709013us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.967479 (+2132742us) spinlock_profiling.cc:243] Waited 589 ms on lock 0x7b5400109ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:31.967874 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1354799872}
I20260709 12:26:31.970525  6376 rpcz_store.cc:276] 0709 12:26:29.336302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.357397 (+ 21095us) service_pool.cc:224] Handling call
0709 12:26:29.357876 (+   479us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.358306 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.358344 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.359104 (+   760us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.641463 (+282359us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:29.641533 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.643001 (+  1468us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.969538 (+2326537us) raft_consensus.cc:1685] finished
0709 12:26:31.969586 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:31.969811 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135406,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":47526,"thread_start_us":225331,"threads_started":1,"wal-append.queue_time_us":634938}
W20260709 12:26:31.986313  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126906us
I20260709 12:26:31.992574  6377 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:31.994035  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:31.993395  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:31.996858 10416 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:31.998514 10416 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:31.996879  6351 connection.cc:256] RPC call timeout handler was delayed by -1.7848818510001365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:26:32.001348  6351 connection.cc:256] RPC call timeout handler was delayed by -1.7155818510000245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15910000000000002s for a response
W20260709 12:26:32.002056  6351 connection.cc:256] RPC call timeout handler was delayed by -1.6903818509999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
W20260709 12:26:32.002450  6351 connection.cc:256] RPC call timeout handler was delayed by -1.6768818509999619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1634s for a response
W20260709 12:26:32.002795  6351 connection.cc:256] RPC call timeout handler was delayed by -1.6273818509998819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 12:26:32.003084  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5769818510002551s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17450000000000002s for a response
W20260709 12:26:32.003381  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5715818510002464s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:26:32.003710  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5517818510002144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1773s for a response
W20260709 12:26:32.004007  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5328818510001838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1794s for a response
W20260709 12:26:32.004385  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5319818510001824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 12:26:32.004715  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5319818510001824s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1795s for a response
W20260709 12:26:32.004987  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5085818510001445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 12:26:32.005286  6351 connection.cc:256] RPC call timeout handler was delayed by -1.5085818510001445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1821s for a response
W20260709 12:26:32.005576  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4392818510000325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 12:26:32.005919  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4392818510000325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
I20260709 12:26:32.006206  6381 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:32.006201  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4320818510000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1906s for a response
W20260709 12:26:32.007541  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4275818510000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:26:32.007915  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4275818510000136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 12:26:32.008219  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4176818509999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
W20260709 12:26:32.008483  6351 connection.cc:256] RPC call timeout handler was delayed by -1.3213818509998418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2029s for a response
W20260709 12:26:32.008735  6351 connection.cc:256] RPC call timeout handler was delayed by -1.3033818510002675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2049s for a response
W20260709 12:26:32.008976  6351 connection.cc:256] RPC call timeout handler was delayed by -1.2997818510002617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 12:26:32.009227  6351 connection.cc:256] RPC call timeout handler was delayed by -1.2772818510002253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20779999999999998s for a response
W20260709 12:26:32.009476  6351 connection.cc:256] RPC call timeout handler was delayed by -1.252981851000186s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
W20260709 12:26:32.009724  6351 connection.cc:256] RPC call timeout handler was delayed by -1.2511818510001831s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21070000000000003s for a response
W20260709 12:26:32.009966  6351 connection.cc:256] RPC call timeout handler was delayed by -1.2223818510001365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21389999999999998s for a response
W20260709 12:26:32.010219  6351 connection.cc:256] RPC call timeout handler was delayed by -1.2178818510001292s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21440000000000003s for a response
W20260709 12:26:32.010464  6351 connection.cc:256] RPC call timeout handler was delayed by -1.1881818510000812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2177s for a response
I20260709 12:26:32.011348  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.012100  6381 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:32.012555  6351 connection.cc:256] RPC call timeout handler was delayed by -1.142281851000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
W20260709 12:26:32.013016  6351 connection.cc:256] RPC call timeout handler was delayed by -1.142281851000007s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
I20260709 12:26:32.019848  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.021747  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.022724  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.023102  6376 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:32.023335  6452 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.021965  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.021844  6457 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.020960  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.021926  6453 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.029119  6451 raft_consensus.cc:2368] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:32.029631  6351 leader_election.cc:400] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11.
I20260709 12:26:32.032131  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.032999  6451 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:32.033713  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.034188  6451 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.036490  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.036824  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.039950  6453 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:32.040813  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.041379  6453 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:32.042169  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.042604  6455 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:32.043411  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.044411  6453 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:32.047143  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.048389  7317 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:32.048836 10400 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:32.049350 10400 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:32.050565 10400 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:32.053154  6348 connection.cc:256] RPC call timeout handler was delayed by -2.3111718569998629s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
I20260709 12:26:32.054798 10477 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.905910  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:32.055562 10477 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.056011  7331 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
W20260709 12:26:32.056759  7331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 614) took 2225 ms (client timeout 2202 ms). Trace:
I20260709 12:26:32.057524 10288 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:32.057516  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.038269  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.066421 10288 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.066445  6376 raft_consensus.cc:2410] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:26:32.068379  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.069231 10288 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:32.069410  6376 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:26:32.070235  7317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 611) took 3073 ms (client timeout 1825 ms). Trace:
W20260709 12:26:32.071238  7317 rpcz_store.cc:269] 0709 12:26:28.996927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.997235 (+   308us) service_pool.cc:224] Handling call
0709 12:26:32.049795 (+3052560us) spinlock_profiling.cc:243] Waited 2.92 s on lock 0x7b5400438ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:32.070138 (+ 20343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6727219456}
W20260709 12:26:32.062821  7331 rpcz_store.cc:269] 0709 12:26:29.831251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.831797 (+   546us) service_pool.cc:224] Handling call
0709 12:26:32.056629 (+2224832us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.072446  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.077620  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.079059  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.080801  6455 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.081499  6457 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:26:32.083940  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 70329.3us
I20260709 12:26:32.087170  6384 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:32.097173  6381 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:32.098959  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.099927  6381 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:32.100831  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.101480  6381 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:32.102568  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.103709  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:32.104503  6385 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.496s	user 0.015s	sys 0.016s
I20260709 12:26:32.104465  6381 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:32.105616  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.105883 10095 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:32.107434  6381 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:26:32.107456  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:32.108680 10262 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:32.110261  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:32.111112 10262 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:32.113116  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.109493  6381 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:32.117835  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:32.120520  6455 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
W20260709 12:26:32.121853  6385 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.513s	user 0.016s	sys 0.017s
I20260709 12:26:32.121654  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.122658  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.123528  6384 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.125880  6451 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:32.129344  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2048) took 2793 ms. Trace:
I20260709 12:26:32.135931  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:32.140028  6381 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:32.144034 10288 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:32.145989  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.132201  6385 rpcz_store.cc:276] 0709 12:26:29.336025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.365596 (+ 29571us) service_pool.cc:224] Handling call
0709 12:26:29.410359 (+ 44763us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.410593 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:29.430788 (+ 20195us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.128733 (+2697945us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.128851 (+   118us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.128940 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.129062 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.129288 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25092,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":85210,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"thread_start_us":2695733,"threads_started":2,"wal-append.queue_time_us":614340,"wal-append.run_cpu_time_us":1877,"wal-append.run_wall_time_us":1149242}
I20260709 12:26:32.149122  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.149827  6385 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:32.140484  6377 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:26:32.156284  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64574.5us
I20260709 12:26:32.157052 10451 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:32.157619 10451 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:32.158617 10451 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.162515 10334 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:32.163213 10288 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:32.163808 10288 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.164824 10288 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.156661 10262 raft_consensus.cc:2764] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:26:32.167572  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.168673 10544 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:32.169091 10544 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.169443 10544 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.169363 10417 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:32.171010 10417 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:32.171449 10417 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.174714  6385 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:32.179410  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.180388  6377 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.180210  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.182047  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.182056  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.184742  6451 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:32.183586 10262 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:26:32.186409 10262 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.187207 10417 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.187968 10262 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.190694  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:32.193063  6384 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:32.194751  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.195500  6384 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:32.194245 10417 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.201025  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.201834  6378 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:32.202755  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.203275  6378 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:32.211714  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.211910 10544 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.214131 10544 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:32.216502  6376 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:32.201666  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.221196  6378 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
W20260709 12:26:32.222028  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.221117  6385 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
W20260709 12:26:32.228328  6459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.438s	user 0.006s	sys 0.032s
W20260709 12:26:32.245591  6459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.455s	user 0.007s	sys 0.032s
I20260709 12:26:32.246117  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.246738  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2161) took 3375 ms. Trace:
I20260709 12:26:32.247426  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.247112  6459 rpcz_store.cc:276] 0709 12:26:28.871850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.788916 (+917066us) service_pool.cc:224] Handling call
0709 12:26:29.789338 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.789606 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:29.789931 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 12:26:32.246110 (+2456179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.246211 (+   101us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:32.246289 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.246401 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.246673 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33003,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"thread_start_us":2455752,"threads_started":1,"wal-append.queue_time_us":1596081}
I20260709 12:26:32.248351  6385 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:32.229449 10417 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:32.214936  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:32.252718  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56596.3us
I20260709 12:26:32.254608 10461 raft_consensus.cc:2764] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:26:32.255394 10461 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:32.257012 10461 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:26:32.230840  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 48223.3us
I20260709 12:26:32.232106 10262 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.260767 10262 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.262585 10544 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:32.263414 10544 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:32.264966  6380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2035) took 3014 ms (client timeout 2077 ms). Trace:
I20260709 12:26:32.264900 10544 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:32.265538  6380 rpcz_store.cc:269] 0709 12:26:29.250307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.250973 (+   666us) service_pool.cc:224] Handling call
0709 12:26:32.264614 (+3013641us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:32.264861 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5528315264}
I20260709 12:26:32.267290  6380 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.268693  6380 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:32.269194  7332 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.271464  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.272547  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 815) took 3259 ms. Trace:
I20260709 12:26:32.272919  7332 rpcz_store.cc:276] 0709 12:26:29.013330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.262433 (+249103us) service_pool.cc:224] Handling call
0709 12:26:32.049982 (+2787549us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.269623 (+219641us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.271749 (+  2126us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400438ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:32.272347 (+   598us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b5400438ad0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:32.272477 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6817998336}
I20260709 12:26:32.275537  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.235074  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.240828 10477 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:32.243268  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:32.243675  6377 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:32.277010  6376 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:32.277726  6377 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:32.244643  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.278846  6385 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:32.244752  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.279702  6384 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:32.282657  6455 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:32.284332  6378 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:32.285415  6383 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:32.287155 10400 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:32.287750 10400 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:32.288725 10400 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:32.302836  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.304447  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:32.304497  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:32.305768 10476 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.302523  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:32.307296 10476 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:32.309260  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:32.312881  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:32.313768  7331 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.320045  6376 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:32.315450 10335 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:26:32.326889 10394 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:32.327616 10394 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.332816 10434 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:32.340546  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:32.342605 10558 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:26:32.343333  7317 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEARNER]: Advancing to term 12
I20260709 12:26:32.347239  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 486) took 3475 ms. Trace:
I20260709 12:26:32.347750  6456 rpcz_store.cc:276] 0709 12:26:28.872083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.828579 (+956496us) service_pool.cc:224] Handling call
0709 12:26:29.829006 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.829444 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.829484 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.830062 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.247373 (+2417311us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.247460 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.247573 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.346803 (+ 99230us) raft_consensus.cc:1685] finished
0709 12:26:32.346829 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.347029 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":212997,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":25905,"thread_start_us":32894,"threads_started":1,"wal-append.queue_time_us":31835}
I20260709 12:26:32.349566  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:32.317327  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63505.9us
I20260709 12:26:32.387225  7331 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
W20260709 12:26:32.400393  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109505us
I20260709 12:26:32.405225  6348 leader_election.cc:400] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:32.405615  6348 leader_election.cc:403] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 12:26:32.409767  6425 connection.cc:256] RPC call timeout handler was delayed by -2.3827652380000472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24380000000000002s for a response
W20260709 12:26:32.411581  6425 connection.cc:256] RPC call timeout handler was delayed by -2.1770963430003576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
W20260709 12:26:32.411887  6425 connection.cc:256] RPC call timeout handler was delayed by -2.0791703140002937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:26:32.412133  6425 connection.cc:256] RPC call timeout handler was delayed by -2.0144472490001135s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
W20260709 12:26:32.412391  6425 connection.cc:256] RPC call timeout handler was delayed by -1.8190703140003279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:26:32.412721  6425 connection.cc:256] RPC call timeout handler was delayed by -1.8190703140003279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:26:32.412983  6425 connection.cc:256] RPC call timeout handler was delayed by -1.8001703140002974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18460000000000001s for a response
I20260709 12:26:32.413086  7317 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:32.413276  6425 connection.cc:256] RPC call timeout handler was delayed by -1.7444765000000189s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21190000000000003s for a response
W20260709 12:26:32.413578  6425 connection.cc:256] RPC call timeout handler was delayed by -1.7227703140001722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
W20260709 12:26:32.413851  6425 connection.cc:256] RPC call timeout handler was delayed by -1.6507703140000558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
W20260709 12:26:32.414124  6425 connection.cc:256] RPC call timeout handler was delayed by -1.6363476129999981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1877s for a response
W20260709 12:26:32.417073  6425 connection.cc:256] RPC call timeout handler was delayed by -1.592270314000416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 12:26:32.417456  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5868703140004072s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
W20260709 12:26:32.417759  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5661703140003738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 12:26:32.418049  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5337703140003214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 12:26:32.418344  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5131162790003145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 12:26:32.418684  6425 connection.cc:256] RPC call timeout handler was delayed by -1.471670314000221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
W20260709 12:26:32.418974  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4123182930002258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21150000000000002s for a response
W20260709 12:26:32.419257  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4083491750002395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 12:26:32.419533  6425 connection.cc:256] RPC call timeout handler was delayed by -1.2884319300001152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
W20260709 12:26:32.419822  6425 connection.cc:256] RPC call timeout handler was delayed by -1.1561319299999013s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
W20260709 12:26:32.420101  6425 connection.cc:256] RPC call timeout handler was delayed by -1.0779504490001273s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 12:26:32.420904  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:32.428646  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.431157 10560 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:26:32.424685 10477 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:32.426797 10554 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:32.427917 10490 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.438788 10554 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.440084 10417 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:32.447281 10417 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.457764  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.459110 10262 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:32.459764 10262 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.461879  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.463171 10477 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:32.463733 10477 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.464756 10477 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.459224 10417 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.480993  6202 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 4 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 6 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.483649  6202 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 9, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 9 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.485204  6202 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 5 to 11, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 11 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.460744 10262 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.486847  6202 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 5, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 5 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:32.491074 10554 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:32.489542  6202 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 10 to 11, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 11 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:32.519773  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 99002.9us
I20260709 12:26:32.520673 10416 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:32.521253  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.524088 10544 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:32.524655  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.525967  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.531658 10394 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:32.526376  6450 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 2.662s	user 0.023s	sys 0.010s
I20260709 12:26:32.551517 10288 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:32.552181 10288 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.553571 10288 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.555531 10554 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:32.556044  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.556502 10554 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:32.557092  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.558357 10095 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:32.558784 10490 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:32.558993 10095 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:32.559341  7319 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 607) took 3565 ms (client timeout 2211 ms). Trace:
I20260709 12:26:32.558127 10554 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:32.559855  7319 rpcz_store.cc:269] 0709 12:26:28.994027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.994151 (+   124us) service_pool.cc:224] Handling call
0709 12:26:32.558989 (+3564838us) spinlock_profiling.cc:243] Waited 1.22 ms on lock 0x7b5400566b54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:32.559145 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2803072}
I20260709 12:26:32.560282 10095 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.559260 10490 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.562273 10288 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.562680 10490 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.558230  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.562589  6450 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:32.568923  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.571285  6458 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:32.572042  6385 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:32.570875  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.579665  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.580628  6382 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:32.574415  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.581624  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.579679  6379 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:32.481020 10417 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.587589  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:32.599362  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54243.4us
W20260709 12:26:32.589124  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.600662  6275 connection.cc:256] RPC call timeout handler was delayed by -1.8881710000000567s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1887s for a response
W20260709 12:26:32.601047  6275 connection.cc:256] RPC call timeout handler was delayed by -1.3224123909999435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260709 12:26:32.590103  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 879685us
W20260709 12:26:32.602735  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44599.8us
W20260709 12:26:32.603479  6274 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.105s (SENT)
W20260709 12:26:32.590610  6450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 490) took 2941 ms (client timeout 2185 ms). Trace:
I20260709 12:26:32.603552  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:32.604067  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:32.604358  7319 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:26:32.604166  6450 rpcz_store.cc:269] 0709 12:26:29.642704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.680389 (+ 37685us) service_pool.cc:224] Handling call
0709 12:26:32.577832 (+2897443us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.605986 10555 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:32.606621 10555 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:32.607666 10555 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:32.590651  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 2.8174e+06us
I20260709 12:26:32.591197  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.611601  6378 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:32.593322  9976 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.594664 10220 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:32.595541  6381 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:26:32.597172  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2168) took 3708 ms. Trace:
I20260709 12:26:32.587793 10531 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.599980 10095 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:32.585227  6202 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 1 to 4, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:32.603704  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.614403  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.615556  6458 rpcz_store.cc:276] 0709 12:26:28.874395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.752565 (+2878170us) service_pool.cc:224] Handling call
0709 12:26:32.571100 (+818535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.571793 (+   693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.577657 (+  5864us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54000fc650. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:32.582253 (+  4596us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881380224}
I20260709 12:26:32.622015  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.622864  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.623986  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.624281  6377 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:32.624774  6384 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:32.623994  6383 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:32.628521 10531 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.631215 10531 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:32.628930  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.621408  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.632622  6379 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:32.634508  6378 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:32.631876 10505 consensus_queue.cc:1243] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 11.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:32.639569  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.639703  6385 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:32.643157  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.639266  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:32.720685  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.704749  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:32.734377  6382 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:32.664750  6304 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:32.708633  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.714344  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:32.757753  7316 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:32.670184  6308 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:32.669934  6458 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:32.644304  6378 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:32.669302  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:32.751999  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.752759  6377 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:32.764709  6381 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:32.663319  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:32.693143  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 90944.3us
I20260709 12:26:32.641636 10554 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:32.667891  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.663941  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:32.758430  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.676049  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.659475  6202 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 10, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 10 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.769358  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:32.770437  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.670579 10315 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:26:32.772758  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:32.767484  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:32.784610  6202 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 5 to 7, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 7 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.661008 10490 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.797808  6455 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:32.792773  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.801594  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:32.768085 10554 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.802371  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 133172us
I20260709 12:26:32.811683  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:32.808775  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 175390us
I20260709 12:26:32.786152 10394 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.653596  9976 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.814301  6305 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:32.793660 10319 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:32.808270 10262 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:32.809129 10315 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:32.812204 10554 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:32.826637  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 223207us
I20260709 12:26:32.822045  7316 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:32.824141 10581 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:32.810366 10373 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:32.817847  6202 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 6 to 10, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 10 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.842238  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.842638 10465 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:26:32.843809  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.844551 10373 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:32.845685 10373 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.847085  9976 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:32.848212 10574 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:32.852320 10490 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:32.843901  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:32.847267 10569 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:32.855778  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.820785 10477 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:32.837893  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:32.860109 10477 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:32.837025  6202 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:32.862903 10477 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:32.865760  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:32.881929  6455 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.890858  6303 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:32.891659 10095 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:32.890889 10315 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.898584  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.898342  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.899490  6305 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:32.904798  6378 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:32.905102 10476 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.907079  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.907776  6302 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:32.908360 10506 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:32.909703  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:32.911036  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.911378  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:32.910986  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:32.837530 10262 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:32.921424  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 153632us
I20260709 12:26:32.853638 10417 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:32.938848  6304 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:26:32.911394 10571 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.274s	user 0.000s	sys 0.003s
W20260709 12:26:32.954692  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 75408.7us
I20260709 12:26:32.955947  6455 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:26:32.957535  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 77746.2us
I20260709 12:26:32.958771  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:32.960692  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.961165 10480 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:32.962725 10505 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:32.963093 10505 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:32.964082 10505 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.964593 10461 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:32.965250 10461 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:32.961608  6377 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:32.966204  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.967808  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.968407  6382 raft_consensus.cc:2376] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11. Re-sending same reply.
I20260709 12:26:32.965935  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:32.972007  6450 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:32.973301  7316 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:32.974290  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.976089  6308 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 6: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:32.934340 10581 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:32.865648 10416 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.091s
I20260709 12:26:32.997778  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:32.998433 10417 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:32.998697 10480 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:32.999254  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:32.999459 10232 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:33.000063 10480 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:32.978962  6202 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 6 to 7, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 7 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:33.000458 10480 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:33.000783  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.670s (SENT) [suppressed 49 similar messages]
I20260709 12:26:33.001497 10480 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:33.002132  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.979712  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:33.002846  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.002008  6202 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 4 to 8, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 8 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:33.003389  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.982504 10262 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:32.983507  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:33.005403  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.984299  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.984982  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.006381  6300 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.985244  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.987278  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:32.987957 10335 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:32.990638  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.965446 10505 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:32.994784  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:32.995070 10461 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:32.997629  9976 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:32.970095 10531 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 12:26:33.028460  6274 connection.cc:256] RPC call timeout handler was delayed by -2.9228482710000208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15510000000000002s for a response
W20260709 12:26:33.029026  6274 connection.cc:256] RPC call timeout handler was delayed by -2.7941482709998127s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
W20260709 12:26:33.029428  6274 connection.cc:256] RPC call timeout handler was delayed by -2.786948270999801s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 12:26:33.029834  6274 connection.cc:256] RPC call timeout handler was delayed by -2.540348270999857s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 12:26:33.030220  6274 connection.cc:256] RPC call timeout handler was delayed by -2.4710482710001997s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 12:26:33.030645  6274 connection.cc:256] RPC call timeout handler was delayed by -2.1803482710001845s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2376s for a response
W20260709 12:26:33.031033  6274 connection.cc:256] RPC call timeout handler was delayed by -2.0587595199999669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22610000000000002s for a response
W20260709 12:26:33.031419  6274 connection.cc:256] RPC call timeout handler was delayed by -1.218159519999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3195s for a response
I20260709 12:26:32.976543  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.003912  6202 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 10, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 10 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:32.982196  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 158660us
W20260709 12:26:33.006217  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 192364us
I20260709 12:26:33.008247  6450 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:33.011606  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:32.989655 10477 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:33.000483  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.017879  6385 raft_consensus.cc:2410] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:26:33.000468 10232 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:33.027632  6303 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:33.038515  6301 raft_consensus.cc:2410] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:33.040299  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.040246  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.039157 10232 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:33.042251 10470 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:26:33.043056  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.047166  9976 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:33.047719  9976 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:33.048154 10476 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.049826  9976 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:33.055783  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:33.057420  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:33.062994  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.064181  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.065196  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.065721  6301 raft_consensus.cc:2376] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9. Re-sending same reply.
W20260709 12:26:33.068718  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 52931.5us
I20260709 12:26:33.041271  6308 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:33.074350  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:33.076668  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43916.2us
W20260709 12:26:33.065852  7366 debug-util.cc:398] Leaking SignalData structure 0x7b08002cc900 after lost signal to thread 5140
I20260709 12:26:33.078333  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.079180  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.050405  9976 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:33.063006 10555 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:33.015748 10416 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:26:33.068483 10480 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:33.114760  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45247.3us
I20260709 12:26:33.085261  6305 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:33.145135  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.143102  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65688.1us
W20260709 12:26:33.155889  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84370.9us
I20260709 12:26:33.144922  6202 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 2 to 4, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:33.143576  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.142858  6303 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:33.160642  6300 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:26:33.161850  7366 debug-util.cc:398] Leaking SignalData structure 0x7b080034d4e0 after lost signal to thread 6240
I20260709 12:26:33.148752  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:33.163498  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73356.3us
I20260709 12:26:33.166893  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.089833  9976 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:33.170459  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.165707  6202 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 3 to 4, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:33.133988 10555 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:33.173851  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.175261  7366 debug-util.cc:398] Leaking SignalData structure 0x7b0800383a80 after lost signal to thread 6334
W20260709 12:26:33.175638  7366 debug-util.cc:398] Leaking SignalData structure 0x7b08003994c0 after lost signal to thread 6335
I20260709 12:26:33.172358  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.177572  6301 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:33.144225 10477 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:33.179342  9985 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:33.179962  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.180042  7366 debug-util.cc:398] Leaking SignalData structure 0x7b080034c6e0 after lost signal to thread 6409
W20260709 12:26:33.180837  7366 debug-util.cc:398] Leaking SignalData structure 0x7b0800377ac0 after lost signal to thread 6410
I20260709 12:26:33.183344  6300 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:33.184499  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.186451  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.179876  9985 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.191054  9985 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.191141 10584 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:33.191828 10584 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:33.192178 10584 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:33.194891  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:33.195137  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:33.194254  7332 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEARNER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:33.196102  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:33.196955  7317 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:33.196601 10232 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:33.188933  6302 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:26:33.148487 10454 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 12:26:33.198469  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 141187us
I20260709 12:26:33.203060  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:33.203258  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.203796  7329 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:33.200590  7366 debug-util.cc:398] Leaking SignalData structure 0x7b0800386fa0 after lost signal to thread 6483
W20260709 12:26:33.204420  7366 debug-util.cc:398] Leaking SignalData structure 0x7b08003bace0 after lost signal to thread 6484
I20260709 12:26:33.153398 10554 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:33.159406 10315 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:26:33.206804  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:33.211412  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.205296  6301 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:33.149042 10506 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:26:33.215101  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.160020 10470 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:26:33.174381 10555 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:33.216719  6308 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:33.194923 10583 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:33.224865  7332 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:33.233402  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:33.217725  7319 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:33.220148  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:33.227520  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.174422 10095 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:26:33.220527  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.230647  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.239200  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.241456  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 80946.5us
I20260709 12:26:33.231796  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.233980  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.247359  6308 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:33.238185 10583 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.271077 10583 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.239121  6459 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:33.232527  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.236474  7366 debug-util.cc:398] Leaking SignalData structure 0x7b0800233e80 after lost signal to thread 7371
I20260709 12:26:33.232779 10454 consensus_queue.cc:1243] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:33.239571 10232 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.271039 10415 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:33.384364  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 146293us
W20260709 12:26:33.357168  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.364177  7366 debug-util.cc:398] Leaking SignalData structure 0x7b0800364980 after lost signal to thread 7373
I20260709 12:26:33.389859  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.357184  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:33.363394  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.374697  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:33.376112  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 141993us
I20260709 12:26:33.381335  6300 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:33.365100 10465 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:33.405591  6301 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:33.406617  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.416922  6303 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:33.417368  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.239195  9976 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:33.421953  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.382273 10415 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:33.430903  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.464593  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.465194  6301 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:33.465696  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.469920  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 249818us
W20260709 12:26:33.471657  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 114082us
W20260709 12:26:33.473376  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92286.3us
I20260709 12:26:33.474794  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.475744  6301 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:26:33.473760  6302 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:33.422685 10506 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:33.499094  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.484434  6303 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:33.509853  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:33.511394  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.512801  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.510468  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 116346us
I20260709 12:26:33.457415 10319 raft_consensus.cc:2764] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:33.515039  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.414427 10470 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:26:33.517649  6348 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:33.525271  6348 leader_election.cc:403] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:33.524047  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.530649  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124254us
I20260709 12:26:33.543483  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.545463  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:33.541967  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:33.542161  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.550980  6301 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:33.587575  7319 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.589056  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:33.589680  6376 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:33.588037  7331 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:33.594929  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.601125  6379 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:33.542419 10477 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:33.606458  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:33.605600  6459 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:33.612758  6385 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:33.587157 10400 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:33.622958  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.641378  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.641963  6303 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:33.623075  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.624171  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:33.625031  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 314249us
I20260709 12:26:33.626528 10595 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.264s
W20260709 12:26:33.627501 10586 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:26:33.629443 10477 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:33.644898 10220 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:33.646669  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:33.647682  6425 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.526s (SENT)
I20260709 12:26:33.660336  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.643282  6302 raft_consensus.cc:2376] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
I20260709 12:26:33.632941 10400 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:33.683981  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 174212us
I20260709 12:26:33.684651  6455 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:33.632663 10506 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:26:33.704783  7288 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.564s (SENT)
I20260709 12:26:33.660838 10608 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:33.712949  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:33.713259  7331 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 6.
I20260709 12:26:33.688269 10262 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.681185 10220 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:33.722388  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75837.3us
I20260709 12:26:33.690308 10400 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:33.727121  6302 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:33.689085 10373 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
W20260709 12:26:33.712592  9985 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.002s
I20260709 12:26:33.701073 10477 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.743002  6376 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:33.702713 10244 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:33.719159 10608 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:33.750154  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.737006  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 379968us
I20260709 12:26:33.683063 10588 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.322s
W20260709 12:26:33.739295 10590 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.351s	user 0.001s	sys 0.001s
I20260709 12:26:33.633566 10415 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:33.773393  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:33.782447  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 151344us
W20260709 12:26:33.773933 10584 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.576s	user 0.015s	sys 0.000s
I20260709 12:26:33.737514 10470 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:26:33.767648 10608 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:33.802098  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.806198  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101466us
I20260709 12:26:33.808250  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:33.814702  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50946.5us
I20260709 12:26:33.816874  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:33.815976  6308 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.000s
I20260709 12:26:33.817958  6308 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:33.737809 10262 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:33.753418 10544 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:26:33.807247  7288 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.629s (SENT)
I20260709 12:26:33.852640  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:33.847373  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.751959  9985 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:33.850291  6303 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:33.767433 10244 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:33.877549  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.884578  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.894188  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.899529  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:33.784296 10477 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:33.915427  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 108231us
W20260709 12:26:33.916163  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.702s (SENT)
I20260709 12:26:33.912015  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:33.912902  6305 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:33.918769  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 134568us
I20260709 12:26:33.799541 10584 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:33.925865  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:33.906607 10555 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.000s
W20260709 12:26:33.976429  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60292.7us
I20260709 12:26:33.910705 10603 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.297s
W20260709 12:26:33.972000  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.907204 10611 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:33.985764 10611 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.986299 10611 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:33.988058 10599 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
W20260709 12:26:33.952270  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.999027  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.983479  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.051301  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:34.052753  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.053920  6303 raft_consensus.cc:2410] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:26:34.073482  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50382.2us
W20260709 12:26:33.956211 10590 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:26:34.089180  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 188981us
I20260709 12:26:33.854993 10220 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:34.099733 10476 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.051s	user 0.003s	sys 0.011s
I20260709 12:26:34.102906 10610 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.102978 10607 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:34.121743 10612 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:34.122251 10607 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:33.984032 10555 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.123378 10607 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.127013 10220 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:34.129890  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 379437us
I20260709 12:26:34.135550  6381 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:34.136168  9976 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:34.139565 10477 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.140558  6302 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
W20260709 12:26:34.146708  6425 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.012s (SENT)
I20260709 12:26:34.147346  9985 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.099279 10544 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.148527 10544 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.151141 10544 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:34.156213 10244 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:34.156912 10604 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:34.157594 10604 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:34.157740  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:34.158628  7332 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.158617 10604 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.150961  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:34.147317  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:34.109144 10594 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 12:26:34.100755 10617 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.083175 10606 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:34.174804  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.187302  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:34.187517  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 162591us
I20260709 12:26:34.188660  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.189693  6303 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:26:34.192653  7288 leader_election.cc:336] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.052s (SENT)
I20260709 12:26:34.193217  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:34.191334  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2106) took 1294 ms. Trace:
W20260709 12:26:34.206171  7319 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.000s
I20260709 12:26:34.208643  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:26:34.191804 10232 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.804s	user 0.001s	sys 0.010s
I20260709 12:26:34.192508 10617 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:34.219429  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 157230us
I20260709 12:26:34.219846 10618 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:34.177785 10609 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.398s
W20260709 12:26:34.221973 10505 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.195s	user 0.010s	sys 0.000s
W20260709 12:26:34.222124  7288 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.079s (SENT)
I20260709 12:26:34.173130 10476 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:26:34.181001 10244 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:34.226604 10598 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.415s	user 0.001s	sys 0.001s
I20260709 12:26:34.227816  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2115) took 1264 ms. Trace:
I20260709 12:26:34.234987  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 517) took 1219 ms. Trace:
I20260709 12:26:34.239876  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:34.196671  6307 rpcz_store.cc:276] 0709 12:26:32.887220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.887384 (+   164us) service_pool.cc:224] Handling call
0709 12:26:32.900540 (+ 13156us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.960300 (+ 59760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.960347 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.003723 (+ 43376us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.102944 (+ 99221us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:33.162183 (+ 59239us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.163321 (+  1138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.163397 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.627569 (+464172us) spinlock_profiling.cc:243] Waited 2.46 ms on lock 0x7b18002b9de0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:34.181220 (+553651us) raft_consensus.cc:1685] finished
0709 12:26:34.181348 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.181559 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54646,"prepare.queue_time_us":61574,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":64516,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":205,"spinlock_wait_cycles":5662464,"thread_start_us":98803,"threads_started":1,"wal-append.queue_time_us":202875}
I20260709 12:26:34.243541  6309 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:34.209985 10335 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:34.247785  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101092us
I20260709 12:26:34.247062  6456 rpcz_store.cc:276] 0709 12:26:33.015136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.015731 (+   595us) service_pool.cc:224] Handling call
0709 12:26:33.016053 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.019207 (+  3154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.019255 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.039723 (+ 20468us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.160870 (+121147us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:33.160911 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.160985 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.234685 (+1073700us) raft_consensus.cc:1685] finished
0709 12:26:34.234715 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.234917 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9863,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":9448,"thread_start_us":46773,"threads_started":1,"wal-append.queue_time_us":292178}
I20260709 12:26:34.250334  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:34.252887  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:34.255412  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:34.256675  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.239796 10373 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:34.245723  6310 rpcz_store.cc:276] 0709 12:26:32.963565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.966044 (+  2479us) service_pool.cc:224] Handling call
0709 12:26:33.019614 (+ 53570us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:33.078851 (+ 59237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.078878 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:33.164753 (+ 85875us) log.cc:844] Serialized 52 byte log entry
0709 12:26:33.383525 (+218772us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:26:33.414458 (+ 30933us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.602867 (+188409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.602982 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.227563 (+624581us) raft_consensus.cc:1685] finished
0709 12:26:34.227582 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.227738 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26286,"prepare.queue_time_us":30736,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":18023,"raft.queue_time_us":217131,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":64513,"thread_start_us":355790,"threads_started":2,"wal-append.queue_time_us":398594}
I20260709 12:26:34.173820 10610 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.267620  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.282997  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.287040  7332 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
W20260709 12:26:34.287899 10593 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.004s	sys 0.047s
I20260709 12:26:34.266409 10244 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:34.295557  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.296435  6310 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:34.290827  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.293488  6303 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:34.270584 10262 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:34.289283 10480 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.135s	user 0.009s	sys 0.004s
I20260709 12:26:34.243592 10618 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.304183  6307 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:34.288908 10335 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:34.307211  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.288522 10606 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:34.292985 10593 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.004s	sys 0.048s
I20260709 12:26:34.308116  6301 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:26:34.308773  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86692.6us
I20260709 12:26:34.309535  6303 raft_consensus.cc:2376] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8. Re-sending same reply.
W20260709 12:26:34.311200 10591 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.146s	user 0.006s	sys 0.011s
I20260709 12:26:34.311373 10617 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.316013 10610 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:34.318845  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68270.4us
I20260709 12:26:34.319195  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.324638  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.326696  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:34.340211 10618 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:34.356676 10620 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:26:34.357064 10464 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.357429  6310 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:34.358101  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:34.337697 10628 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:34.358973 10480 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:34.357595 10464 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.360499  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:34.360225  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.361377 10480 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:34.361946  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:34.362540 10509 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:34.363873  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.376979  6310 raft_consensus.cc:2368] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:34.383996  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1912) took 1385 ms. Trace:
W20260709 12:26:34.384014  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57402.8us
W20260709 12:26:34.341329 10400 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.000s
I20260709 12:26:34.386446  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.384532  6304 rpcz_store.cc:276] 0709 12:26:32.998456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.035436 (+ 36980us) service_pool.cc:224] Handling call
0709 12:26:33.044151 (+  8715us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:33.078254 (+ 34103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.078283 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:33.090270 (+ 11987us) log.cc:844] Serialized 52 byte log entry
0709 12:26:34.248936 (+1158666us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b180001f000. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:34.325958 (+ 77022us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:26:34.326002 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.326075 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.335094 (+  9019us) raft_consensus.cc:1685] finished
0709 12:26:34.335117 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.383925 (+ 48808us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26571,"raft.queue_time_us":41354,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":42193,"spinlock_wait_cycles":8750976,"thread_start_us":76813,"threads_started":1,"wal-append.queue_time_us":1175213}
I20260709 12:26:34.393679  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.330233 10584 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:34.390772 10480 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:34.335177 10612 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:26:34.394709  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.396720  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.362063 10464 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:34.397572  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.398838  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:34.402689  7288 leader_election.cc:336] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.236s (SENT)
I20260709 12:26:34.431339  6303 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.388648 10509 raft_consensus.cc:2764] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:34.432209  6310 raft_consensus.cc:2368] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:34.432004  6307 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:34.395772 10480 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.442308  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:34.450233 10583 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.062s	user 0.008s	sys 0.004s
I20260709 12:26:34.433676  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:34.400842  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.438651 10335 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:34.445780  6307 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:34.456641  6303 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:34.460453  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.462566  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.390754 10400 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.464510  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.463332  6304 raft_consensus.cc:2376] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9. Re-sending same reply.
I20260709 12:26:34.468483  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.469342  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.471947  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.472935  6304 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:26:34.482946 10212 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:34.483547 10212 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.484733 10212 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:34.487046 10589 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.107s	user 0.010s	sys 0.006s
I20260709 12:26:34.488240 10335 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.491350 10400 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:34.488816  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 514) took 1558 ms. Trace:
I20260709 12:26:34.496203  6451 rpcz_store.cc:276] 0709 12:26:32.930703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.930878 (+   175us) service_pool.cc:224] Handling call
0709 12:26:32.931226 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.965832 (+ 34606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.965873 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.990768 (+ 24895us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.071814 (+ 81046us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:33.071867 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.071953 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.412669 (+1340716us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:34.488432 (+ 75763us) raft_consensus.cc:1685] finished
0709 12:26:34.488460 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.488736 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26238,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6542,"spinlock_wait_cycles":5830912,"thread_start_us":80624,"threads_started":1,"wal-append.queue_time_us":235545}
I20260709 12:26:34.503141  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.503289  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.504113  6303 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:34.504742  9985 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:34.505541  9985 consensus_queue.cc:1243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:34.505995  9985 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:26:34.506132  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 103483us
W20260709 12:26:34.507503 10597 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.911s	user 0.008s	sys 0.016s
I20260709 12:26:34.508496 10480 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.506356  9985 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.512989  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.515774  6451 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 10. Ops: []
I20260709 12:26:34.517045  6456 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:34.471081 10490 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:34.508378 10641 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:34.520244  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:34.467260 10509 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:26:34.526139  6304 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:26:34.455101 10583 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:34.517495 10608 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.006s
I20260709 12:26:34.527658  6307 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:34.533257  7335 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:34.533438  6455 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.534699  6304 pending_rounds.cc:85] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 6
W20260709 12:26:34.537688 10611 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.006s
I20260709 12:26:34.538153 10611 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.539746 10583 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.540539 10611 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:34.540762 10220 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:34.498725 10609 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:26:34.533931 10490 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.539443  9985 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.555816  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:34.561403  6451 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:34.562740  6303 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:34.563158  6303 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.563948  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:34.564662  7335 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:34.519721 10212 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:34.567816  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:34.571578  6376 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:26:34.572098  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:34.573128  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.573383  6380 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:34.561229 10608 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.536888  6304 pending_rounds.cc:107] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 11.7
I20260709 12:26:34.572450  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.571769  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:34.580068  6377 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:34.583266  6307 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:34.518352 10464 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:26:34.585451  6304 raft_consensus.cc:2917] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Aborting config change with OpId 11.7: Aborted: Op aborted by new leader
I20260709 12:26:34.586829  6304 raft_consensus.cc:2889] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.589592  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.569510 10616 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.588620  6303 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.597749  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:34.599792  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.600345  6304 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:34.601284  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:34.601830  6377 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.602187  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.603071  6304 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:26:34.606528  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.608986  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:34.607657  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:34.612090  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.569862 10583 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:26:34.624749  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.625449  6304 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:34.622439 10476 raft_consensus.cc:2764] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:26:34.632100 10495 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:34.632885 10495 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.633314 10495 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:34.633796  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:34.634624 10647 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:26:34.609982 10490 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.635315  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:34.625844 10519 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.986s	user 0.024s	sys 0.019s
I20260709 12:26:34.622293 10335 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:34.641477  6455 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:34.641366  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64371.9us
I20260709 12:26:34.645327  6304 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:34.630640  9985 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.647253  6304 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.652841  6304 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.653843  6304 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:34.618432 10334 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:26:34.649525  6455 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:34.643792  9976 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:34.658862 10465 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:34.659315 10465 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:34.660171 10465 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.661284 10465 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.664465  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2143) took 1028 ms. Trace:
I20260709 12:26:34.664845  6301 rpcz_store.cc:276] 0709 12:26:33.640125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.660092 (+ 19967us) service_pool.cc:224] Handling call
0709 12:26:34.664292 (+1004200us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54001b26d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:34.664408 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448093056}
W20260709 12:26:34.641183 10519 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.153s	user 0.024s	sys 0.020s
I20260709 12:26:34.668130  9976 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.668326  6302 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:34.668741  9976 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:34.650949 10646 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:34.635667 10647 consensus_queue.cc:1243] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:34.666406  6301 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:34.640858 10476 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:34.679342  6377 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:34.685041  6304 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:26:34.685554  6304 pending_rounds.cc:85] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 3
I20260709 12:26:34.685791  6304 pending_rounds.cc:107] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:26:34.686026  6304 raft_consensus.cc:2889] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.686525  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:34.646160 10244 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:34.605751 10616 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.689158  6302 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:26:34.646109 10335 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.689728  6301 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:34.691275  6301 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.671032 10609 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.704932  6455 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:34.698841  6301 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.690408 10334 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:34.698014 10335 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:34.721971 10334 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:34.723438 10334 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:34.727231 10621 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.495s	user 0.003s	sys 0.001s
I20260709 12:26:34.733628  6455 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:34.721397  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:34.740262  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:34.740761  6301 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:34.741042  7332 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:34.752995  6455 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:34.775599  7332 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:34.778079  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:34.738958 10608 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.784456  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:34.785449  6455 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:34.786885 10656 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:34.787985 10554 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:34.788759 10647 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:34.789319 10647 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:34.790190 10647 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.793953 10555 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.799017  6455 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:34.799973  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:34.800743  6377 raft_consensus.cc:2368] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11.
I20260709 12:26:34.801723  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:34.803329  6383 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:34.805222  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:34.805871  6381 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:34.807173  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1920) took 1742 ms. Trace:
I20260709 12:26:34.807562  6300 rpcz_store.cc:276] 0709 12:26:33.064395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.476754 (+412359us) service_pool.cc:224] Handling call
0709 12:26:33.518205 (+ 41451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.616004 (+ 97799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.616036 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.635077 (+ 19041us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.875704 (+240627us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:33.875749 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.706880 (+831131us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:34.806865 (+ 99985us) spinlock_profiling.cc:243] Waited 68.8 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:34.806916 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.806960 (+    44us) raft_consensus.cc:1685] finished
0709 12:26:34.806974 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.807113 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3517,"raft.queue_time_us":27325,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":16927,"spinlock_wait_cycles":163522816,"thread_start_us":174159,"threads_started":1,"wal-append.queue_time_us":483734}
W20260709 12:26:34.812614  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.813678  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.814254  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.814628  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.819089  6301 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:34.748588 10531 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.718183 10609 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:34.831818  6455 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:26:34.833251  6424 leader_election.cc:400] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11.
I20260709 12:26:34.848037  6424 leader_election.cc:403] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:34.826874  6381 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:34.885105  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:34.892128  6384 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:26:34.884087  6309 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1932) took 1431 ms (client timeout 1782 ms). Trace:
W20260709 12:26:34.884760  6305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1925) took 1772 ms (client timeout 2202 ms). Trace:
W20260709 12:26:34.900421  6305 rpcz_store.cc:269] 0709 12:26:33.089497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.202970 (+113473us) service_pool.cc:224] Handling call
0709 12:26:34.861229 (+1658259us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54002de6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:34.861366 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265705344}
W20260709 12:26:34.899549  6309 rpcz_store.cc:269] 0709 12:26:33.420268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.464403 (+ 44135us) service_pool.cc:224] Handling call
0709 12:26:34.851277 (+1386874us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54002067d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:34.851431 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678558592}
W20260709 12:26:34.909600  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76585.3us
W20260709 12:26:34.912055  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.913647  6452 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:26:34.917095  6455 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:34.909487 10658 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:26:34.922421  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.923185  6450 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:34.926965  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.928478  6381 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:34.929049  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.930912  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.933394  6459 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:34.845944 10531 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:34.947432  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:34.954854 10650 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.351s	user 0.003s	sys 0.000s
I20260709 12:26:34.955843  9976 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:34.963044  6458 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:34.935557  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:34.966010  6455 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:34.968482  6380 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:34.963300  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:34.972232  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.973301  6301 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:34.974021  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.979861  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.984305  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.987133  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:34.987656  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:34.987839  6301 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:34.989272  6459 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:34.989719  6451 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
W20260709 12:26:34.991320  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79855.6us
I20260709 12:26:34.969985 10583 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.999212  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.001551  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.001667  6450 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:35.000929  6455 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:35.011099  6302 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:35.021776  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:35.022429  6300 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
W20260709 12:26:35.025080  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 303724us
I20260709 12:26:35.030059  6450 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:35.030632  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:34.998629 10667 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:35.035091  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.032881  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2145) took 1285 ms. Trace:
I20260709 12:26:35.036947  6310 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:26:35.038668  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54826.4us
W20260709 12:26:35.040401  7288 leader_election.cc:336] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.895s (SENT)
I20260709 12:26:35.012920 10616 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:35.041601  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.037180  6308 rpcz_store.cc:276] 0709 12:26:33.747064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.892853 (+145789us) service_pool.cc:224] Handling call
0709 12:26:35.032697 (+1139844us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54002ddcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:35.032812 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":775196032}
I20260709 12:26:35.050820  6308 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:35.016520 10583 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:35.059330 10509 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:26:35.060138 10662 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 12:26:35.061224 10655 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:26:35.067643 10649 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.462s	user 0.003s	sys 0.000s
I20260709 12:26:35.068924 10673 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:35.071563  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:35.078763  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:35.096182  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55818us
I20260709 12:26:35.111222  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.114459  6458 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:35.117879  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:35.118167  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61779.5us
I20260709 12:26:35.123848  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:35.124575  6452 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:35.069726 10531 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:35.099378 10509 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:35.133347 10509 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:35.135741 10394 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.136377  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:35.136312 10394 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:35.139554  6351 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.514s (SENT)
I20260709 12:26:35.139999  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.140772  6303 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:35.141034  6351 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.591s (SENT)
I20260709 12:26:35.141515  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.144083  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.144838  6308 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:35.145946  6301 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:35.148430  6452 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:35.154630  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 271901us
I20260709 12:26:35.151937  6450 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:35.156503  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:35.160159  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.160818  6301 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:35.160614  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.161509  6308 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:35.162279  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.138401 10394 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:35.172592  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.173677  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.174226  6308 raft_consensus.cc:2376] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11. Re-sending same reply.
W20260709 12:26:35.176683  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.178399  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.179230  6308 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:35.142938 10509 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:35.171131  6351 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.619s (SENT)
I20260709 12:26:35.169251  6301 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:35.186571  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.175074  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:35.191982  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:35.192898  6459 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:35.194142  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:35.162895  9985 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
W20260709 12:26:35.171018 10495 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.006s
I20260709 12:26:35.198768  6452 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:35.199908  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.200891  6301 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:35.202230  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.203006  6301 raft_consensus.cc:2376] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10. Re-sending same reply.
I20260709 12:26:35.185158  6303 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:35.204072  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.204859  6301 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:35.206960  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.208516  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.209756  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.211372  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.210812  6301 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:35.214253  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:35.214781  6455 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:35.215027  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.216768  6301 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:35.214104  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.146178 10335 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:35.226835  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.227649  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.232625  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:35.218820 10232 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
W20260709 12:26:35.236177  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.236578  6459 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:35.199548 10417 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:35.237279  6301 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:35.185801 10262 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:26:35.210913 10400 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:35.250716  6301 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:35.239153  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:35.252103  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.253115  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:35.253896  6450 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:35.252908  6308 raft_consensus.cc:2376] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11. Re-sending same reply.
I20260709 12:26:35.230582 10505 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:35.257910  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:35.227248 10610 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:35.259311  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88240.9us
I20260709 12:26:35.266675  6455 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
W20260709 12:26:35.267249  6351 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.634s (SENT)
I20260709 12:26:35.263180  6308 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:35.235878  9985 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:35.215483 10495 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:35.276348  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 245827us
I20260709 12:26:35.271718  6308 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:35.248997 10416 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:35.254000 10670 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:35.266412  6452 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:35.253875 10672 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:26:35.302539  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:35.303069  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.319166 10614 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 12:26:35.319803  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:35.321220 10568 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.129s
I20260709 12:26:35.314038  6194 maintenance_manager.cc:643] P 08c731ab128f4ef881e4c3c73ff48bd8: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 9.120s	user 0.241s	sys 0.066s Metrics: {"bytes_written":28783,"cfile_init":1,"dirs.queue_time_us":398762,"dirs.run_cpu_time_us":659,"dirs.run_wall_time_us":80391,"drs_written":1,"lbm_read_time_us":12006,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":242566,"lbm_writes_1-10_ms":3,"lbm_writes_10-100_ms":8,"lbm_writes_lt_1ms":19,"peak_mem_usage":0,"raft.queue_time_us":45619,"raft.run_cpu_time_us":13509,"raft.run_wall_time_us":282371,"rows_written":63,"thread_start_us":225416,"threads_started":3,"wal-append.queue_time_us":149109}
I20260709 12:26:35.271885 10619 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.266044 10505 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:35.323359  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.272959 10610 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.326540  6305 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:35.337158 10647 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:26:35.265267 10400 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:35.322568 10568 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:35.346645 10668 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:35.337803 10583 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:35.362783  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 121161us
W20260709 12:26:35.368800  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45501.1us
I20260709 12:26:35.338728 10610 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:35.402693  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 99207.2us
I20260709 12:26:35.444036  6308 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:35.339279 10619 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:35.405403  6195 maintenance_manager.cc:419] P 08c731ab128f4ef881e4c3c73ff48bd8: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9034 bytes on disk
I20260709 12:26:35.407073  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1950) took 1215 ms. Trace:
I20260709 12:26:35.455103  6310 rpcz_store.cc:276] 0709 12:26:34.192184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.512491 (+320307us) service_pool.cc:224] Handling call
0709 12:26:35.033229 (+520738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.037426 (+  4197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.406872 (+369446us) spinlock_profiling.cc:243] Waited 509 ms on lock 0x7b54002ddcd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:35.407012 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1171577728}
W20260709 12:26:35.411826 10675 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
W20260709 12:26:35.413101 10674 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.213s	user 0.000s	sys 0.003s
I20260709 12:26:35.398751 10568 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:35.414860 10701 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:35.418521 10681 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:26:35.424124  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:35.383378 10622 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:35.431034  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.436309 10609 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:35.446535  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.443764  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:35.447666  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:35.452842  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 240062us
W20260709 12:26:35.534606  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 378136us
I20260709 12:26:35.526556 10415 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:35.530061 10335 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:35.530580 10495 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:35.442731 10505 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:35.863618  6194 maintenance_manager.cc:643] P 08c731ab128f4ef881e4c3c73ff48bd8: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.407s	user 0.012s	sys 0.043s Metrics: {"cfile_init":1,"lbm_read_time_us":556,"lbm_reads_lt_1ms":4}
I20260709 12:26:35.920855  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:35.908828 10465 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.240s	user 0.017s	sys 0.008s
W20260709 12:26:35.933903  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 487478us
W20260709 12:26:35.937182  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 492687us
W20260709 12:26:35.923870 10416 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.622s	user 0.018s	sys 0.002s
W20260709 12:26:35.955576  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 511874us
I20260709 12:26:35.926924 10609 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:35.954880 10495 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:35.955372 10608 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:26:35.945398 10699 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:36.019251  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:26:35.972530 10696 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:36.020139  6450 raft_consensus.cc:606] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:36.020601  6450 raft_consensus.cc:614] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:35.971236 10519 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.052s
W20260709 12:26:35.974861 10667 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.931s	user 0.010s	sys 0.006s
W20260709 12:26:35.967558  9985 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.688s	user 0.015s	sys 0.000s
I20260709 12:26:35.964028 10416 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:36.044391 10667 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11.
I20260709 12:26:35.946897 10610 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:35.970916 10703 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.972373 10608 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:36.021253 10220 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:26:35.974355 10244 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:35.967958 10495 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:36.041883  9985 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:35.973315 10619 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:36.016465 10668 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
W20260709 12:26:36.020187 10509 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.824s	user 0.009s	sys 0.008s
I20260709 12:26:35.972848 10699 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:36.025522 10696 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:36.135095  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 200327us
I20260709 12:26:35.959744 10568 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.117786 10608 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:36.121758 10400 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:36.240460  7288 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.754s (SENT)
I20260709 12:26:36.027266 10609 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:36.362989  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:36.122187 10703 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:36.361346  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 440701us
W20260709 12:26:36.365238  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 451544us
I20260709 12:26:36.260247  6457 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:36.369029  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:36.368152  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:36.371767  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:36.381172  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 140862us
I20260709 12:26:36.381081  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:36.374531 10617 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:36.383878  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.363772 10696 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:36.375824 10416 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:36.368480 10609 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:36.386353  7288 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.794s (SENT)
I20260709 12:26:36.378780 10622 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:36.380501 10619 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:36.390642 10703 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:36.365185 10495 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.391474 10608 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:36.388367 10416 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.407793  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:36.410741  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:36.411383 10622 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:36.417557  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:36.396245 10619 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.422600  6458 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:36.428231  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:36.400287 10495 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.424044 10710 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:36.400949 10608 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:36.397809 10707 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:36.436856 10711 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:36.440816 10710 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:36.399083 10609 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.460878 10609 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:36.426316 10672 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:36.446020 10688 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:36.410583 10416 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.420403 10610 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:36.398361 10708 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:26:36.450263 10707 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:36.453732  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:36.454267  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74698.6us
I20260709 12:26:36.481521 10707 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:36.482968  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:36.454860 10679 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:36.399188 10699 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:36.445279  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 1.12548e+06us
I20260709 12:26:36.465106 10711 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:36.486752 10679 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:36.487341 10679 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:36.488658 10711 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:36.425786 10619 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:36.478077  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:36.491024  6452 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:36.488238 10679 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.478370 10672 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.512816  6384 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:36.515167  6384 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:36.518225  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:36.522017  6376 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:26:36.523767  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 58163.1us
I20260709 12:26:36.524825  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:36.507853 10490 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.547966  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:36.524943 10610 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:36.552299  6452 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:26:36.557538  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 129389us
I20260709 12:26:36.558507  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:36.560263  6382 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:36.570886  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:36.571482  6378 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:36.552796 10415 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.000s
I20260709 12:26:36.541986 10710 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:36.577095  6382 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:36.546940 10490 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.580735  6379 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:36.582291  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:36.583365  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:36.583590  6381 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:36.568229 10595 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:36.594290  6378 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:26:36.595781  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71082.6us
I20260709 12:26:36.599545  6380 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:36.603410  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:36.608624  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:36.615664  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:36.616165  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:36.617136  6453 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:36.581272 10531 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.626751  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:36.629948  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:36.599020 10617 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:36.597374 10595 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:36.662602  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59232.9us
I20260709 12:26:36.628753 10709 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:36.658898 10244 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:36.679509 10244 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.679828 10490 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:36.668195 10717 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:26:36.682107 10717 consensus_queue.cc:1243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:26:36.623332 10610 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:26:36.628504 10703 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:26:36.672199 10669 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:36.587417 10710 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.622447 10711 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:26:36.668151 10581 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.212404 10581 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:36.674628 10718 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:26:36.650861 10400 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:36.628986 10713 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.261554 10400 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:36.709380 10709 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:36.679984 10244 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:36.629237 10531 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:36.727605 10710 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:36.669584 10595 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:36.674005 10722 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:26:36.671168  6453 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:36.631249 10704 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:26:36.732563 10696 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.391677 10696 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:36.824645 10509 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:37.392313 10509 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:37.393085 10509 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:36.821535 10619 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.395885 10619 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:36.846650  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 453225us
W20260709 12:26:37.396689  7288 leader_election.cc:336] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.012s (SENT)
I20260709 12:26:37.397074  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:36.861033 10677 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.490s	user 0.010s	sys 0.000s
W20260709 12:26:36.893419  6303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.975s	user 0.012s	sys 0.001s
W20260709 12:26:36.901401  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.973908  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:37.400476  6303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.482s	user 0.013s	sys 0.002s
I20260709 12:26:37.401135  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 582) took 2888 ms. Trace:
W20260709 12:26:37.401350  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.42741e+06us
W20260709 12:26:36.945541  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 345040us
W20260709 12:26:36.965101  6457 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 525) took 1791 ms (client timeout 1674 ms). Trace:
I20260709 12:26:37.401544  7320 rpcz_store.cc:276] 0709 12:26:34.512913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.513058 (+   145us) service_pool.cc:224] Handling call
0709 12:26:34.523644 (+ 10586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.547550 (+ 23906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.547587 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:34.569453 (+ 21866us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.811384 (+241931us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b18002271a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:35.177224 (+365840us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:35.177261 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.177325 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:36.860194 (+1682869us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:37.400922 (+540728us) raft_consensus.cc:1685] finished
0709 12:26:37.400945 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.401081 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23475,"raft.queue_time_us":25113,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":8231,"spinlock_wait_cycles":13801728,"thread_start_us":365641,"threads_started":1,"wal-append.queue_time_us":726211}
W20260709 12:26:37.402737  6457 rpcz_store.cc:269] 0709 12:26:35.174093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.174203 (+   110us) service_pool.cc:224] Handling call
0709 12:26:36.964814 (+1790611us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b54001617d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:36.964980 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1958512384}
I20260709 12:26:36.969949  6455 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:37.404080  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:37.005805  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:37.019847 10615 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:37.405385 10615 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:37.405815  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 400091us
W20260709 12:26:37.042091 10609 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.581s	user 0.003s	sys 0.007s
I20260709 12:26:37.406302 10615 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.406925 10609 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:26:37.044955  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.050450 10519 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:37.407424 10615 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:37.407681 10519 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:37.407994  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.408454  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.408659 10519 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.409718 10519 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:37.409822  7288 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.153s (SENT)
I20260709 12:26:37.410259  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:37.407447 10609 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:37.057818 10679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.010s	sys 0.026s
I20260709 12:26:37.411623 10609 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:37.106355 10416 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.599s	user 0.009s	sys 0.003s
I20260709 12:26:37.105995  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 530) took 2547 ms. Trace:
I20260709 12:26:37.412678 10705 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.412724 10416 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.413467 10705 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.413115  6385 rpcz_store.cc:276] 0709 12:26:34.558390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.558901 (+   511us) service_pool.cc:224] Handling call
0709 12:26:34.576201 (+ 17300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.141244 (+565043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.141272 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:35.141729 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:26:35.257352 (+115623us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:35.301614 (+ 44262us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.301871 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.301960 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.102415 (+1800455us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b1800425c80. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:37.103770 (+  1355us) spinlock_profiling.cc:243] Waited 2.58 ms on lock 0x7b1800425c80. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:37.105754 (+  1984us) raft_consensus.cc:1685] finished
0709 12:26:37.105775 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.105916 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":565908,"prepare.queue_time_us":44107,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":5363,"raft.queue_time_us":566007,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":16352,"spinlock_wait_cycles":10005504,"thread_start_us":113032,"threads_started":1,"wal-append.queue_time_us":204378}
I20260709 12:26:37.415143 10416 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:37.415741 10416 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.109656 10616 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.113464  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 499689us
I20260709 12:26:37.415889 10611 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:37.418565  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.415970 10690 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
W20260709 12:26:37.419044  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.419601  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.194977 10698 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.420094 10698 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.420612 10480 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.209468 10495 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.000s
W20260709 12:26:37.231904 10695 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.858s	user 0.000s	sys 0.005s
I20260709 12:26:37.233196 10581 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.248559 10700 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.262338 10709 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:37.264662 10400 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.376008 10244 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.268994 10688 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:37.272431  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:37.412755 10606 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.458441  6458 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:37.458094  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.275457  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2213) took 2365 ms. Trace:
I20260709 12:26:37.291507  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:37.459671  6376 raft_consensus.cc:606] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:37.336951  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:37.338847 10682 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.355245 10568 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:37.365315 10531 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:37.372282  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:37.409090 10464 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:37.412218 10679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.010s	sys 0.027s
W20260709 12:26:37.093055  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.466198  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:37.418759  6384 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:37.141438 10505 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:37.175638  6379 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.001s
I20260709 12:26:37.421873  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:37.418197  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:37.473644  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55509.6us
I20260709 12:26:37.475776  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:37.394244 10509 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:37.443236 10616 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:37.453419  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51035.8us
I20260709 12:26:37.491559  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:37.421200 10480 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.461014  7329 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:26:37.452116 10718 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:37.459455  6456 rpcz_store.cc:276] 0709 12:26:34.910215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.916878 (+  6663us) service_pool.cc:224] Handling call
0709 12:26:34.917217 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:35.013159 (+ 95942us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:35.023607 (+ 10448us) log.cc:844] Serialized 14 byte log entry
0709 12:26:36.983888 (+1960281us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b1800451d80. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:26:37.266089 (+282201us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:37.266910 (+   821us) log.cc:844] Serialized 52 byte log entry
0709 12:26:37.267114 (+   204us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:37.267173 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.267264 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.275025 (+  7761us) raft_consensus.cc:1685] finished
0709 12:26:37.275165 (+   140us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.275370 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":265528,"prepare.queue_time_us":9508,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":377826,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":14350,"spinlock_wait_cycles":2316032,"thread_start_us":111716,"threads_started":2,"wal-append.queue_time_us":1972095}
I20260709 12:26:37.454958 10495 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:37.478222  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.527810  6301 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:37.528522  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 573) took 2349 ms. Trace:
I20260709 12:26:37.479947  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:37.472559  6379 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.005s	sys 0.002s
I20260709 12:26:37.531204  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:37.531992  6382 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:37.445035  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:37.483251  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:37.533555  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:37.462414 10688 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.499348 10744 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:37.534600 10744 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.488688 10416 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:37.489142 10710 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:37.502333  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2095) took 2700 ms. Trace:
I20260709 12:26:37.455866 10700 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.481118 10704 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.546622 10704 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.547094 10704 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.545042  6383 rpcz_store.cc:276] 0709 12:26:34.801927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.804227 (+  2300us) service_pool.cc:224] Handling call
0709 12:26:34.804572 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:35.170375 (+365803us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.170406 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:35.187881 (+ 17475us) log.cc:844] Serialized 52 byte log entry
0709 12:26:35.536185 (+348304us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:35.859741 (+323556us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.898880 (+ 39139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.898981 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:37.079191 (+1180210us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180031d800. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:37.231377 (+152186us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:37.502086 (+270709us) raft_consensus.cc:1685] finished
0709 12:26:37.502108 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.502265 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":396510,"prepare.queue_time_us":315374,"prepare.run_cpu_time_us":8729,"prepare.run_wall_time_us":47724,"raft.queue_time_us":354808,"raft.run_cpu_time_us":357,"raft.run_wall_time_us":24702,"spinlock_wait_cycles":27701888,"thread_start_us":304188,"threads_started":1,"wal-append.queue_time_us":980531}
I20260709 12:26:37.549439  7320 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:37.550223 10700 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.504145 10244 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:37.510313  6205 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 7, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 7 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:37.552069  6205 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 6, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 6 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:37.553606  6205 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 5, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 5 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:37.555166  6205 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 10 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:37.514838  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 108347us
I20260709 12:26:37.472613 10531 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.558761  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 547) took 2493 ms (client timeout 1754 ms). Trace:
I20260709 12:26:37.559370  6455 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
W20260709 12:26:37.559262  6304 rpcz_store.cc:269] 0709 12:26:35.034446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.034694 (+   248us) service_pool.cc:224] Handling call
0709 12:26:37.527167 (+2492473us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54002d2dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:37.527323 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5340682240}
I20260709 12:26:37.560482  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:37.561756  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 574) took 2396 ms. Trace:
I20260709 12:26:37.562129  6301 rpcz_store.cc:276] 0709 12:26:35.165734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.261907 (+ 96173us) service_pool.cc:224] Handling call
0709 12:26:37.527446 (+2265539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.528538 (+  1092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.561586 (+ 33048us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54002d2dd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:37.561703 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5209423232}
I20260709 12:26:37.555383  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:37.525928  6458 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:26:37.556679  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76876.6us
I20260709 12:26:37.564575  6383 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
W20260709 12:26:37.512756  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 296) took 35965 ms (client timeout 15000 ms). Trace:
I20260709 12:26:37.561106  6459 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:37.529024  6303 rpcz_store.cc:276] 0709 12:26:35.165533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.256278 (+ 90745us) service_pool.cc:224] Handling call
0709 12:26:35.256673 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:35.258287 (+  1614us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:35.537751 (+279464us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.514868 (+1977117us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:37.514926 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:37.514985 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.515059 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.515181 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16525,"prepare.queue_time_us":112662,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":17087,"raft.run_cpu_time_us":57,"raft.run_wall_time_us":57,"thread_start_us":1817806,"threads_started":3,"wal-append.queue_time_us":547148,"wal-append.run_cpu_time_us":1525,"wal-append.run_wall_time_us":1045724}
I20260709 12:26:37.567101  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:37.567800  6458 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:37.569797  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 124914us
I20260709 12:26:37.570211  6377 pending_rounds.cc:85] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 4
I20260709 12:26:37.570640  6377 pending_rounds.cc:107] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:26:37.570694  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:37.571065  6377 raft_consensus.cc:2889] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.557299 10609 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.525465 10698 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.573339 10609 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.464238 10606 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.575791 10480 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.576202 10609 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.582448  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:37.591058  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 99592.4us
I20260709 12:26:37.597157  6378 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:26:37.465274 10682 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.605499  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:37.606181  6455 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:37.568055 10244 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:37.577174 10465 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.611078  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:37.608132 10593 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:26:37.615546  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.616201  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.592720 10480 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:37.596131  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:37.581023 10667 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:37.580282 10416 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:37.594071 10609 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.593338 10746 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.564091 10495 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.596720  6205 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:37.626256  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:37.637486  6381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:37.653640  6381 raft_consensus.cc:606] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:26:37.654026  6381 raft_consensus.cc:614] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:37.640823  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44760.7us
I20260709 12:26:37.644357  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:37.661154  6383 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:37.665251  6455 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:37.573351 10666 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:26:37.710165 10744 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:37.710780 10744 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.569897 10531 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.712502 10593 raft_consensus.cc:1064] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:26:37.712234 10416 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.713613 10531 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:37.715807 10495 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:37.639926 10480 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.717896 10480 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.683178 10611 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:26:37.718453  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:37.673863 10747 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:37.665936  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 178361us
I20260709 12:26:37.686069 10754 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:26:37.724113 10754 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:37.724489 10754 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:37.724946 10519 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.725263 10754 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.726253 10531 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:37.725803 10519 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.729292 10688 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:37.730261 10519 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.612604 10465 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:37.597632 10718 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:26:37.590559 10678 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:26:37.673906  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:37.739084  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65996.5us
W20260709 12:26:37.565788  6202 rpcz_store.cc:269] 0709 12:26:01.546952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.547450 (+   498us) service_pool.cc:224] Handling call
0709 12:26:37.512602 (+35965152us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:34.332480 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:34.333217 (+   737us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:34.333276 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:26:34.333313 (+    37us) tablet.cc:662] Decoding operations
0709 12:26:34.334729 (+  1416us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:34.334817 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:34.334850 (+    33us) tablet.cc:685] Acquiring locks for 15 operations
0709 12:26:34.543294 (+208444us) tablet.cc:701] Row locks acquired
0709 12:26:34.543351 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:26:34.543590 (+   239us) write_op.cc:270] Start()
0709 12:26:34.543738 (+   148us) write_op.cc:276] Timestamp: P: 1783599994543497 usec, L: 0
0709 12:26:34.543771 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:26:34.544670 (+   899us) log.cc:844] Serialized 5814 byte log entry
0709 12:26:34.703367 (+158697us) op_driver.cc:464] REPLICATION: finished
0709 12:26:35.038589 (+335222us) write_op.cc:301] APPLY: starting
0709 12:26:35.045836 (+  7247us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:35.914523 (+868687us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:35.957077 (+ 42554us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:37.133942 (+1176865us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:37.134350 (+   408us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=30,key_file_lookups=30,delta_file_lookups=0,mrs_lookups=0
0709 12:26:37.134403 (+    53us) write_op.cc:312] APPLY: finished
0709 12:26:37.135656 (+  1253us) log.cc:844] Serialized 139 byte log entry
0709 12:26:37.166298 (+ 30642us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:37.166821 (+   523us) write_op.cc:454] Released schema lock
0709 12:26:37.166901 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:37.170590 (+  3689us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:37.170627 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":6,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":295183,"apply.run_cpu_time_us":42635,"apply.run_wall_time_us":2138337,"cfile_cache_hit":58,"cfile_cache_hit_bytes":60639,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":1,"lbm_read_time_us":110611,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":3,"lbm_reads_lt_1ms":4,"num_ops":15,"prepare.queue_time_us":422846,"prepare.run_cpu_time_us":6996,"prepare.run_wall_time_us":213676,"replication_time_us":159400,"thread_start_us":481987,"threads_started":3,"wal-append.queue_time_us":21331}]]}
I20260709 12:26:37.684527  6205 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 8, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 8 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:37.635927 10606 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.660272 10603 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:26:37.669580 10464 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.659118 10746 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.691746 10616 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.686787  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:26:37.708266  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:37.752530  6274 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.648s (SENT)
I20260709 12:26:37.775336  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:37.778373  6383 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:37.696684  6458 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:37.711279 10744 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.774673 10603 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:37.648155  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:37.659482 10609 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:37.744390  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 121457us
W20260709 12:26:37.753393  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08001caa80 after lost signal to thread 5140
I20260709 12:26:37.694466 10705 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.785045  6457 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:37.732728  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59315.8us
I20260709 12:26:37.750952 10581 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:37.792135  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08000b61c0 after lost signal to thread 6240
I20260709 12:26:37.795619  6205 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 6, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 6 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:37.777530  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:37.703148 10682 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:37.736043 10465 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.809208  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:37.812934  6331 debug-util.cc:398] Leaking SignalData structure 0x7b080011a8c0 after lost signal to thread 6334
W20260709 12:26:37.819183  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08000f0580 after lost signal to thread 6335
I20260709 12:26:37.819222  6458 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:37.774780 10754 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:37.751044 10747 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:37.820900  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08003b67e0 after lost signal to thread 6409
I20260709 12:26:37.822093  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:37.774657 10688 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:26:37.823803  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08003b6820 after lost signal to thread 6410
I20260709 12:26:37.824180  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:37.830010  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08002bef80 after lost signal to thread 6483
I20260709 12:26:37.830323  6454 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:26:37.779443 10464 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.799528 10603 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:37.825791  6205 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 9, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:37.835568  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:37.830332  6331 debug-util.cc:398] Leaking SignalData structure 0x7b08003bcae0 after lost signal to thread 6484
W20260709 12:26:37.838672  6331 debug-util.cc:398] Leaking SignalData structure 0x7b0800247540 after lost signal to thread 7371
I20260709 12:26:37.837210  6459 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:37.846343  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:37.852754  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 241753us
I20260709 12:26:37.811661 10752 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.854280  6205 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 9, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:37.864264  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:37.821403 10480 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:37.774173 10595 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.828293 10519 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.845263 10464 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.871654  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:37.885490  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 109596us
I20260709 12:26:37.822976 10581 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.874086  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.889372  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.828811 10688 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:37.875252  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 122756us
I20260709 12:26:37.875767  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.861233  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:37.893041  7329 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:37.893290  6274 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.670s (SENT)
I20260709 12:26:37.830334 10509 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:37.893711  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:37.863441  6457 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:37.852339 10611 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:37.887220  6453 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:37.867610  6205 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 10, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 10 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:37.890944  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.894783  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.898231  7317 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:37.896044  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:37.900374  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:37.901017  7318 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:37.871014 10495 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:26:37.902189  6205 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 9, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 9 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:37.902587 10509 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:37.904311  6205 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:37.905586  6205 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:37.907780 10595 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:37.907368  6205 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 6, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 6 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:37.872157 10752 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.897253 10611 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.909482  6205 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 8, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 8 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:37.910157 10611 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:37.910928  6205 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 8, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 8 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:37.913156  6205 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 7, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 7 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:37.915337 10746 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.844882 10747 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:37.895206 10688 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:37.917371 10581 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:37.919564 10609 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:37.923048  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.945390  6205 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 8, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 8 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:26:37.947821  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.926434 10709 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:37.949579  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.950259  6425 connection.cc:256] RPC call timeout handler was delayed by -1.912597906999963s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:26:37.950647  6425 connection.cc:256] RPC call timeout handler was delayed by -1.7478979070001515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 12:26:37.950953  6425 connection.cc:256] RPC call timeout handler was delayed by -1.7379979070001355s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17300000000000001s for a response
W20260709 12:26:37.951267  6425 connection.cc:256] RPC call timeout handler was delayed by -1.7325979070001267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:26:37.951584  6425 connection.cc:256] RPC call timeout handler was delayed by -1.6911979070000598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:26:37.951880  6425 connection.cc:256] RPC call timeout handler was delayed by -1.6524979069999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1825s for a response
W20260709 12:26:37.952171  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5561979069998415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1932s for a response
I20260709 12:26:37.946463  6383 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:37.924320 10760 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:37.924198 10531 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:37.881482 10519 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.946907  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:37.952459  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5399979069998153s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.195s for a response
I20260709 12:26:37.945329  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:37.960258 10746 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:37.960803 10746 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:37.961282 10746 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:37.965442  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5129979069997717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.198s for a response
W20260709 12:26:37.970670  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4454979070001173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
W20260709 12:26:37.971243  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4256979070000853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 12:26:37.971699  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4202979070000765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20830000000000004s for a response
W20260709 12:26:37.972162  6425 connection.cc:256] RPC call timeout handler was delayed by -1.3995979070000431s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 12:26:37.972636  6425 connection.cc:256] RPC call timeout handler was delayed by -1.3977979070000401s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
I20260709 12:26:37.967504 10464 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:37.989622  6425 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.536s (SENT)
W20260709 12:26:37.999806  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 226773us
I20260709 12:26:38.000630  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 543) took 1401 ms. Trace:
I20260709 12:26:37.988507 10480 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:38.004215  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 421873us
W20260709 12:26:38.009819  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114443us
W20260709 12:26:38.007568  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 153177us
I20260709 12:26:37.923547 10753 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:38.001155  6379 rpcz_store.cc:276] 0709 12:26:36.597823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.598171 (+   348us) service_pool.cc:224] Handling call
0709 12:26:36.623023 (+ 24852us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.623284 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:37.531414 (+908130us) log.cc:844] Serialized 14 byte log entry
0709 12:26:37.998591 (+467177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:37.998644 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:37.998689 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.998767 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.998866 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":330950,"prepare.queue_time_us":149258,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":329446,"spinlock_wait_cycles":40929536,"thread_start_us":1019893,"threads_started":2,"wal-append.queue_time_us":155205}
I20260709 12:26:37.999174 10614 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:26:37.944837 10704 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:37.991479 10465 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.000372  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:38.107286  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.109454  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 83358.1us
W20260709 12:26:38.111735  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.181s (SENT) [suppressed 45 similar messages]
I20260709 12:26:38.112013  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:38.120735  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.121310  6304 raft_consensus.cc:2376] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8. Re-sending same reply.
I20260709 12:26:38.121855  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.122602  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.122830  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.122982  6308 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:38.123193  6304 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11. Re-sending same reply.
I20260709 12:26:38.124150  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.124408  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.125042  6304 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:38.125622  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.125808  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.126528  6304 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:38.127408  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.123611  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.128077  6305 raft_consensus.cc:2376] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7. Re-sending same reply.
I20260709 12:26:38.129251  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:38.135028  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 134764us
I20260709 12:26:38.136242  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:38.138386  6307 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:38.147522  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.140897  6304 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:38.128268  6309 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:38.150668  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.151726  6309 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:38.152671  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:38.152900  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 120865us
I20260709 12:26:38.153659  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:38.155392  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.157339  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.157538  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:38.114327 10568 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.002s
I20260709 12:26:38.156147  6309 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:38.158387  6310 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:38.158839  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:38.154503  6425 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.719s (SENT)
I20260709 12:26:38.164188  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.164678  7318 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:26:38.166018  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:38.166034  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47324.1us
I20260709 12:26:38.112874 10709 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.177112  6303 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:38.164199  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:38.166951  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:38.181501  6304 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:38.182456  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.182961  6300 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:38.148581 10760 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:38.012207 10480 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:38.164865 10758 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
W20260709 12:26:38.191140  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.166710  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:38.153884 10678 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:38.185379  6309 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:38.153060 10464 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.179674  7335 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:38.183779 10760 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.187091  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.188068  6300 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:26:38.188397  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:38.165648 10682 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:38.194351  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.184809  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.177009 10262 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:38.201632  6310 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:38.203982  6304 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:38.206851  6307 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:38.208256  6351 leader_election.cc:400] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:38.218389  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:38.219189  6383 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:38.224459  6381 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:38.164510 10686 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:26:38.227221  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.227412  6300 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:26:38.227946  6300 pending_rounds.cc:85] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 6
I20260709 12:26:38.228241  6300 pending_rounds.cc:107] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:26:38.193396 10568 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:38.232262  6300 raft_consensus.cc:2889] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.192461 10770 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:38.241360  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.252104  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.212400 10767 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:26:38.202677 10682 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:38.257256  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.268107  6310 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:26:38.260030 10480 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.223752 10710 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.203707 10760 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:38.245244 10768 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:38.214896 10581 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:38.192605 10531 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
W20260709 12:26:38.265172  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57033.4us
I20260709 12:26:38.266081  6305 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:38.283313  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:38.212023 10619 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:38.252611 10262 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:38.233747 10505 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.250695 10753 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.289093 10713 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:38.289799  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:38.275115 10682 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.265172 10568 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:38.274590 10519 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.240082 10705 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.046s
W20260709 12:26:38.275903  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 89601.5us
W20260709 12:26:38.301431  6425 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.736s (SENT)
I20260709 12:26:38.296913  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:38.299995 10531 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:38.339928 10531 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:38.340602 10686 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.340165 10262 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.340943 10531 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.290591 10581 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:38.299232 10708 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:38.343020 10465 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.343292 10581 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.297513 10505 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.300935 10616 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:38.341141 10686 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:38.300555 10771 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
W20260709 12:26:38.349783  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60144.7us
I20260709 12:26:38.361990  6351 leader_election.cc:400] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 6 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 6. Current term is 7.
I20260709 12:26:38.279100 10746 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.280547 10710 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.365281  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:38.347012 10505 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.348821 10531 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.379647 10713 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.372675  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:38.363298 10758 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:26:38.349848 10616 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:38.398216  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 96844.2us
W20260709 12:26:38.400089  6425 leader_election.cc:336] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.782s (SENT)
I20260709 12:26:38.400517  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.406282  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:38.405974  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.408586  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.409207  6381 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:38.418998  6425 leader_election.cc:336] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.825s (SENT)
W20260709 12:26:38.419463  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.419412  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.420558 10679 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:38.343319 10708 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.393091 10416 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:38.379135 10752 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.404790 10616 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:38.347187 10753 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:38.420246 10781 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.117s	user 0.003s	sys 0.000s
W20260709 12:26:38.430596  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.459784  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.435339  6425 leader_election.cc:336] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.932s (SENT)
I20260709 12:26:38.478842  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.435520  6385 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:38.448918  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:38.454877  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.473865 10744 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.416121 10666 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:38.487535  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.476292 10696 raft_consensus.cc:993] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:26:38.485282  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:38.493368  6309 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:38.494263 10696 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:38.416345 10464 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:26:38.496052  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.497346  6303 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.422832 10606 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
W20260709 12:26:38.509555  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.511824 10505 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.512434 10505 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:38.515071 10776 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:26:38.415961 10568 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.432087 10698 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:26:38.524436 10464 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:38.525202 10464 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:38.537935  6303 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:38.539921  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.542722  6303 raft_consensus.cc:2376] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10. Re-sending same reply.
W20260709 12:26:38.527261 10611 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.002s
I20260709 12:26:38.546401  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 553) took 1016 ms. Trace:
I20260709 12:26:38.499084 10480 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.570787  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.567582  6382 rpcz_store.cc:276] 0709 12:26:37.530281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.532513 (+  2232us) service_pool.cc:224] Handling call
0709 12:26:37.532793 (+   280us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:37.582424 (+ 49631us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.582453 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:38.165176 (+582723us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:26:38.165725 (+   549us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.322958 (+157233us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:38.405676 (+ 82718us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.424899 (+ 19223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.424987 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.527095 (+102108us) raft_consensus.cc:1685] finished
0709 12:26:38.546172 (+ 19077us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.546343 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74196,"prepare.queue_time_us":111343,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":442237,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":17974,"spinlock_wait_cycles":47034368,"thread_start_us":166662,"threads_started":2,"wal-append.queue_time_us":171223}
I20260709 12:26:38.498644 10744 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.531401 10719 raft_consensus.cc:1081] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
W20260709 12:26:38.578724  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 154508us
I20260709 12:26:38.536861 10778 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:38.581231  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 132418us
I20260709 12:26:38.567981 10609 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.546332 10505 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.591861  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:38.610576  7331 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:38.593235  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:38.610883  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 175552us
W20260709 12:26:38.611621  6425 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.950s (SENT)
I20260709 12:26:38.594184  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:38.619939  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 196473us
I20260709 12:26:38.620803  6351 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12.
I20260709 12:26:38.621647  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.623714  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:38.633221  6425 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.980s (SENT)
W20260709 12:26:38.644083  6425 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.055s (SENT)
I20260709 12:26:38.644712  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:38.610033 10778 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.646373 10719 raft_consensus.cc:2764] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:38.647536 10465 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:38.648283 10465 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:38.648147 10719 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:38.648667 10753 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.609882 10698 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:26:38.571552 10611 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.651388  7320 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 12. Ops: []
I20260709 12:26:38.652048  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.652215 10611 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.578922 10480 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.652714 10719 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:38.652715  6300 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:38.654542  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.655601  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.636873 10792 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:38.656214  6300 raft_consensus.cc:2376] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14. Re-sending same reply.
I20260709 12:26:38.656374 10792 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:38.656772 10792 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:38.657182  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.657768  6300 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:38.657435 10760 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:38.658316 10756 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.875s	user 0.006s	sys 0.004s
I20260709 12:26:38.658732  6308 raft_consensus.cc:2368] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9.
I20260709 12:26:38.648705 10465 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.663666 10465 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:38.667332 10754 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.836s	user 0.003s	sys 0.007s
I20260709 12:26:38.674341  6303 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:38.675486 10708 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.528265 10464 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.613065 10609 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:38.679632 10603 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.813s	user 0.011s	sys 0.000s
W20260709 12:26:38.681826 10688 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.737s	user 0.011s	sys 0.000s
I20260709 12:26:38.682758 10686 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:38.685527  6425 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.077s (SENT)
I20260709 12:26:38.702741  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:38.685982 10672 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:38.687508  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:38.580083 10744 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.686393 10792 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.686074 10593 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 6, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.694624 10710 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:38.700183  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.698958 10773 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.324s	user 0.000s	sys 0.004s
W20260709 12:26:38.704834  6425 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.083s (SENT)
W20260709 12:26:38.697981 10786 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.129s	user 0.000s	sys 0.003s
I20260709 12:26:38.692159 10746 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.719477  6452 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:38.719871  6300 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:38.686883 10616 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:38.728114  6425 leader_election.cc:336] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.106s (SENT)
I20260709 12:26:38.693885  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:38.717128  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.714818  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.743683  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.744444  6302 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:38.715227  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.705338 10770 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:38.695874 10465 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.713222 10705 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:38.733173  6452 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:38.733737  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.732587  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:38.736265  6453 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:38.703133 10688 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.740103  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.745227  6305 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
W20260709 12:26:38.755539  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 292705us
I20260709 12:26:38.712232  6301 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:38.747078  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:38.749527  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55742.3us
I20260709 12:26:38.749125  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.702978 10603 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.763140  6302 raft_consensus.cc:2376] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13. Re-sending same reply.
I20260709 12:26:38.761979  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:38.762341  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:38.771618  6351 connection.cc:256] RPC call timeout handler was delayed by -2.880693531000361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:26:38.777132  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:38.778007  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.780541  6384 raft_consensus.cc:1217] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Deduplicated request from leader. Original: 6.3->[7.4-7.4]   Dedup: 7.4->[]
I20260709 12:26:38.780890  6382 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 13. Ops: [11.10-11.10]
I20260709 12:26:38.781195  6384 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:38.781845  6385 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:38.776410  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.783203  6382 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:38.783877  6384 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:38.784765  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:38.786351  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.789662  6459 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:38.789842  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.795681  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.796885  6452 raft_consensus.cc:2410] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:38.800585  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:38.801206  6300 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:38.801605  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.802227  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.806838  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.756208 10770 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:38.755918 10794 raft_consensus.cc:2764] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:26:38.821453  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 93387.2us
W20260709 12:26:38.822218  6425 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.108s (SENT)
I20260709 12:26:38.822467  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.822732  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:38.756023 10705 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:38.772606 10709 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:38.737378 10682 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.809628  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.809741  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.809168  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.817065  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:38.787580 10686 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:38.812150  6383 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
W20260709 12:26:38.813447  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 328233us
I20260709 12:26:38.813576  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.811357  6300 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:38.830106  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.827811  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.832993  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.837841  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:38.832399 10705 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:38.845055  6305 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:38.851751  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.774369 10710 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:26:38.846354  7316 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:38.849848  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.850247  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.821888  6450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:26:38.850590  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.853778  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:38.853963  6457 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 6. Current term is 7. Ops: []
I20260709 12:26:38.804458 10619 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:38.862390  6458 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:38.837002 10709 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.857452  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:38.874577  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.875196  6300 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:38.860389 10792 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.863054  6450 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:38.875869  6450 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:38.808880 10615 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:26:38.857620  6380 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:38.856570  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:38.862342  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.881898  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:38.871014  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.876070  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.877650  6455 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:26:38.886039  6300 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:38.841740 10798 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:38.878746  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.841948 10760 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.856055 10686 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:38.896318 10713 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:38.897364 10686 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.870775 10672 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.901741 10672 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:38.903230 10603 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.903647 10616 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.903250  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2108) took 1345 ms. Trace:
I20260709 12:26:38.904229  6381 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:38.888367  6383 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:38.905217  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.857167 10746 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.905890  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:38.905984 10746 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:38.906843 10509 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.907032  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.908079 10746 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.908208  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.888885  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:38.890108  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.893079  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.888412  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:38.909546  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.893353 10682 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.895659 10718 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:38.877516 10709 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:38.823135 10797 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:26:38.903962 10672 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:38.895401 10799 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:38.895471  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.889931  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:38.978344  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.995195  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.024408  6452 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:38.995393  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:39.025509  6300 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:38.995786  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.995388  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:38.979301  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:39.003086  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:39.028667  6453 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:38.888315 10666 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:39.004081  6457 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:26:39.012499  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.013154  6305 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:39.013828  6380 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:39.010188  6450 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.015666  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:38.896397 10609 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:38.904356  6377 rpcz_store.cc:276] 0709 12:26:37.557733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.569538 (+ 11805us) service_pool.cc:224] Handling call
0709 12:26:37.569949 (+   411us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:37.993859 (+423910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.993885 (+    26us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:37.994325 (+   440us) log.cc:844] Serialized 78 byte log entry
0709 12:26:38.375721 (+381396us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:38.406800 (+ 31079us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.447314 (+ 40514us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.447615 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.447742 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.903009 (+455267us) raft_consensus.cc:1685] finished
0709 12:26:38.903028 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:38.903178 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":431430,"prepare.queue_time_us":202745,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":89,"raft.queue_time_us":519989,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":37733,"thread_start_us":377999,"threads_started":1,"wal-append.queue_time_us":449138}
W20260709 12:26:39.010625  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.032498  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:39.011188 10744 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:39.035646  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:39.039322  6305 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:39.040361  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:39.041774  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 151923us
I20260709 12:26:39.042245 10672 raft_consensus.cc:2764] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:26:39.042829 10688 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:39.043169 10804 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
I20260709 12:26:39.043555  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:39.058077  6455 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:39.044329  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:39.044994  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:39.046794  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.058157 10796 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.063382  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.063319 10796 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.064664  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.065824  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:39.066051 10806 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:39.067157  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 627) took 1212 ms. Trace:
I20260709 12:26:39.067607 10796 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.067540  7320 rpcz_store.cc:276] 0709 12:26:37.858854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.858978 (+   124us) service_pool.cc:224] Handling call
0709 12:26:37.859291 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.651767 (+792476us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.066889 (+415122us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54004547d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:39.067063 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821198720}
I20260709 12:26:39.059617  6453 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:39.060113  6377 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:39.060295  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.069192  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:39.048151  6300 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:39.023193 10813 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:39.075771  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:39.051748  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.077641  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.078085  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:39.079100  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.080121  6377 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:26:39.052662  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:39.069587  7329 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:39.060770  6457 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:39.082149  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.066324  6381 raft_consensus.cc:2376] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10. Re-sending same reply.
W20260709 12:26:39.124414  7288 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 3.158s (SENT)
I20260709 12:26:39.059139  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.071480  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:39.145059  6300 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:39.074833  6455 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:39.257187  6457 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:39.258826  6455 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:26:39.261471  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 202425us
I20260709 12:26:39.264159  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.265316  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.291966  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:39.293648  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:39.295645  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 213582us
I20260709 12:26:39.296588  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.298023  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.303540  6458 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:39.305291  6300 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:39.294428  6381 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
W20260709 12:26:39.327280  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62020.6us
I20260709 12:26:39.333300  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.333976  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:39.334748  6300 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:39.335072  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 276661us
I20260709 12:26:39.336292  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.337284  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:39.336539  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.340027  6377 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:39.343241  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:39.062600 10744 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.285s
I20260709 12:26:39.333335 10708 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:39.061755 10480 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.343524 10796 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.362076  6300 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:26:39.364804  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 240452us
I20260709 12:26:39.295460 10416 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:39.294972 10619 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:39.392983 10619 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.393430 10619 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:39.349633 10505 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.381908 10708 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:39.395309 10708 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.366737  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:39.369518  7331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.019s	sys 0.042s
W20260709 12:26:39.396636  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60771.2us
I20260709 12:26:39.397584  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.397704  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.378834  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:39.322201 10465 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:39.399111  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:39.399333  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:39.399814  6300 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:39.336870 10616 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:39.345003 10593 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.334232 10531 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:39.344780 10746 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:39.384953 10774 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.126s	user 0.003s	sys 0.009s
I20260709 12:26:39.292382 10813 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.354378 10568 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.342111 10614 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.392458 10744 consensus_queue.cc:1243] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:39.399915 10416 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:39.400882  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:39.425743  6300 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:39.401172 10718 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.395727 10708 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:39.414299  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.398964  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.397228  6305 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
I20260709 12:26:39.413975 10593 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:39.401584 10616 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.439080 10616 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:39.439702 10593 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:39.377861 10796 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:39.442320  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.442847 10796 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.402784 10797 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:26:39.448388  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2112) took 1845 ms. Trace:
I20260709 12:26:39.450814 10568 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.453662 10568 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.457571 10619 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.448444  6382 raft_consensus.cc:1217] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Deduplicated request from leader. Original: 4.3->[7.4-7.4]   Dedup: 7.4->[]
I20260709 12:26:39.460697  6382 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:26:39.422999 10465 raft_consensus.cc:2764] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:26:39.449201  6378 rpcz_store.cc:276] 0709 12:26:37.602969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.603113 (+   144us) service_pool.cc:224] Handling call
0709 12:26:37.603498 (+   385us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:37.993369 (+389871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.993402 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:37.993960 (+   558us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.188611 (+194651us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:38.298781 (+110170us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.299036 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.322530 (+ 23494us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:38.663136 (+340606us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b48001f0158. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:26:38.682332 (+ 19196us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b180003a900. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:39.384604 (+702272us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b2400297ae8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:26:39.448064 (+ 63460us) raft_consensus.cc:1685] finished
0709 12:26:39.448098 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.448311 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398713,"prepare.queue_time_us":157922,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":394779,"raft.run_cpu_time_us":15871,"raft.run_wall_time_us":681542,"spinlock_wait_cycles":39652736,"thread_start_us":156670,"threads_started":2,"wal-append.queue_time_us":255145}
W20260709 12:26:39.468012  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68742.6us
I20260709 12:26:39.472756  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:39.428556 10416 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:39.423600 10747 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:39.477240  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2110) took 1898 ms. Trace:
I20260709 12:26:39.480856  6376 rpcz_store.cc:276] 0709 12:26:37.579208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.579343 (+   135us) service_pool.cc:224] Handling call
0709 12:26:37.579752 (+   409us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:37.952574 (+372822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:37.952612 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:37.953166 (+   554us) log.cc:844] Serialized 52 byte log entry
0709 12:26:37.969580 (+ 16414us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:37.969901 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.003881 (+ 33980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.003953 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.473106 (+1469153us) spinlock_profiling.cc:243] Waited 3.22 ms on lock 0x7b2400297ae8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f8fb0 00007f6fa2e6f488 00007f6fa2e6fbe8 00007f6fa2e47398 00007f6fa2e4276a 00007f6fa2e41bb6
0709 12:26:39.474519 (+  1413us) raft_consensus.cc:1685] finished
0709 12:26:39.476939 (+  2420us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.477159 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":520765,"prepare.queue_time_us":179,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":410349,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":48830,"spinlock_wait_cycles":7395712,"thread_start_us":16005,"threads_started":1,"wal-append.queue_time_us":16926}
I20260709 12:26:39.485052  6376 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:39.488914  6376 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:39.486855  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:39.491704 10718 raft_consensus.cc:2764] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:26:39.493206  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:39.425310 10614 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.501015  6303 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:39.501900  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1995) took 1140 ms. Trace:
I20260709 12:26:39.502298  6303 rpcz_store.cc:276] 0709 12:26:38.361806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.497659 (+1135853us) service_pool.cc:224] Handling call
0709 12:26:39.500789 (+  3130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.501591 (+   802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.501831 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.478313 10581 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:39.517103  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 180919us
I20260709 12:26:39.518105  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:39.497107 10705 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.634s	user 0.010s	sys 0.002s
I20260709 12:26:39.523944 10718 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:39.531322 10705 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.532045 10581 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:39.532780  7335 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:39.534509 10705 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:39.540300  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53540.8us
I20260709 12:26:39.498174 10568 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:39.528653 10721 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:39.550668 10465 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:39.551285 10619 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.523358 10262 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:26:39.556805  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:39.565766  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1975) took 1440 ms. Trace:
I20260709 12:26:39.566356  6307 rpcz_store.cc:276] 0709 12:26:38.125621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.147254 (+ 21633us) service_pool.cc:224] Handling call
0709 12:26:39.565532 (+1418278us) spinlock_profiling.cc:243] Waited 58.3 ms on lock 0x7b5400063bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:39.565688 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":133989760}
I20260709 12:26:39.554189 10721 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:39.573735  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 174846us
I20260709 12:26:39.574776  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:39.559286 10796 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:39.527274  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:39.626901  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55784us
I20260709 12:26:39.554476 10718 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.020s
W20260709 12:26:39.649961  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.571638 10606 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.018s	sys 0.029s
I20260709 12:26:39.569108 10813 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:39.555303 10797 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:26:39.570405 10759 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:39.558136 10593 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
W20260709 12:26:39.701009  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 173824us
I20260709 12:26:39.702996  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:39.705281 10595 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.705065 10825 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:39.705797 10595 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:39.708771 10823 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:39.709412 10823 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:39.710004 10568 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:39.581801 10721 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:39.722316  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.725379  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.727100  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:39.643762 10705 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:39.584074 10796 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:39.729918  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:39.695947 10606 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.018s	sys 0.029s
I20260709 12:26:39.708272 10595 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:39.750188 10823 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:39.750922 10614 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.733551 10759 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:39.752691 10759 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:39.733922 10622 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:39.731643 10784 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:39.729573 10822 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.764545 10705 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:39.768599 10262 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:39.801790 10792 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:39.808485 10622 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:39.809548 10262 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:39.852420  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 170159us
I20260709 12:26:39.802520 10708 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.861534  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:39.863268 10622 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.852977 10262 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:39.868012 10705 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.874404  6378 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:39.875137 10615 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.875451 10708 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:39.875684 10615 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.878103 10615 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:39.889611  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1970) took 1765 ms (client timeout 2077 ms). Trace:
I20260709 12:26:39.890458  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 565) took 1057 ms. Trace:
W20260709 12:26:39.908149  6308 rpcz_store.cc:269] 0709 12:26:38.124384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.152340 (+ 27956us) service_pool.cc:224] Handling call
0709 12:26:39.889297 (+1736957us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54002de1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:39.889517 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161655040}
I20260709 12:26:39.910425  6308 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:39.911794  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 579) took 1372 ms. Trace:
I20260709 12:26:39.912242  6308 rpcz_store.cc:276] 0709 12:26:38.539686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.909681 (+1369995us) service_pool.cc:224] Handling call
0709 12:26:39.910121 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.911481 (+  1360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.911731 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.908365  6383 rpcz_store.cc:276] 0709 12:26:38.833494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.394276 (+560782us) service_pool.cc:224] Handling call
0709 12:26:39.394745 (+   469us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:39.395193 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:39.395528 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.563607 (+168079us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.700001 (+136394us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:39.720931 (+ 20930us) log.cc:844] Serialized 52 byte log entry
0709 12:26:39.721156 (+   225us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:39.721206 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.721308 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.890174 (+168866us) raft_consensus.cc:1685] finished
0709 12:26:39.890196 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.890376 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150498,"prepare.queue_time_us":7690,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":764,"raft.queue_time_us":148570,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":16181,"spinlock_wait_cycles":8006400,"thread_start_us":167161,"threads_started":1,"wal-append.queue_time_us":384861}
I20260709 12:26:39.918942  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 581) took 1373 ms. Trace:
I20260709 12:26:39.919489  6308 rpcz_store.cc:276] 0709 12:26:38.540515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.913308 (+1372793us) service_pool.cc:224] Handling call
0709 12:26:39.913641 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.913832 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.913869 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.913920 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:39.913963 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.914029 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.914145 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.922845  6308 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 12:26:39.924397  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 583) took 1383 ms. Trace:
I20260709 12:26:39.926067  6308 rpcz_store.cc:276] 0709 12:26:38.541431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.921284 (+1379853us) service_pool.cc:224] Handling call
0709 12:26:39.921627 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.924160 (+  2533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.924336 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:39.810868 10796 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:39.927399  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 332330us
I20260709 12:26:39.919029 10704 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:39.935040 10709 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.905s	user 0.013s	sys 0.000s
W20260709 12:26:39.950508  6274 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.801s (SENT)
I20260709 12:26:39.956203  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:39.958623  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:39.939481 10673 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:39.960708  7316 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:39.961150 10673 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:39.964231  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:39.965337  6452 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:39.970016  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:39.963495 10581 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:39.968510 10747 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:39.942173 10809 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
W20260709 12:26:39.994856  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 267839us
W20260709 12:26:40.003933  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.988588 10262 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:39.964869 10721 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:39.964543 10709 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:40.009742 10616 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.004s
I20260709 12:26:40.038054  6381 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:40.039528  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:40.040521  6383 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
W20260709 12:26:40.047886  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.054693  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95162.4us
I20260709 12:26:40.055702 10673 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:40.042567 10616 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:40.072595 10833 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:26:40.075080  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.087380  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:40.083523 10834 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 12:26:40.084877 10772 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.508s	user 0.017s	sys 0.007s
W20260709 12:26:40.084401 10843 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:26:40.123101  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.130894  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.087891  6376 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:40.131414  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.132853 10845 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.133833 10615 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.140580 10772 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.566s	user 0.018s	sys 0.008s
I20260709 12:26:40.143035  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.142931 10615 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.144389 10845 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.145447 10615 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:40.146737  6425 leader_election.cc:336] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.126s (SENT)
I20260709 12:26:40.147392  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:40.135962 10809 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:40.136401  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 573) took 1116 ms. Trace:
I20260709 12:26:40.148451  6381 rpcz_store.cc:276] 0709 12:26:39.020144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.803813 (+783669us) service_pool.cc:224] Handling call
0709 12:26:39.804271 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.043272 (+239001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.136235 (+ 92963us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400065754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.136357 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":536909056}
I20260709 12:26:40.137228  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2126) took 1331 ms. Trace:
I20260709 12:26:40.150733 10673 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:40.135346  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1994) took 1786 ms. Trace:
I20260709 12:26:40.151329  6309 rpcz_store.cc:276] 0709 12:26:38.349474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.508630 (+159156us) service_pool.cc:224] Handling call
0709 12:26:38.509023 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.509296 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:39.985556 (+1476260us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.029242 (+ 43686us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.134858 (+105616us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b18001f6ba0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:26:40.135007 (+   149us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.135064 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:40.135116 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.135180 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.135295 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1593360,"prepare.queue_time_us":1503357,"prepare.run_cpu_time_us":217,"prepare.run_wall_time_us":2442,"raft.queue_time_us":107895,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":4789,"spinlock_wait_cycles":27187968,"thread_start_us":43075,"threads_started":1,"wal-append.queue_time_us":99062}
I20260709 12:26:40.147534 10616 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:40.143440  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:40.150141  6379 rpcz_store.cc:276] 0709 12:26:38.805899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.806660 (+   761us) service_pool.cc:224] Handling call
0709 12:26:38.807035 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.880394 (+ 73359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.880427 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:38.880469 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:38.880504 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.136986 (+1256482us) spinlock_profiling.cc:243] Waited 73.1 ms on lock 0x7b54001090d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.137044 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.137171 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168154880}
I20260709 12:26:40.134270 10792 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.164256 10809 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.166875 10606 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:40.167258 10606 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:40.168053 10606 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.168990 10606 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:40.169471 10809 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.172096 10845 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.173681  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:40.182588 10262 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:40.161463 10847 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:26:40.183702  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2556) took 1381 ms. Trace:
I20260709 12:26:40.184058  6382 rpcz_store.cc:276] 0709 12:26:38.802840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.805887 (+  3047us) service_pool.cc:224] Handling call
0709 12:26:39.448314 (+642427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.461144 (+ 12830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.183517 (+722373us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400064850. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.183645 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476646144}
I20260709 12:26:40.184746 10760 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:40.185119 10760 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:40.182792 10802 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.323s	user 0.011s	sys 0.016s
W20260709 12:26:40.192332  6301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 1960) took 2072 ms (client timeout 1730 ms). Trace:
I20260709 12:26:40.192597  6310 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:26:40.192714  6301 rpcz_store.cc:269] 0709 12:26:38.120037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.122628 (+  2591us) service_pool.cc:224] Handling call
0709 12:26:40.192153 (+2069525us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54002debd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.192266 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1315019136}
I20260709 12:26:40.197391  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2740) took 1462 ms. Trace:
I20260709 12:26:40.197746  6459 rpcz_store.cc:276] 0709 12:26:38.738991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.739168 (+   177us) service_pool.cc:224] Handling call
0709 12:26:40.197185 (+1458017us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b54001635d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.197333 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37058944}
I20260709 12:26:40.198127 10822 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.200946  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.202329  6458 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:40.203511  6381 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:40.204850  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:40.199071 10822 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.185875 10760 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.207615  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:40.208977  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:40.212930  6458 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:40.212756  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.215705  7314 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:40.216281  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.216630  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.217165  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.217514  6455 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 7: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:40.218353  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1996) took 1847 ms. Trace:
I20260709 12:26:40.218910  6450 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:40.218812  6303 rpcz_store.cc:276] 0709 12:26:38.371104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.503882 (+1132778us) service_pool.cc:224] Handling call
0709 12:26:39.513648 (+  9766us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.514130 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.514173 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.514767 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.787006 (+272239us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:40.002255 (+215249us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.218005 (+215750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.218086 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.218130 (+    44us) raft_consensus.cc:1685] finished
0709 12:26:40.218153 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.218284 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216403,"prepare.queue_time_us":240257,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":334,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":9338,"thread_start_us":467099,"threads_started":2,"wal-append.queue_time_us":404443}
I20260709 12:26:40.207011  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.225214  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:40.228238  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:40.229897  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:59648 (request call id 2245) took 1415 ms. Trace:
I20260709 12:26:40.230506  6450 rpcz_store.cc:276] 0709 12:26:38.804512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.805110 (+   598us) service_pool.cc:224] Handling call
0709 12:26:40.208080 (+1402970us) spinlock_profiling.cc:243] Waited 6 ms on lock 0x7b54001c1954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa329ea3d 00007f6fa826ecd0 00007f6fa17f3254 00007f6fa17f31e2 00007f6fa17f3111 00007f6fa17f308c 00007f6fa17f2342 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.219396 (+ 11316us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13797248}
I20260709 12:26:40.243456  6383 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:40.243247  6459 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:40.245810  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.251258  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:40.174211 10792 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.256793  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:40.260246  7314 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.257849  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:40.258237  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.190908 10673 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:26:40.265949  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.271100 10778 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:40.271306  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:40.274171  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 564) took 1449 ms. Trace:
I20260709 12:26:40.274552  6384 rpcz_store.cc:276] 0709 12:26:38.825135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.856271 (+ 31136us) service_pool.cc:224] Handling call
0709 12:26:40.273974 (+1417703us) spinlock_profiling.cc:243] Waited 914 ms on lock 0x7b54001941d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.274096 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2101461248}
I20260709 12:26:40.275027  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.275724  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:40.276345  6384 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:40.276942  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.278878 10792 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.279279  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 590) took 1463 ms. Trace:
I20260709 12:26:40.279688  7329 rpcz_store.cc:276] 0709 12:26:38.816374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.816490 (+   116us) service_pool.cc:224] Handling call
0709 12:26:39.069418 (+252928us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.081625 (+ 12207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.279114 (+1197489us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54004547d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.279225 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580954368}
I20260709 12:26:40.242416 10678 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:26:40.280990 10822 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.286314  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2130) took 1476 ms. Trace:
I20260709 12:26:40.286726  6377 rpcz_store.cc:276] 0709 12:26:38.810294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.340698 (+530404us) service_pool.cc:224] Handling call
0709 12:26:39.340941 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.367600 (+ 26659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.367634 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.367685 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:39.367726 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.286097 (+918371us) spinlock_profiling.cc:243] Waited 26.4 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.286144 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.286260 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60786048}
W20260709 12:26:40.287402 10416 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.791s	user 0.014s	sys 0.000s
I20260709 12:26:40.287964  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:40.288591  6377 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:40.288142 10416 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.289901  6379 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:40.294906  6383 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:40.298102  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:40.295673  6377 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:40.294279  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1993) took 1963 ms. Trace:
I20260709 12:26:40.304878  6310 rpcz_store.cc:276] 0709 12:26:38.331077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.335276 (+  4199us) service_pool.cc:224] Handling call
0709 12:26:40.192268 (+1856992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.192980 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.294050 (+101070us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b54002debd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:40.294208 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4208311040}
W20260709 12:26:40.307243  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.318043  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 563) took 1507 ms. Trace:
I20260709 12:26:40.318928  6385 rpcz_store.cc:276] 0709 12:26:38.810990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.344019 (+533029us) service_pool.cc:224] Handling call
0709 12:26:39.344337 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.441973 (+ 97636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.442011 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:39.442060 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:39.442101 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.317604 (+875503us) spinlock_profiling.cc:243] Waited 97.4 ms on lock 0x7b54003e4ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.317694 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.317938 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":223958400}
I20260709 12:26:40.321446  6385 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:26:40.321902  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.322885  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.323828  6452 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:40.325029  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.250845 10709 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.306113  7314 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:40.332453  6385 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 12:26:40.337962  6310 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:40.339775  6384 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:40.339905  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2183) took 1562 ms. Trace:
I20260709 12:26:40.340690  6310 rpcz_store.cc:276] 0709 12:26:38.777334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.305763 (+1528429us) service_pool.cc:224] Handling call
0709 12:26:40.337730 (+ 31967us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.339595 (+  1865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.339848 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:40.343782  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.345284  6452 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:40.346560  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:40.347097  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68322.9us
I20260709 12:26:40.348250  7319 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:40.351428  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.350167  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 578) took 1810 ms. Trace:
I20260709 12:26:40.355444  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1998) took 1962 ms. Trace:
I20260709 12:26:40.355679  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1987) took 2131 ms. Trace:
I20260709 12:26:40.354065  6305 rpcz_store.cc:276] 0709 12:26:38.539408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.864692 (+1325284us) service_pool.cc:224] Handling call
0709 12:26:39.865027 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.865368 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.865399 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.865850 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.991472 (+125622us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:40.044477 (+ 53005us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.172976 (+128499us) spinlock_profiling.cc:243] Waited 3.04 ms on lock 0x7b1800446080. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:40.328499 (+155523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.328602 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.328660 (+    58us) raft_consensus.cc:1685] finished
0709 12:26:40.349877 (+ 21217us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.350083 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128344,"prepare.queue_time_us":56908,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":92177,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"spinlock_wait_cycles":6990592,"thread_start_us":318461,"threads_started":2,"wal-append.queue_time_us":181198}
I20260709 12:26:40.355883  6307 rpcz_store.cc:276] 0709 12:26:38.393285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.567610 (+1174325us) service_pool.cc:224] Handling call
0709 12:26:39.568032 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.574972 (+  6940us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.575002 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.575448 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.008555 (+433107us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:40.057134 (+ 48579us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.355163 (+298029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.355228 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.355265 (+    37us) raft_consensus.cc:1685] finished
0709 12:26:40.355278 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.355389 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":340787,"prepare.queue_time_us":52813,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":6573,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":65402,"spinlock_wait_cycles":1532160,"thread_start_us":297732,"threads_started":1,"wal-append.queue_time_us":386277}
W20260709 12:26:40.346846  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 121548us
I20260709 12:26:40.356015  6304 rpcz_store.cc:276] 0709 12:26:38.223978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.224121 (+   143us) service_pool.cc:224] Handling call
0709 12:26:38.224425 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.224646 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:38.298790 (+ 74144us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.041273 (+742483us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b1800390340. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:39.942021 (+900748us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b1800390340. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa6c9f3c7 00007f6fa32b19a7 00007f6fa32c10bf 00007f6fa32c1038
0709 12:26:39.957949 (+ 15928us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.354891 (+396942us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b1800390340. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:26:40.355264 (+   373us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.355390 (+   126us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.355425 (+    35us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:40.355473 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.355535 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.355631 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1083125,"prepare.queue_time_us":1750396,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":72,"raft.queue_time_us":100,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":5588,"spinlock_wait_cycles":111804928,"thread_start_us":231266,"threads_started":3,"wal-append.queue_time_us":467261,"wal-append.run_cpu_time_us":3147,"wal-append.run_wall_time_us":1027847}
I20260709 12:26:40.365751  6310 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: [10.5-12.6]
I20260709 12:26:40.366302  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.366624  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2184) took 1589 ms. Trace:
I20260709 12:26:40.367368  6310 rpcz_store.cc:276] 0709 12:26:38.777564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.341622 (+1564058us) service_pool.cc:224] Handling call
0709 12:26:40.365465 (+ 23843us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:40.366236 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.366550 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:40.368942  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.380129  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2188) took 1500 ms. Trace:
I20260709 12:26:40.380370  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:40.380613  6305 rpcz_store.cc:276] 0709 12:26:38.858448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.357460 (+1499012us) service_pool.cc:224] Handling call
0709 12:26:40.358020 (+   560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.358247 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.358324 (+    77us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.358471 (+   147us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:40.358542 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.358614 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.359043 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:40.336957 10400 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:40.411190  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59727.9us
W20260709 12:26:40.411326  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50081.3us
I20260709 12:26:40.351035 10760 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:40.360457 10713 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.326658 10822 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:26:40.348745 10795 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:26:40.357746 10673 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:26:40.411746 10710 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:40.410733 10335 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.363590 10709 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:26:40.410976 10615 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.382988 10616 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 12:26:40.431838 10859 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:40.425873 10713 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.448585 10616 consensus_queue.cc:1243] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:40.438378 10335 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.441524 10615 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.467378 10416 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.476624 10859 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.477531 10859 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:40.489187  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70455.6us
I20260709 12:26:40.491012  7331 raft_consensus.cc:2410] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:40.492424  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.502852  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.486203 10335 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.504742  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:40.511931  6454 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:40.503165 10678 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:40.507233 10588 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:40.500042 10595 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 12:26:40.542544 10850 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.239s	user 0.001s	sys 0.001s
W20260709 12:26:40.548592 10861 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:26:40.527451 10588 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:40.530866 10840 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:40.558213  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 588) took 1754 ms. Trace:
I20260709 12:26:40.558859  7335 rpcz_store.cc:276] 0709 12:26:38.803532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.803662 (+   130us) service_pool.cc:224] Handling call
0709 12:26:38.803962 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.533185 (+729223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.557795 (+1024610us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b54007c0e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.558109 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1675690880}
I20260709 12:26:40.544183  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 1999) took 2139 ms. Trace:
W20260709 12:26:40.553737 10864 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:26:40.552631 10666 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:26:40.555167 10860 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:26:40.583078  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.550649 10870 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:40.586474  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 499149us
I20260709 12:26:40.587499  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.588490  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:40.568965  6300 rpcz_store.cc:276] 0709 12:26:38.405125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.760439 (+1355314us) service_pool.cc:224] Handling call
0709 12:26:39.760870 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.761285 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.761324 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.761882 (+   558us) log.cc:844] Serialized 26 byte log entry
0709 12:26:39.944190 (+182308us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:40.096335 (+152145us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.543852 (+447517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.543931 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.543977 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:40.543988 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.544126 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":298128,"prepare.queue_time_us":84484,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":306404,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":7848,"thread_start_us":443429,"threads_started":3,"wal-append.queue_time_us":251757}
I20260709 12:26:40.552050 10678 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:26:40.607191  7314 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:40.584757 10666 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:40.610416  7335 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:40.586645 10870 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.613761  7335 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:40.598186  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:40.595952 10866 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:40.646881  7335 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.653856  7335 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:40.607101 10710 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.651500 10822 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:26:40.655859 10710 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:26:40.607364 10713 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.659421 10666 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.661355 10666 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:40.652417 10864 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:26:40.690776  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92680.6us
I20260709 12:26:40.692056  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.645464 10870 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.613520 10696 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:26:40.669432 10710 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:40.709070  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.670176 10867 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:40.702675 10874 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:26:40.675226 10866 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:40.738118  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46130us
I20260709 12:26:40.729751 10859 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:40.732936 10867 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.652081 10588 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:40.748194  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 237173us
I20260709 12:26:40.737957 10875 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.760986  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:40.760968  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:40.751396 10878 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.747429 10867 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:40.756461 10873 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.763453 10859 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.764518 10876 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.742157 10416 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:40.765969 10673 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:26:40.777140 10872 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:40.795456 10588 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.768034 10875 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.795845 10871 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:40.780210 10881 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:26:40.804481 10416 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:40.833595  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70389.8us
I20260709 12:26:40.818200 10667 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:26:40.836835  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 92838.6us
I20260709 12:26:40.849795  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:40.823843 10887 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:40.798777 10859 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.855468 10887 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.856992 10859 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.857616 10859 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:40.857784 10887 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:40.858057 10859 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.853750  6385 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:40.858369 10887 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.794227 10878 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.861636 10878 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.858439 10859 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:40.863822  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.859675  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:40.859351 10887 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:40.883862  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86684.1us
I20260709 12:26:40.884776  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:40.796013 10873 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.890477  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:40.791308 10866 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:40.899968  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.961562  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71014.2us
I20260709 12:26:40.944510 10747 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:40.963385  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.909866 10878 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:26:40.912735 10890 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:26:40.914994 10796 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:40.915328 10882 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:26:40.915570 10886 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:40.917997  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.962846 10896 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:40.919304 10885 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:40.854972 10667 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:40.803783 10876 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.944732  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 569) took 2068 ms. Trace:
I20260709 12:26:40.833120 10893 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:26:40.825414 10872 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:40.954809  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2191) took 2060 ms. Trace:
I20260709 12:26:40.999119  6378 rpcz_store.cc:276] 0709 12:26:38.876074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.479072 (+602998us) service_pool.cc:224] Handling call
0709 12:26:39.479387 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.874837 (+395450us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.944368 (+1069531us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.944642 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907999104}
I20260709 12:26:40.911094  7335 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:40.970045 10868 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:40.967046  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:40.919132  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 575) took 1387 ms. Trace:
I20260709 12:26:41.005009  6376 rpcz_store.cc:276] 0709 12:26:39.535888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.855399 (+319511us) service_pool.cc:224] Handling call
0709 12:26:39.919193 (+ 63794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.144636 (+225443us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.918820 (+774184us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:40.919066 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380957952}
I20260709 12:26:40.979645  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:40.870060 10887 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:40.944689 10416 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:41.014958  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47988.2us
I20260709 12:26:41.015323  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.016764  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:40.998379  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:40.999351  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:41.000119  6307 rpcz_store.cc:276] 0709 12:26:38.893970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.358384 (+1464414us) service_pool.cc:224] Handling call
0709 12:26:40.359350 (+   966us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.762126 (+402776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.762157 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.762742 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.802514 (+ 39772us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:40.818639 (+ 16125us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.954415 (+135776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.954523 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.954577 (+    54us) raft_consensus.cc:1685] finished
0709 12:26:40.954591 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.954736 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65269,"prepare.queue_time_us":60681,"raft.queue_time_us":160633,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":127,"spinlock_wait_cycles":111203968,"thread_start_us":497498,"threads_started":4,"wal-append.queue_time_us":79624}
I20260709 12:26:41.003728  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.019788  6456 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
W20260709 12:26:40.982964 10888 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:26:41.008116  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2196) took 1957 ms. Trace:
I20260709 12:26:40.967814 10747 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:40.908958 10889 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.125s
I20260709 12:26:41.020015  6451 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:26:41.007104  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:41.021286  6310 rpcz_store.cc:276] 0709 12:26:39.050868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.368400 (+1317532us) service_pool.cc:224] Handling call
0709 12:26:40.416424 (+ 48024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.547703 (+131279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.547732 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.548183 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.734227 (+186044us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:40.817188 (+ 82961us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.960392 (+143204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.960906 (+   514us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.007804 (+ 46898us) raft_consensus.cc:1685] finished
0709 12:26:41.007837 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.008028 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9220,"prepare.queue_time_us":98593,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":104725,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":150,"thread_start_us":492298,"threads_started":4,"wal-append.queue_time_us":247507}
I20260709 12:26:41.004758 10886 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.024603  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.010982 10622 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:41.030021  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:41.036244  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2195) took 1985 ms. Trace:
I20260709 12:26:40.985221 10896 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.032645  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 592) took 2420 ms. Trace:
I20260709 12:26:41.001199 10845 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:26:41.031909 10903 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:41.024528 10913 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:41.067139  6304 rpcz_store.cc:276] 0709 12:26:39.050578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.375731 (+1325153us) service_pool.cc:224] Handling call
0709 12:26:40.376121 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.457897 (+ 81776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.457945 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.461517 (+  3572us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.591912 (+130395us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:40.591976 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.592095 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.035944 (+443849us) raft_consensus.cc:1685] finished
0709 12:26:41.035963 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.036179 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40382,"raft.queue_time_us":99891,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"thread_start_us":144028,"threads_started":2,"wal-append.queue_time_us":265150}
I20260709 12:26:41.071264  6309 rpcz_store.cc:276] 0709 12:26:38.612523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.153244 (+1540721us) service_pool.cc:224] Handling call
0709 12:26:40.153578 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.433911 (+280333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.433951 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.442874 (+  8923us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.767061 (+324187us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:40.957963 (+190902us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.958181 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.958268 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.032338 (+ 74070us) raft_consensus.cc:1685] finished
0709 12:26:41.032368 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.032563 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":204921,"prepare.queue_time_us":95697,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":317211,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":11833,"spinlock_wait_cycles":14334208,"thread_start_us":552844,"threads_started":3,"wal-append.queue_time_us":362777}
I20260709 12:26:41.093981  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.091255 10886 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:26:41.100113 10667 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.101974 10893 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:41.103513  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:41.103545 10890 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:26:41.120695  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:41.092185 10696 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:41.073203 10875 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.068892 10910 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:41.067418 10887 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:41.070884 10673 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:41.037074 10416 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:41.139803 10718 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:41.140059  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 593) took 2527 ms. Trace:
I20260709 12:26:41.140472 10718 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:41.144280 10666 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:41.144498 10896 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:41.144692  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.145041 10896 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:41.144908 10666 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.147097 10666 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:41.091455 10879 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:41.149299 10879 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.145583 10896 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.149706 10879 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:41.150733  6383 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:41.151759 10760 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.729s	user 0.012s	sys 0.001s
I20260709 12:26:41.152549 10872 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.152670 10615 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.153276  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.155965 10873 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:41.159653 10909 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:41.129007 10886 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:41.160621 10886 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.160967 10886 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.161682 10898 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:41.162083 10898 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:41.162395 10792 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:41.163017 10792 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:41.140578  6301 rpcz_store.cc:276] 0709 12:26:38.613150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.193589 (+1580439us) service_pool.cc:224] Handling call
0709 12:26:40.193925 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.758611 (+564686us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.758646 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.763638 (+  4992us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.963990 (+200352us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:40.976550 (+ 12560us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.028853 (+ 52303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.028927 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.139849 (+110922us) raft_consensus.cc:1685] finished
0709 12:26:41.139869 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.140001 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166466,"prepare.queue_time_us":12391,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":1193,"raft.queue_time_us":275245,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":12908,"spinlock_wait_cycles":1111296,"thread_start_us":624010,"threads_started":3,"wal-append.queue_time_us":262058}
I20260709 12:26:41.162958 10898 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.164474 10898 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:41.164036 10792 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.127003  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.103683 10889 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.134537 10673 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:41.125957 10696 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.168953  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.141834 10718 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.168198  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:41.185334  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 145626us
W20260709 12:26:41.159660 10704 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:26:41.169104  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.189483  6379 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:41.165192 10792 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:41.194247  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:41.171239 10878 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.198676  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.197266  7318 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:41.237205  6382 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:41.325012 10868 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:26:41.326357  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 127750us
I20260709 12:26:41.327164  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.327885 10868 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:41.329470  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.331950 10868 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:41.335698 10588 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:41.336253 10622 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.336428 10588 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.336730 10622 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.338960 10588 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:41.339202 10615 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:41.339671 10588 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:41.340545 10840 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:41.340148 10615 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.341015 10840 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:41.340245 10588 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.343386 10615 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:41.343583 10588 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:41.252060  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.189471 10696 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:41.344404  7318 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 12:26:41.348552 10892 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.481s	user 0.002s	sys 0.000s
I20260709 12:26:41.348650 10879 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:41.350160  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.230758 10905 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:41.224552 10915 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:41.203118 10887 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:41.246299  6458 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:41.354305 10909 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.355089  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2260) took 1127 ms. Trace:
I20260709 12:26:41.354992 10909 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.355569  6458 rpcz_store.cc:276] 0709 12:26:40.227502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.227708 (+   206us) service_pool.cc:224] Handling call
0709 12:26:41.245477 (+1017769us) spinlock_profiling.cc:243] Waited 922 us on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:26:41.245802 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.354786 (+108984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.355024 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2120576}
I20260709 12:26:41.356207  6309 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:41.357086 10909 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:41.357777  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:41.244539  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 633) took 2383 ms. Trace:
I20260709 12:26:41.358564 10828 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.240334  6453 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:41.359476  6453 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:41.359289 10828 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.353032 10696 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.360296  6453 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.361496 10828 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.224385 10673 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.372474  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.372674  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.241040 10859 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.205217 10595 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:41.279340 10721 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.250015 10886 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.218719 10873 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:26:41.185258 10889 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.245440 10896 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:41.371171 10876 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.339378  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.290771  6381 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:41.351693 10879 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.352692  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:41.358808  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:41.410985  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 58424.8us
I20260709 12:26:41.362389  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:41.358667  7319 rpcz_store.cc:276] 0709 12:26:38.861593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.861905 (+   312us) service_pool.cc:224] Handling call
0709 12:26:38.886702 (+ 24797us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.348655 (+1461953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.244177 (+895522us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:41.244449 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3360805760}
I20260709 12:26:41.362830  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:41.362830  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.412181  6307 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:26:41.361534  6453 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:41.364800  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:41.415464  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:41.416530  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:41.417081  6302 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10. Re-sending same reply.
I20260709 12:26:41.417866  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:41.366196  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:41.206252  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.279000 10867 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:41.238454 10870 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.512s	user 0.007s	sys 0.005s
W20260709 12:26:41.375905  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2756) took 2468 ms (client timeout 1998 ms). Trace:
I20260709 12:26:41.375887  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 591) took 2763 ms. Trace:
W20260709 12:26:41.379568  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.380697  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:41.421060  6308 rpcz_store.cc:276] 0709 12:26:38.612246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.929247 (+1317001us) service_pool.cc:224] Handling call
0709 12:26:40.076226 (+146979us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.611655 (+535429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.611701 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.612293 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.800348 (+188055us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:40.815207 (+ 14859us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.927746 (+112539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.927823 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.348258 (+420435us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b240017b0e8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:26:41.366239 (+ 17981us) raft_consensus.cc:1685] finished
0709 12:26:41.375673 (+  9434us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.375821 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":293774,"prepare.queue_time_us":61459,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":386369,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":10630,"spinlock_wait_cycles":47559424,"thread_start_us":457705,"threads_started":4,"wal-append.queue_time_us":216267}
I20260709 12:26:41.422931  7318 raft_consensus.cc:2410] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
W20260709 12:26:41.394320 10916 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:26:41.423480  6454 rpcz_store.cc:269] 0709 12:26:38.907155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.995052 (+ 87897us) service_pool.cc:224] Handling call
0709 12:26:41.375744 (+2380692us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400266954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:41.375834 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3412720512}
I20260709 12:26:41.424697  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:41.362293 10696 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.418738  6305 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:41.412212  6300 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:41.422565  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:41.384191  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.428438  6307 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:41.423779  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84486us
I20260709 12:26:41.429239  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.429688  6384 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:41.430085  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:41.430933  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.431151  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.432165  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.433524  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.433609  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:41.434468  6305 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:41.281524  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 853) took 1078 ms. Trace:
I20260709 12:26:41.436239  7331 rpcz_store.cc:276] 0709 12:26:40.202957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.206777 (+  3820us) service_pool.cc:224] Handling call
0709 12:26:41.281174 (+1074397us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b540016dd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:41.281402 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612306688}
I20260709 12:26:41.396358 10622 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.383934 10595 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.443382  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.444185  6453 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:41.446949  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.451597  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.452222  6450 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:41.447517  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.453995  6458 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:41.454370  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:41.455058  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:41.455790  6458 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:41.456928  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.455083  6310 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:41.406904 10673 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:41.449162  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:41.474207  6308 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:41.475548  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.466015  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.485749  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:41.490934  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60954.4us
I20260709 12:26:41.492075  6300 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:41.493567 10588 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:41.495023 10910 raft_consensus.cc:2764] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:41.497431 10568 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:41.499289  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.499523  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.499876  6310 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:26:41.500329  6310 pending_rounds.cc:85] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:26:41.460318  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.457747 10619 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:41.500588  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.500906 10619 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:41.466034 10889 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:41.501420 10619 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:41.500566  6310 pending_rounds.cc:107] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:26:41.503767  6310 raft_consensus.cc:2889] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:41.443981 10870 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.421200 10876 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:41.506752 10876 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.507177 10876 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:41.453260  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.454811 10595 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:41.468961 10869 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:41.473376  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:41.501437  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.500100  6380 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:41.510670  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.511255  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:41.511317  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.512305  6384 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:41.512176  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.513348  6378 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:41.511556  6377 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:26:41.515004  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.515827  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.516096  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:41.516301  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.516690  6377 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.506227 10870 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:41.516870  6451 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:41.519904  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:41.518451  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.519551  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.521431  6450 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:41.448694 10721 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:41.518652  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 314184us
W20260709 12:26:41.522593  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.523128  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.541790  6308 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:41.517877  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.524560  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.545610  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:41.547375  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2135) took 1342 ms (client timeout 1644 ms). Trace:
I20260709 12:26:41.535952  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:41.547950  6381 rpcz_store.cc:269] 0709 12:26:40.204554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.204647 (+    93us) service_pool.cc:224] Handling call
0709 12:26:41.547130 (+1342483us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54003d36d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:41.547288 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477544192}
W20260709 12:26:41.515621 10759 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:26:41.546360  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:41.552340  6308 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:26:41.554610  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 102620us
I20260709 12:26:41.555425  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.557721  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.569507  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.575116 10903 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:41.575949  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:41.576851  6457 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:41.527336 10878 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.550384 10760 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:41.555984 10667 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:41.579027 10667 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:41.579561 10667 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:41.584372 10619 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.589356  6458 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:41.556352 10875 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:41.636298 10875 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.636827 10875 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.534627 10866 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:26:41.534845 10872 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:41.638919 10872 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.639070 10866 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:41.639490 10872 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:41.639655 10866 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:41.641010 10866 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.643461  6452 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:26:41.592083  6379 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:41.594869 10893 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.594801 10760 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:41.595834  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.600138  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:41.646512  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46442.8us
I20260709 12:26:41.646667 10922 raft_consensus.cc:2764] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 12:26:41.647261  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50780us
I20260709 12:26:41.647374  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.570030 10913 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:41.527698 10868 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:41.648936 10868 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:41.650192 10866 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:41.613978  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.656352 10606 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:41.657105  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:41.658890 10927 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:41.660879 10927 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:41.661382 10927 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:41.662302 10927 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.674238  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:41.676347  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:41.676658 10868 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:41.678680  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:41.680169 10872 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.686587 10828 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:41.687340 10868 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.688046 10868 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.616345  6456 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:41.601555 10840 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:41.692642 10840 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:41.697453 10872 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.698751 10868 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.699000  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:41.699884  6457 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:41.623245  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:41.707054  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:41.625334  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:41.612798 10696 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:41.709412 10696 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.709949 10696 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:41.691576  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:41.736384  6307 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:41.802006  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:41.805368  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.812520  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.834208  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.794903 10915 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:41.594957 10704 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.201s
W20260709 12:26:41.838047  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 409269us
W20260709 12:26:41.860320  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 196034us
I20260709 12:26:41.865821  6377 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:41.871500  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:41.979627  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 108136us
I20260709 12:26:41.982044  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:41.985111  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:41.991006  6384 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:41.996812  6307 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: [9.6-11.7]
I20260709 12:26:41.971038  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 594) took 3282 ms. Trace:
I20260709 12:26:41.999017  6303 rpcz_store.cc:276] 0709 12:26:38.688388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.220981 (+1532593us) service_pool.cc:224] Handling call
0709 12:26:40.245696 (+ 24715us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.446149 (+200453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.446181 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.446857 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:26:41.593384 (+1146527us) spinlock_profiling.cc:243] Waited 2.16 ms on lock 0x7b180029a160. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:41.969814 (+376430us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:41.969937 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.970099 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.970211 (+   112us) raft_consensus.cc:1685] finished
0709 12:26:41.970240 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.970726 (+   486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115930,"raft.queue_time_us":213050,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":4126,"spinlock_wait_cycles":4966144,"thread_start_us":461448,"threads_started":2,"wal-append.queue_time_us":1474333}
I20260709 12:26:42.002245  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.008270  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:42.013087  6450 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:26:42.023658  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.876714  6457 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:41.895598 10619 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:41.840672 10696 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.056143 10262 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:42.057711 10922 raft_consensus.cc:2764] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
W20260709 12:26:41.877661 10792 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.672s	user 0.010s	sys 0.000s
I20260709 12:26:41.876147 10875 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:41.921440 10876 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.060217  6376 raft_consensus.cc:2410] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:41.892738 10912 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:26:42.062018  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:42.062503 10912 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:42.062826  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:42.065096 10704 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:42.064459 10912 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.039098 10416 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:42.039915 10673 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.533s	user 0.009s	sys 0.000s
I20260709 12:26:42.041131 10568 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:42.041605 10678 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:42.069250 10568 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:42.069584 10678 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.070544  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:42.071228  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:42.071118 10568 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.043318 10760 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.046900 10778 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:42.062325 10925 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:42.073589 10778 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:42.049546  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.078172  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.062064 10875 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.071753  7320 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:42.072667 10760 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.083369  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.084709 10696 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.046008 10759 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:26:42.085932 10760 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.072850 10678 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:42.042171 10588 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:41.904903 10926 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:26:42.073609 10912 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:26:42.079427 10778 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.049073 10868 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:42.094630 10868 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:42.096405 10868 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.073815 10925 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.062029 10876 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.100826 10588 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:42.101680 10925 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.101799 10588 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.104688 10588 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.105271 10588 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:42.116449  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55103us
I20260709 12:26:42.112744 10588 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.133411  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:42.143535  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.135416  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:42.146165 10667 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.015s
I20260709 12:26:42.146895 10667 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.146215  6456 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.162341 10747 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.166832  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.169451 10886 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:42.185750 10606 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.529s	user 0.005s	sys 0.017s
I20260709 12:26:42.193907  6450 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:42.197285  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:42.192377 10606 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:26:42.243429 10939 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.329s	user 0.000s	sys 0.004s
I20260709 12:26:42.245153  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:42.246511  6381 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:42.247139 10949 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 12:26:42.250037 10929 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
W20260709 12:26:42.252103  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.252331  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:42.253850 10912 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:42.254617 10867 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.262118  7320 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:42.227300 10667 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.241852 10588 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.268793  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:42.285117  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.287820  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:42.288904  6376 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:42.282066  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:42.291679  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:42.292495  6384 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.294271  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.293776  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2149) took 1443 ms. Trace:
I20260709 12:26:42.286065  6303 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:26:42.300165  7320 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:42.245165 10859 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.297688 10871 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:42.300222  6382 rpcz_store.cc:276] 0709 12:26:40.849711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.854534 (+  4823us) service_pool.cc:224] Handling call
0709 12:26:40.854791 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.324021 (+469230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.293267 (+969246us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400064d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:42.293586 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878591232}
I20260709 12:26:42.305051  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.312167  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:42.313521  6379 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:42.316351  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:42.316691  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.317970  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.318625  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.319118  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.319818 10913 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:42.320655  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.321084 10871 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:42.321485 10568 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:42.322211 10568 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:42.322542  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.322320 10871 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.323566 10568 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.324079  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.276372 10912 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:42.333925  6425 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.730s (SENT)
I20260709 12:26:42.336329 10912 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:42.339078 10595 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.831s	user 0.002s	sys 0.020s
I20260709 12:26:42.339705 10595 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.343143  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:42.350715  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:42.352077  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.352988  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:42.358654  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.361985  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54421.7us
I20260709 12:26:42.363880  6377 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:26:42.364259  6425 connection.cc:256] RPC call timeout handler was delayed by -1.5735514279999734s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.196s for a response
I20260709 12:26:42.380554  6456 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
W20260709 12:26:42.366549 10866 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.714s	user 0.007s	sys 0.024s
I20260709 12:26:42.345438  6384 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:42.373739  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.351567  6453 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:42.362746 10611 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:42.378412  6310 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:42.423024  6302 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:42.423334  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:42.424556  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.425720  6302 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:42.430958  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.433285  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.443506  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:42.430948 10792 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:42.450644  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.452657  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.442937  6307 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:42.456804 10792 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:42.460685  6425 connection.cc:256] RPC call timeout handler was delayed by -1.040615613999762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
I20260709 12:26:42.462934  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:42.463817  6457 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:42.465746 10870 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:42.457221  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.471230  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2013) took 1063 ms. Trace:
I20260709 12:26:42.471738  6305 rpcz_store.cc:276] 0709 12:26:41.403688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.296059 (+892371us) service_pool.cc:224] Handling call
0709 12:26:42.296753 (+   694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.297420 (+   667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.297448 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.298137 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.411625 (+113488us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:42.411698 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.411796 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.466806 (+ 55010us) raft_consensus.cc:1685] finished
0709 12:26:42.466833 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.466998 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2912,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":6738,"thread_start_us":112979,"threads_started":1,"wal-append.queue_time_us":164714}
I20260709 12:26:42.478662 10588 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:42.479663 10588 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.483376  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:42.483753  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.485052  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.485275  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.474107  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.493734  6307 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:26:42.494896  6456 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:42.511636  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.482040 10678 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:26:42.515187  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.516698  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.519624  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.523867  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:42.524073  6310 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:42.525388  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.503554 10871 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:42.511956 10912 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.472261 10718 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:42.528569  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.499835 10866 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.533229  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2285) took 1064 ms. Trace:
I20260709 12:26:42.533720  6453 rpcz_store.cc:276] 0709 12:26:41.469008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.473073 (+  4065us) service_pool.cc:224] Handling call
0709 12:26:42.532975 (+1059902us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54000eb854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:42.533143 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1911487744}
I20260709 12:26:42.535015  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:42.536491  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:42.537640  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:42.539793  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2008) took 1164 ms. Trace:
I20260709 12:26:42.543067  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:42.534736 10912 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.548290  6453 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:42.503887 10792 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:42.549928  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52229.7us
I20260709 12:26:42.543519  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.517158 10595 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:42.556370 10893 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:42.556962  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:42.557934  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:42.557032 10912 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.529959 10871 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:42.570802  6307 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:42.572566 10922 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:26:42.573748  6453 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:42.573761  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2278) took 1221 ms. Trace:
I20260709 12:26:42.577482  6459 rpcz_store.cc:276] 0709 12:26:41.352638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.359856 (+  7218us) service_pool.cc:224] Handling call
0709 12:26:41.360162 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.364273 (+  4111us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:42.234260 (+869987us) spinlock_profiling.cc:243] Waited 907 us on lock 0x7b18003780a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa6c9f3c7 00007f6fa32b19a7 00007f6fa32c10bf 00007f6fa32c1038
0709 12:26:42.420274 (+186014us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.531239 (+110965us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.531989 (+   750us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.532105 (+   116us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:42.532135 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.532245 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.573081 (+ 40836us) raft_consensus.cc:1685] finished
0709 12:26:42.573414 (+   333us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.573565 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33572,"prepare.queue_time_us":988171,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"raft.queue_time_us":21119,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":10234,"spinlock_wait_cycles":2085632,"thread_start_us":207950,"threads_started":2,"wal-append.queue_time_us":93993}
I20260709 12:26:42.595480  6457 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:42.545460  6308 rpcz_store.cc:276] 0709 12:26:41.375356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.556877 (+181521us) service_pool.cc:224] Handling call
0709 12:26:41.572759 (+ 15882us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.849490 (+276731us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.849539 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.850208 (+   669us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.211431 (+361223us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:42.257189 (+ 45758us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.264107 (+  6918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.276222 (+ 12115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.477300 (+201078us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b180048ee00. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:42.539273 (+ 61973us) raft_consensus.cc:1685] finished
0709 12:26:42.539397 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.539557 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11589,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":8628,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":15291,"spinlock_wait_cycles":32657152,"thread_start_us":399610,"threads_started":2,"wal-append.queue_time_us":303532}
I20260709 12:26:42.611799  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.600816  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.619005  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.580899 10871 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:42.612596  6308 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:42.613936 10893 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:42.639242  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.643081  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.643687  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:42.643816  6310 raft_consensus.cc:2376] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15. Re-sending same reply.
I20260709 12:26:42.639911  6303 raft_consensus.cc:2376] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9. Re-sending same reply.
I20260709 12:26:42.648274  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.642377  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.651777  6308 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:42.659231 10870 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:42.660501  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.661434  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.663766  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:42.664641  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.666916  6303 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:42.668063  6300 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:42.669893  6300 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:42.671110  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.674404  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.679340 10792 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:42.685559 10953 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:42.686065  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:42.689635  6300 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:26:42.690871  6303 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:42.694334  6309 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:42.698437  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:42.702719 10957 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:26:42.702596 10876 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:42.686182 10792 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:42.690918 10866 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:42.704277 10792 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:42.705083  6308 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:26:42.714123  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69082.6us
I20260709 12:26:42.716300  6310 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:42.717527 10866 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:42.718257 10866 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.720518 10866 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.721118 10588 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.722997  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2775) took 1147 ms. Trace:
I20260709 12:26:42.723399  6452 rpcz_store.cc:276] 0709 12:26:41.575045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.575201 (+   156us) service_pool.cc:224] Handling call
0709 12:26:41.643143 (+ 67942us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.643978 (+   835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.722597 (+1078619us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b54001e14d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:42.722818 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155280768}
I20260709 12:26:42.728760 10925 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:42.729202 10925 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:42.731731  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 313522us
I20260709 12:26:42.733026  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:42.735101  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:42.736311 10927 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:42.737299 10927 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.737790 10927 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:42.738726  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 318818us
I20260709 12:26:42.739161 10611 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:42.739699  6383 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:26:42.740291 10611 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:42.741487 10927 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:42.741719 10611 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:42.742070 10927 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:42.756500 10927 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:42.757648 10927 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.759224 10927 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.761812  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:42.762818  6382 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:42.764123  6304 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:26:42.764823  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 607) took 1093 ms. Trace:
I20260709 12:26:42.747630  6302 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:42.752146  6309 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:26:42.755244  6454 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:42.707773 10893 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.768046  6303 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:42.769240  6308 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:42.769374  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 149458us
I20260709 12:26:42.765122  6304 rpcz_store.cc:276] 0709 12:26:41.671156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.763264 (+1092108us) service_pool.cc:224] Handling call
0709 12:26:42.763684 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.764555 (+   871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.764765 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:42.774452 10967 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:26:42.779904  6456 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:42.784996  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.765414 10679 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:26:42.783535 10611 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:42.730217 10925 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:42.851244  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63554.1us
W20260709 12:26:42.857196 10963 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:26:42.844288 10778 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.799578 10588 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.870278  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.865257 10679 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:42.882978  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59332.3us
W20260709 12:26:42.887457  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 116443us
I20260709 12:26:42.888859  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.886583  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.876664 10696 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.879362 10905 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:42.895571  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:42.900002 10595 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:42.900270 10896 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:26:42.900774 10595 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:42.901904 10595 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:42.902963  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 115074us
W20260709 12:26:42.903851  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.906925 10696 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:42.907814  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2772) took 1457 ms. Trace:
W20260709 12:26:42.898038 10966 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:26:42.909132 10696 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:42.908269  6451 rpcz_store.cc:276] 0709 12:26:41.450071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.520563 (+ 70492us) service_pool.cc:224] Handling call
0709 12:26:41.528354 (+  7791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.528743 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:42.444767 (+916024us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.525008 (+ 80241us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.612113 (+ 87105us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.612307 (+   194us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:42.612565 (+   258us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.612797 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.612921 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.907495 (+294574us) raft_consensus.cc:1685] finished
0709 12:26:42.907516 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.907633 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":742258,"prepare.queue_time_us":839462,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":19289,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":23581,"thread_start_us":205325,"threads_started":2,"wal-append.queue_time_us":219485}
I20260709 12:26:42.896054 10913 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:42.898335 10588 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:42.914603  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.900168 10909 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:42.915796  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:42.916159 10909 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.918645 10611 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:42.943861  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55708.5us
W20260709 12:26:42.943835  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 208883us
W20260709 12:26:42.943846  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57381.4us
I20260709 12:26:42.946458  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:42.951084  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.880937 10778 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:42.952581 10879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.015s	sys 0.006s
W20260709 12:26:42.952943 10879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.015s	sys 0.006s
I20260709 12:26:42.953150  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:42.954242  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.956542  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:42.957438  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.953020 10828 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:42.966147  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95951.3us
I20260709 12:26:42.918391 10909 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:42.890362 10679 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:42.976740  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:42.965523  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:42.967334  6454 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:42.945618 10913 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:42.957785 10588 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:42.984717  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:43.003116  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50046.8us
I20260709 12:26:43.004088  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:43.005286  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:43.011639  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.009621 10696 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:43.014518 10696 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:42.980903 10912 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:43.016115  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:42.986068 10909 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.963255 10905 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.007025  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:43.031575 10866 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:26:43.032713  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57299.4us
W20260709 12:26:43.025749 10619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.033s	sys 0.042s
I20260709 12:26:43.034394 10464 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:43.035044 10909 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.038838 10696 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.040819 10678 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:42.983474 10871 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:43.088371  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81704.2us
W20260709 12:26:43.096997  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 217386us
I20260709 12:26:43.087225  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
W20260709 12:26:43.100136  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94930.3us
I20260709 12:26:43.043754 10893 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:43.094316  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88128us
I20260709 12:26:43.111845  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:43.112272  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:43.090039 10619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.034s	sys 0.042s
I20260709 12:26:43.095062 10977 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:43.123467 10977 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:43.123571 10899 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:43.108211  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:43.100028 10588 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:43.145061  6301 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:43.092283 10866 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:43.155821  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48617.6us
W20260709 12:26:43.156124  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48023.5us
I20260709 12:26:43.088074 10913 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.160267  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:43.162041  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49918.9us
I20260709 12:26:43.124631 10977 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.163295 10942 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:43.175375  6456 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:43.175614 10913 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.176103 10971 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.180547  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:43.156204 10568 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:43.185664 10971 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.160171 10982 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:43.192492  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.198381  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.203751 10871 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.183573 10927 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:43.204871 10927 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:43.195254 10718 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.660s	user 0.013s	sys 0.000s
I20260709 12:26:43.212416 10593 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:43.178800 10942 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:43.217425 10909 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.217985 10922 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:26:43.219856 10922 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:43.220836 10922 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.222757 10922 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:26:43.223624 10792 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 12:26:43.224170 10792 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:43.220206  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60198.5us
I20260709 12:26:43.158543 10678 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.197813 10619 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:43.264353 10905 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:43.264653 10679 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:43.265407 10593 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.220218 10942 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.267683 10593 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.267858 10942 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.268572  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.269259  6303 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:43.270877  6455 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:43.257308 10876 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:43.271368 10866 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.272061 10876 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.273304 10704 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:43.274595 10606 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:43.275038 10606 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:43.276283 10759 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.276818 10759 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.226151 10982 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:43.278774  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.279887  6454 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:43.281946 10870 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Committing config change with OpId 12.8: config changed from index 2 to 8, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:43.283411 10975 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:43.283818 10975 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:43.284672 10975 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.285881 10975 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:43.288756  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.290316  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.290776 10981 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.291361 10981 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.292312 10778 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.293553 10981 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:43.294111 10981 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.279394 10982 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.294552  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:43.278892 10971 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.275892 10606 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:43.301685  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 144725us
I20260709 12:26:43.303337  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.307914  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:43.309563  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.311144 10759 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:43.312116  6455 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 574) took 2165 ms (client timeout 1888 ms). Trace:
W20260709 12:26:43.312633  6455 rpcz_store.cc:269] 0709 12:26:41.151151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.153040 (+  1889us) service_pool.cc:224] Handling call
0709 12:26:43.311850 (+2158810us) spinlock_profiling.cc:243] Waited 2.08 s on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:43.311995 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4775905024}
I20260709 12:26:43.303324  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:43.315541  6456 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:43.317365 10983 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:43.318154 10983 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.320497 10983 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.321583 10619 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:43.322058 10619 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.322422 10619 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:43.324986  6308 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:43.264254 10978 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:43.274410 10876 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.367710  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:43.301504 10679 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:43.432569  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 127163us
W20260709 12:26:43.438265  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70657.7us
I20260709 12:26:43.369483  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.328361 10778 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.442018  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.442787  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.443871  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:43.448909  6301 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:43.441519 10876 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:43.450847 10876 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.453382  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.450136 10946 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:43.440434  6300 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:43.366051 10971 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:43.456683  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.343318 10792 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.461483  6310 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:43.335186 10912 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:43.465839  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.471786  6308 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:43.479533  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.358603 10913 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 12:26:43.483515  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.437923 10593 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:43.335908 10588 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:26:43.359695 10981 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.497316  6456 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:43.498129  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.464587 10893 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.504171  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50695.1us
W20260709 12:26:43.503737  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 187024us
I20260709 12:26:43.466761 10971 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.491413  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.495926  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:43.527339  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58226.6us
I20260709 12:26:43.470438 10857 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.491055 10606 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.494158 10778 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.546696 10593 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.548942 10593 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:43.551489  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.518064 10971 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:43.557940  6458 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:43.559556  6456 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:26:43.562239  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100182us
I20260709 12:26:43.565896  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:43.568724  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:43.563107  6453 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:43.532774  7319 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Committing config change with OpId 12.8: config changed from index 2 to 8, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:43.541209  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:43.558113 10871 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.467108 10978 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.533792 10912 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.599375  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.601068 10996 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.531054 10893 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.570459 10696 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.570806 10927 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:43.614658  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.613335  6302 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:43.613713  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 82033.4us
I20260709 12:26:43.615692  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:43.608572 10985 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:26:43.607059 10983 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:43.593003 10899 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:43.599479 10778 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.575358 10704 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:26:43.621918 10792 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:43.619586  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:43.612751 10879 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.564122 10857 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.613065 10678 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.607709 10871 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.638733  6376 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:43.650868  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:43.622241 10999 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:43.655922  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.665997 10993 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:26:43.667629 10992 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:26:43.669494  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.670478  6452 raft_consensus.cc:2410] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:43.672374  6307 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:43.675953  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:43.622257 10696 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.661077 10778 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:43.632421 10983 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.620023 10996 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:43.719391  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.649222 10899 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.720409  6379 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:43.682348 11003 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:43.652052 10879 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.756745  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.759886  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.760644  7316 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:26:43.769588  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 111246us
I20260709 12:26:43.775204  7331 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:43.749821 10981 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:43.784813  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:43.772729  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:43.777408  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:43.753924 10987 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
I20260709 12:26:43.721961 10778 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.787277  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.801131 11002 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:43.712666 10999 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:43.798787 10922 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.575s	user 0.009s	sys 0.003s
I20260709 12:26:43.814062  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:43.779098 10978 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.814286  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:43.814929 10997 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.247s	user 0.002s	sys 0.000s
I20260709 12:26:43.858108  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.857735  6450 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:43.846076 11002 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.874472  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:43.874884  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:43.804072 10809 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:26:43.855123 10912 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.749764 11003 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:43.894541 10989 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.199s	user 0.001s	sys 0.001s
I20260709 12:26:43.894831  6385 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:43.895368 10778 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.895663 11014 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:43.895934 11010 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.896587 10988 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:26:43.873819 10857 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:43.900149 11004 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:26:43.900400 10619 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.001s
I20260709 12:26:43.895783 11012 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:43.907162 10990 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.343s	user 0.002s	sys 0.000s
I20260709 12:26:43.907481 10696 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.911551  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:43.914386  6385 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:43.916838  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.891595 11002 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.919328  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.926549 10983 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:43.931855  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.589s (SENT) [suppressed 17 similar messages]
I20260709 12:26:43.864600 11001 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:43.953204  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:43.962592  6456 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:43.940907 10871 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.948815 10778 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:26:43.959686  7318 raft_consensus.cc:2410] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:43.940658 11014 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:43.969633 11014 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.970130  6385 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Committing config change with OpId 12.8: config changed from index 2 to 8, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:43.953293 10619 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:43.968905 10925 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.968612 11012 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:43.976220  6378 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:43.978120 11012 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.979915 10879 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:43.976241 10619 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:43.984932  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:43.987735 11003 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:43.946316 10927 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.001309 11014 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.007143  6382 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:44.013247 10991 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.416s	user 0.002s	sys 0.002s
W20260709 12:26:44.017025  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 479308us
I20260709 12:26:44.018224 10876 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.005172 10942 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:26:44.023020  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.028092  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 486923us
I20260709 12:26:44.029508  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:26:44.031087  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 462417us
I20260709 12:26:44.032470 10678 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:44.033474  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:44.034760 10988 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.039008 10870 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:44.042467  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.044130  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.045033  6458 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:44.045687  6456 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:43.967530 10996 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.075784  6307 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:44.049197  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:44.076292  7319 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:26:44.081076  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:44.081480  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63732.9us
I20260709 12:26:44.081873  6307 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:43.952057 11010 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.023689 10942 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:44.096421  6304 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.003680 10927 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:44.105316  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71947.5us
I20260709 12:26:44.107553  6456 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:26:44.097913 10971 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.530s	user 0.011s	sys 0.001s
I20260709 12:26:44.110054 10971 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.114853 10971 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.116150 11001 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:44.116958  7320 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:44.118410 10925 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:44.106925  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.126884  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:44.127722  6307 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:44.130053 10975 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.844s	user 0.011s	sys 0.003s
I20260709 12:26:44.131732 10942 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.133019 10942 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:44.135905 10828 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.136345 10828 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.136742 10828 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:44.144109  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94989.5us
I20260709 12:26:44.145028  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:44.106271 10899 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.100873 10718 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.105365  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.152925 11010 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.154837 10927 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:44.156059 11022 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:26:44.156594 11018 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:26:44.146292  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:44.099536 11021 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.161494 11021 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.163743 11021 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.171815 10759 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:44.172690  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:44.174037  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:44.173785 10759 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:26:44.186762  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 174997us
I20260709 12:26:44.189381  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.190884  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:44.192011  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 85665.2us
I20260709 12:26:44.194440  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:44.195040 11019 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.215s	user 0.002s	sys 0.000s
I20260709 12:26:44.195327 11006 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.195611  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.195830 11006 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.196223 11006 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.196280  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.196723  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.197392  6458 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:44.198268  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.193279  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:44.202414  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.203038  6309 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:44.203169  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.203859 10679 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:44.204308 11024 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.204583 10679 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:44.205016 11024 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:44.205207 10679 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:44.205718 11024 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.175258  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:44.207386 10996 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.207998 10996 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.209077 10971 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.101992 11014 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.211467 10971 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:44.212368  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.210203 10996 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.213433 10922 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.213680 10971 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.213892 10922 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.214079 10718 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.214787 10718 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.217068 10718 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.217243  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.218627 10996 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.218586 11012 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:44.219501 10996 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.221616 10876 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:44.222173 10876 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.222666 10876 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.222934  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:44.214569 10922 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.226003  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.226579  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 7.
I20260709 12:26:44.226675 10912 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.227476 10912 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.229854  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.232095 10912 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.178108  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:44.246806  6452 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.178859  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.180042  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.183585  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.205793 10679 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.174726 10981 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:26:44.256829  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67603us
I20260709 12:26:44.235415 10893 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.176218 10905 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:44.269825  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.252133  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.255358  6304 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:26:44.253448  6455 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:44.255280  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.255497  6307 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:44.258165  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.261659 10912 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.214560 10971 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:44.256368  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 72919.1us
I20260709 12:26:44.286357  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.271095  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:44.256142  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 324336us
I20260709 12:26:44.279572  6376 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:44.280912  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:44.245188 10828 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.290467  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.271994 10667 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:44.269937 10977 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:44.316895  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.313688  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:44.312693  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 119462us
I20260709 12:26:44.319473  7331 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:26:44.321815  7331 pending_rounds.cc:85] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 7
I20260709 12:26:44.321048  6453 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:44.325217  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.289000 10879 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.317154 10899 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.328835 10879 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:44.329326 11017 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
I20260709 12:26:44.329149 10899 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.281599 10619 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.330456 10619 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.332114 10977 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:44.332775 10619 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.332898  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.333129 10977 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.325371 10696 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:26:44.329380 10879 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.334637 11021 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:44.335124 11021 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.335453 11021 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:44.337339 10619 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:44.320573 10667 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.339107 10667 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:44.339318 11007 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.392s	user 0.001s	sys 0.000s
I20260709 12:26:44.312621 10996 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.331677 10899 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.289006 10925 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:44.346067 10925 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.346516 10925 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:44.346940 10999 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.347226 10996 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.347844 10996 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.349186  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2789) took 1042 ms. Trace:
I20260709 12:26:44.350104 10996 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.292717 11030 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:26:44.355181 10759 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:44.355607 10759 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:44.355994 10927 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:44.356649 10927 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:44.356407 10759 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:44.357066  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109349us
I20260709 12:26:44.357630 10927 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.357955 10759 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.316920 10913 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.342303 10994 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:44.359589 10913 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.344215 10870 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.345237 10942 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:44.361078 10870 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.361644 10876 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.362769  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.362974 10978 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:26:44.364334  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.326395 10871 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:26:44.367492 10971 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.368649  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:44.358001  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:44.312944 11025 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:44.322149  7331 pending_rounds.cc:107] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:26:44.266520 10866 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.052s
I20260709 12:26:44.375022  7331 raft_consensus.cc:2889] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.361999 10913 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.349606  6450 rpcz_store.cc:276] 0709 12:26:43.306881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.306998 (+   117us) service_pool.cc:224] Handling call
0709 12:26:44.348993 (+1041995us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400162bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:44.349116 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259056384}
I20260709 12:26:44.339120  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:44.308421 10893 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:44.388761  7320 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:26:44.385152  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.404243  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.404515  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.407169 10922 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.408269  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:44.408903  7335 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:44.288823 10912 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.387827 10896 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.421576  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.426347  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.431982  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.432907  6304 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
W20260709 12:26:44.434976  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 117346us
I20260709 12:26:44.390362 10893 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.442142  6376 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:44.416576 11031 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:44.422736 10978 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:44.383299 11002 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:44.452150  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 113153us
I20260709 12:26:44.430626 10759 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:44.454167  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.406594 10879 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:44.464870 11013 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.370s	user 0.002s	sys 0.000s
I20260709 12:26:44.469961 10975 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:44.437673 10913 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:44.473644 11024 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.441166 10778 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:44.427197 11014 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 12:26:44.489356  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 288132us
I20260709 12:26:44.440769 10893 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:44.504848  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.503289 11001 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:44.509233 10977 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:26:44.509094 10778 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.517829  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:44.518368 10978 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:44.485695  6452 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:44.521310 10913 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:44.461159 10759 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.549517  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.524096 11024 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.525900 11002 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.540091  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:44.453562 10870 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.504321 10975 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.497480 11021 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.542840 10778 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.545562 11042 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.519081 10667 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.540758 10977 consensus_queue.cc:1243] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:44.551476 10759 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.540167 11001 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.568972 10975 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.570824 10792 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:44.570390 10828 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:44.604074 11036 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
W20260709 12:26:44.661697  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 143977us
I20260709 12:26:44.670538  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:44.558771 11006 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:44.606575 11038 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:26:44.690925 11006 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.695259  7335 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:26:44.698877  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:44.708213  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 180071us
I20260709 12:26:44.591477 11042 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.683858 10876 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.711220  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.727093  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:44.727782  6383 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.740309  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.747753 10464 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.747996 11001 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.727692 11051 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:44.718961 10792 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:44.755625  6455 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:44.757383  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:44.723825 10912 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:44.761221  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62489.7us
I20260709 12:26:44.762451  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:44.729122 11006 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:44.726965 10879 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:44.753075 11035 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.289s	user 0.002s	sys 0.000s
W20260709 12:26:44.759557 11041 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.002s
I20260709 12:26:44.768296  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:44.774259  6450 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.772363  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:44.701381 10913 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.777164  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.748411 10464 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.797734 10922 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.798125 10464 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:44.799286 11043 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:26:44.799777 11042 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.800729 10975 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.803068 10975 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.777809  6381 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:26:44.780788 11045 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:26:44.813838  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:44.811429 10870 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.767603 10925 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.786823 11021 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:44.786964 11039 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:26:44.776952 10978 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.789119 10792 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.815883 10978 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.816614 10925 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.817414  7318 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:44.817912 10978 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:44.790634 11037 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.269s	user 0.000s	sys 0.002s
I20260709 12:26:44.752893 10905 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:26:44.822199  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.805078 10759 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.822916  6451 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
W20260709 12:26:44.805773 10679 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.002s
I20260709 12:26:44.826707 10679 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.829010 10679 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.826437 10759 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:44.812561 10619 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:44.788355 10667 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:44.836103  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:44.836274  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63978.7us
I20260709 12:26:44.837391  6383 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:44.845112  6451 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:44.845024  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:44.845839  6459 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:44.848508  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:44.835932 11006 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.863543  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:44.864472  7319 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:44.842900 10879 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:44.869860  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:44.871039  7331 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:44.874874  6305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:44.875698  6305 raft_consensus.cc:606] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:44.876030  6305 raft_consensus.cc:614] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:44.878746 10893 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.879012  6305 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:44.881870  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:44.884150  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:44.878913  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:44.857499 10871 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:44.861531 10913 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.895215  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:44.895509 10893 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:44.905251  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 142989us
I20260709 12:26:44.892767 10464 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.891496 10679 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:44.918102  7319 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:44.925797  6302 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:44.880743 11006 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.932828  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.898851 10879 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:44.934237  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.907063 10942 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:44.941686  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:44.905670 10896 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:44.936187  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:44.945644  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:44.937057  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.937603  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.901705 10871 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:44.939281  6385 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:44.939283  6304 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 7. (term mismatch)
I20260709 12:26:44.948020  6304 pending_rounds.cc:85] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 6
I20260709 12:26:44.945091  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:44.948351  6304 pending_rounds.cc:107] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:26:44.949510  6304 raft_consensus.cc:2889] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.934881  6302 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:44.942286  6303 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.944901  6301 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:44.946164  6310 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:44.952541  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:44.953608  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:44.953823  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:44.954299  6382 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.955941  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.944322 11042 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.906716 10978 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:26:44.966504 10871 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:44.967056 11042 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:44.971433  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.904637 10913 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:44.974040  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92290.3us
I20260709 12:26:44.980693  6453 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:44.941339 10942 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.981637  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:44.986356  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.981745  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:44.989200  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:44.989868  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 141456us
I20260709 12:26:44.989136  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.945461 11048 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:44.988212  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78684.8us
I20260709 12:26:44.993798  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:44.995672  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:44.980140  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:45.013298  6381 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:44.979111 10912 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:44.986632 10942 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:44.983033 10893 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.018429  7319 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.022948  6383 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:26:45.022862  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.025915  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:45.023870  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.032918  6384 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.029663  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.035266  6381 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.038345  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:45.014379 11042 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:44.986336 10975 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:45.026430 10667 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:44.991813 10778 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.043646  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.044958  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.047251  6383 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:45.043095  7320 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:45.071753  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.073241  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:45.014348 11048 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.048442 10667 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.075850  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.077978 10778 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:45.045403 10792 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.079638  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:45.078274 11038 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.379s	user 0.002s	sys 0.001s
I20260709 12:26:45.084249  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 8.
I20260709 12:26:45.090251  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.092585  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.035686 10893 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.099160  6379 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:26:45.100023  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.103961  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.104178 11006 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:45.037388 10912 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.080780 11008 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:45.133800  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:45.135432  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.137130  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:45.088768 10667 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:45.082525 11061 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:45.107227 11057 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:26:45.140684  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.140837  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.136744 11050 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:45.143476  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:45.186439  6455 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:45.144164 10778 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.134799 11008 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:45.036150 10879 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.151604 10913 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.152299  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.153209  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2791) took 1665 ms. Trace:
I20260709 12:26:45.153478  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:45.153961  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:45.162855  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.163462  6454 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:45.164074 11048 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.170949  6309 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.166153 10912 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:45.098588 10792 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.174139  6303 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:45.214753  6305 raft_consensus.cc:606] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:45.215421  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.205979  6457 rpcz_store.cc:276] 0709 12:26:43.487443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.487747 (+   304us) service_pool.cc:224] Handling call
0709 12:26:43.517288 (+ 29541us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.087667 (+570379us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:44.104232 (+ 16565us) log.cc:844] Serialized 14 byte log entry
0709 12:26:44.400290 (+296058us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:44.411903 (+ 11613us) log.cc:844] Serialized 26 byte log entry
0709 12:26:44.412099 (+   196us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:44.412146 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.412246 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:45.077921 (+665675us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b24001ea928. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:26:45.152889 (+ 74968us) raft_consensus.cc:1685] finished
0709 12:26:45.152997 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.153140 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":492356,"prepare.queue_time_us":15678,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":509971,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":7774,"spinlock_wait_cycles":15801728,"thread_start_us":201211,"threads_started":2,"wal-append.queue_time_us":281897}
I20260709 12:26:45.216408  6303 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:45.175666  6450 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:45.178109  6310 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:45.172529  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:45.182904  6382 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:45.183907  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.183624 11042 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:45.188432  6383 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.193604 11008 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.185881 11050 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:45.197641  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:45.221614  6452 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:45.206770  7335 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:45.206980  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53117.8us
W20260709 12:26:45.210240  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 214634us
I20260709 12:26:45.204329  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.210611  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:45.222759  6301 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.211055  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.211500  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:45.219470  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47046.1us
I20260709 12:26:45.224346  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:45.224422  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.223004  7318 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.225298  6382 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.191917 11061 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:45.230598  6384 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.235522  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:45.234570  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.236364  7316 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:26:45.212970 11062 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:45.237145  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.238085  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.239532  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.237936  6381 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:45.242660  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.245157  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.246521  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.248718  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.249433  6303 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:45.252588  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:45.249502  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.255333  6384 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:45.256397  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75787.2us
I20260709 12:26:45.210883 10778 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.239097 11008 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:45.264163  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.265528  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.259950  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.235013 10912 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.237048 11048 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:45.268227  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.274858  7335 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:45.275189  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.276834 10876 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.273030  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.282116  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.282899  6304 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:45.248067 10913 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:45.290411  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.253219 11021 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.301738  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:45.257304 11062 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:45.284593 10893 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.280488 11023 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:45.331208  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.284153 11006 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:26:45.338754  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 86206us
I20260709 12:26:45.346443  6304 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:26:45.347965  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:45.349978  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75754.8us
I20260709 12:26:45.349346  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.352249  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.278960 10696 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:26:45.350114  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.316699 10828 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:45.358331  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.361723 11061 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.283809 10912 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.369243 10912 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.358328 10879 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:26:45.373232 11062 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.318888 10871 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:26:45.383173 11071 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:45.308336 10876 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:26:45.385092 10792 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:45.396551  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 138483us
I20260709 12:26:45.397629  6309 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:45.399448  6304 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:45.404249  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:45.404662  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.405539  6304 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:45.408365  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 98747.9us
I20260709 12:26:45.339488 11006 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.310140 10913 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.410478  6452 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:45.415097  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.414005  6303 pending_rounds.cc:85] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 5
I20260709 12:26:45.411352  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.409818  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.415940  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.420334  6303 pending_rounds.cc:107] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:26:45.333407 11021 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.428201 11021 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.428871 10942 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.430199  7331 raft_consensus.cc:2368] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:26:45.432646 10667 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.433881  6459 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:26:45.409379 10893 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:45.435146 10893 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.435557 10893 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:45.388691 11042 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:45.420573  6304 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:45.420801  6303 raft_consensus.cc:2889] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:45.403123 10828 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.424553 10913 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:45.441740  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.441900  6301 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:45.449653  6301 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.442389  7318 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:45.409716 11053 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.419512 11006 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:45.438859 10619 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.594s	user 0.013s	sys 0.000s
I20260709 12:26:45.414433 11048 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:45.466138  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.466782  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:45.478874  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:45.479780  7320 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.475328 10696 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.482792  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:45.482949  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:45.483654  6380 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:45.484668  7316 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.488123  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:45.457789 11042 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:45.462008 10994 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:45.503885  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:45.506872  7318 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:45.471256 10619 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:45.475015 10778 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.495273 11040 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.489462  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:45.475706 11048 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.507462  6454 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:45.471174 10912 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:45.475466 10978 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:26:45.513967  7316 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:45.495812 10828 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:45.530401  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 120087us
W20260709 12:26:45.530584  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 109197us
I20260709 12:26:45.465095 11053 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.538197  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:45.514887 11052 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:45.538659  6384 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:26:45.555214  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 65848.8us
I20260709 12:26:45.573834  7320 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:26:45.576027  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:45.577329  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57497.4us
I20260709 12:26:45.525211 10994 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:45.582049  7316 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:45.512084 10982 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:45.591581  6272 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:45.548940 11008 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:45.594731  6272 leader_election.cc:403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:26:45.593094  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:45.595731 10696 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:45.597404 10942 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.598858  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.526732 11062 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 12:26:45.525578 11023 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:45.609584  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:45.579741 11079 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:26:45.598301 10667 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:45.617938  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:45.617993  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 213867us
I20260709 12:26:45.618520  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:45.621106  6378 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:26:45.619298  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:45.621886  6457 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.621398  6376 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:45.570585 10981 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:45.580859 11052 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:45.604635 11008 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:45.636310  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.613996 11040 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:45.641779  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.627264 10982 consensus_queue.cc:1243] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:45.664526  6457 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:45.666467  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:45.667760 11001 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:45.668434 11001 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:45.669639 11001 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.671828  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:45.672266  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:45.671331  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49890.6us
I20260709 12:26:45.639166 10977 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:45.643047 11040 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.674808  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.676932  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.678164  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.681020  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.682359  6308 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:45.685245  6376 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:45.643975 11085 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:45.614326 11042 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.687767  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.684969 11040 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:45.699719  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58002.5us
I20260709 12:26:45.700414  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:45.663980 10778 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10.
I20260709 12:26:45.712404  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.704326  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.677663 10977 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.677935 10975 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.651787 10879 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:26:45.734632  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.734664  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.701655 11024 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:45.736251  6308 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:26:45.699457 10619 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:45.673141 10912 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:26:45.648092 11023 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.676703 11079 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:26:45.750798  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.759298  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:45.761255  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:45.740478 11097 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:45.738025 11006 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.763495  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:45.690290 11008 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.723850 11086 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:45.733786 10977 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:45.774580  6300 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:45.724562 11088 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:45.774467 10619 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:45.740935 11048 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:45.812081  6457 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
W20260709 12:26:45.807760  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70605.1us
I20260709 12:26:45.813583  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.776353 10971 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:45.818269 10942 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:45.819172 10942 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:45.819609 10942 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.820096 10667 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:45.820796 10667 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:45.821560 10942 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.779273 10792 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:45.830912 10792 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:45.779348 11098 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:45.865767  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52287us
I20260709 12:26:45.803526 11053 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:45.727785 11085 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.869192  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:45.890290  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:45.832942 10971 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.810608 10778 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:26:45.935357  6300 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:45.895578 11092 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:26:45.738224 11052 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.779881 11097 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.933665 10971 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.821928 10667 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:45.996574  7335 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:45.998526 11097 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:46.018194 11085 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:46.019718 10619 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:46.023422 11089 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
I20260709 12:26:45.994663 10912 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:46.024605 11099 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:26:46.025480 11107 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:45.946499 11080 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:46.002061 11079 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:46.084481  6300 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:26:46.069625 11092 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:26:46.168874  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 303762us
I20260709 12:26:46.190330 10679 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:46.191316 10679 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:46.191979  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:46.192581  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:46.213275  6453 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:46.214030 10982 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:46.217725 11095 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.443s	user 0.006s	sys 0.004s
I20260709 12:26:46.081182 10667 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:46.215421  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:46.192704 10679 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:46.225390 10982 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:46.232491 10982 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:46.075853 10912 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:46.244690 10912 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:46.233542  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2050) took 1069 ms. Trace:
I20260709 12:26:46.245589  6304 rpcz_store.cc:276] 0709 12:26:45.164547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.487359 (+322812us) service_pool.cc:224] Handling call
0709 12:26:45.517476 (+ 30117us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:45.527176 (+  9700us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:45.527225 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:45.527840 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.616465 (+ 88625us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.765848 (+149383us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.812448 (+ 46600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.812532 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.217263 (+404731us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b240017b0e8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:26:46.233092 (+ 15829us) raft_consensus.cc:1685] finished
0709 12:26:46.233252 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.233448 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207346,"prepare.queue_time_us":152625,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":99237,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":32174,"spinlock_wait_cycles":31231360,"thread_start_us":36711,"threads_started":1,"wal-append.queue_time_us":149836}
I20260709 12:26:46.256124 11006 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:46.256323 10982 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:46.258410 10982 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:46.231962 10679 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:46.264274  7331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 619) took 1216 ms (client timeout 1506 ms). Trace:
W20260709 12:26:46.264814  7331 rpcz_store.cc:269] 0709 12:26:45.047821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.047977 (+   156us) service_pool.cc:224] Handling call
0709 12:26:46.264064 (+1216087us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:46.264197 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":818713856}
W20260709 12:26:46.272223  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56897.7us
I20260709 12:26:46.273907  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:46.275979  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2051) took 1111 ms. Trace:
I20260709 12:26:46.086288 10792 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:46.277412  6300 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:46.219439 11050 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:46.282812 11050 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:46.291980  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2815) took 1160 ms. Trace:
W20260709 12:26:46.271699 10893 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.836s	user 0.007s	sys 0.006s
I20260709 12:26:46.276533  6309 rpcz_store.cc:276] 0709 12:26:45.164883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.600025 (+435142us) service_pool.cc:224] Handling call
0709 12:26:45.600389 (+   364us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:26:45.608990 (+  8601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:45.609022 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:26:45.613937 (+  4915us) log.cc:844] Serialized 78 byte log entry
0709 12:26:45.676252 (+ 62315us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.770676 (+ 94424us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.832909 (+ 62233us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.275077 (+442168us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.275453 (+   376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.275573 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.275637 (+    64us) raft_consensus.cc:1685] finished
0709 12:26:46.275657 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.275885 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":155414,"prepare.queue_time_us":143982,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":179459,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":19846,"spinlock_wait_cycles":60537600,"thread_start_us":470999,"threads_started":2,"wal-append.queue_time_us":157806}
I20260709 12:26:46.292567  6459 rpcz_store.cc:276] 0709 12:26:45.131416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.131587 (+   171us) service_pool.cc:224] Handling call
0709 12:26:45.132042 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:45.434274 (+302232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.291571 (+857297us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400266e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:46.291869 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":693565184}
I20260709 12:26:46.342420  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2249) took 1417 ms. Trace:
I20260709 12:26:46.280176 11023 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:46.375598  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 506450us
I20260709 12:26:46.289073 10879 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:46.381068 11091 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.013s	sys 0.037s
I20260709 12:26:46.343279  6310 rpcz_store.cc:276] 0709 12:26:44.925244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.238741 (+313497us) service_pool.cc:224] Handling call
0709 12:26:45.239091 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:45.413318 (+174227us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:26:45.733258 (+319940us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.208860 (+475602us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.209466 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.209611 (+   145us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:46.209899 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.210408 (+   509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.211099 (+   691us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54001b2bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:46.211159 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.341986 (+130827us) raft_consensus.cc:1685] finished
0709 12:26:46.342035 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.342308 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":359734,"prepare.queue_time_us":349270,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":132946,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":23396,"spinlock_wait_cycles":300972800,"thread_start_us":392699,"threads_started":1,"wal-append.queue_time_us":500741}
I20260709 12:26:46.381778  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:46.312110 10982 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:46.380553 11099 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:46.323062 10893 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:46.408257 11091 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.013s	sys 0.037s
I20260709 12:26:46.435650  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:46.392544 11023 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:46.420353 11082 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:46.442589  6308 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
W20260709 12:26:46.443506  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 169667us
I20260709 12:26:46.408201 10879 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:46.451686  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.428277 10982 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:46.456223  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.453959  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:46.459857  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:46.463781  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:46.464854  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:46.461045  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:46.861598  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 401822us
I20260709 12:26:46.465605  6459 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:46.485541  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.014s	sys 0.019s
I20260709 12:26:46.466327  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:46.864332  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.014s	sys 0.038s
I20260709 12:26:46.557861  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:46.459713 10982 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:46.865793 10982 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:46.502990 11001 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:46.870041 11001 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:46.870872  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 410834us
I20260709 12:26:46.648087  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2250) took 1722 ms. Trace:
I20260709 12:26:46.863353  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:46.873281  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 227393us
I20260709 12:26:46.888767  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:46.656915 11052 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.005s	sys 0.019s
W20260709 12:26:46.823415 11124 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:46.544911 11042 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:46.511579 11118 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:26:46.823556 11040 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.092s	user 0.011s	sys 0.002s
I20260709 12:26:46.648571 11127 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.893515  6452 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:46.872004  6302 rpcz_store.cc:276] 0709 12:26:44.926859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.382395 (+455536us) service_pool.cc:224] Handling call
0709 12:26:45.383011 (+   616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:45.418845 (+ 35834us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:45.745010 (+326165us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.023653 (+278643us) spinlock_profiling.cc:243] Waited 9.08 ms on lock 0x7b4800002958. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:26:46.328925 (+305272us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.329362 (+   437us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.374907 (+ 45545us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.375105 (+   198us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:46.375331 (+   226us) log.cc:844] Serialized 14 byte log entry
0709 12:26:46.418258 (+ 42927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.426810 (+  8552us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:46.647727 (+220917us) raft_consensus.cc:1685] finished
0709 12:26:46.647759 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:46.647985 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":472440,"prepare.queue_time_us":370513,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":81,"raft.queue_time_us":254489,"raft.run_cpu_time_us":12576,"raft.run_wall_time_us":625706,"spinlock_wait_cycles":43290752,"thread_start_us":531323,"threads_started":2,"wal-append.queue_time_us":63828}
W20260709 12:26:46.824415 10977 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.014s	user 0.009s	sys 0.002s
I20260709 12:26:46.862838 11023 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:46.678771 11122 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:46.874188 10896 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:46.893018 11052 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.007s	sys 0.023s
I20260709 12:26:46.894635 10879 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:46.919087 10893 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:46.921307 11122 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:46.913146 11127 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:46.907936 11042 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:46.912945 10977 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:46.947212 11121 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:26:46.913012 11040 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:46.944687 11125 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:26:46.959097  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 95865.6us
I20260709 12:26:46.998147 11125 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:46.998705  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.959352  6453 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:26:47.000978 11125 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:46.950519 11001 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:46.957032 11131 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:47.003046 10994 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:47.003489 11127 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:47.003587 10994 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:47.005303 10994 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.006350 10994 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.006943 10977 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.008747 11042 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.011338 11040 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:47.011451 10667 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.749s	user 0.014s	sys 0.000s
W20260709 12:26:47.016233 11128 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
W20260709 12:26:47.017285 11130 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:47.018528  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2266) took 1452 ms. Trace:
I20260709 12:26:47.018888  6300 rpcz_store.cc:276] 0709 12:26:45.566273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.294388 (+728115us) service_pool.cc:224] Handling call
0709 12:26:46.294869 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.377689 (+ 82820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.377747 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.385760 (+  8013us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.580712 (+194952us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:46.580783 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:46.580901 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.018301 (+437400us) raft_consensus.cc:1685] finished
0709 12:26:47.018321 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.018450 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18049,"raft.queue_time_us":18448,"spinlock_wait_cycles":12276992,"thread_start_us":139850,"threads_started":1,"wal-append.queue_time_us":501874}
I20260709 12:26:47.021598  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2221) took 1423 ms. Trace:
I20260709 12:26:47.021950  6385 rpcz_store.cc:276] 0709 12:26:45.597767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.601100 (+  3333us) service_pool.cc:224] Handling call
0709 12:26:45.745120 (+144020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.745356 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:45.745395 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:45.745459 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:45.745534 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.745614 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.021424 (+1275810us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540022f450. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:47.021527 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":330472832}
W20260709 12:26:47.023619 11050 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.740s	user 0.010s	sys 0.001s
I20260709 12:26:47.024119 11138 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.024205  7318 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:46.948616 11122 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
W20260709 12:26:46.989142  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2059) took 1696 ms (client timeout 2109 ms). Trace:
W20260709 12:26:46.960652  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87802.2us
I20260709 12:26:47.024834 11138 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.035856  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.035964  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.037452 11138 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:47.030192  6307 rpcz_store.cc:269] 0709 12:26:45.285676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.329797 (+ 44121us) service_pool.cc:224] Handling call
0709 12:26:46.982061 (+1652264us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:46.982233 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2142367104}
W20260709 12:26:47.026556 10679 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.764s	user 0.013s	sys 0.000s
I20260709 12:26:47.026674 10896 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:47.040048 10896 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:47.043021 11133 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:26:47.040009 10971 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.972s	user 0.012s	sys 0.000s
I20260709 12:26:47.047201 10971 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.048534 11122 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:47.048938 11122 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:46.957437 11052 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.050506 10971 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:47.051236 11134 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:47.050909 11052 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.026275 10464 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.052822 10464 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.053349 11052 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.049777 11122 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:46.981253 11136 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.075824 11131 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:47.093727  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 96032.5us
I20260709 12:26:47.092913 11122 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:47.102627  7316 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:47.103907 11093 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.104174 11040 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.104496 11093 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.104628 11040 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.102519 10988 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.107281 10988 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.109117  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.106793 11139 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.102550 10679 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.092051 10982 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.100327 11051 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.121565 11051 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.100119 11042 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.113183 11144 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.126780 11144 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.128822 11144 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:47.109754 11040 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.129834 11040 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:47.113492 10879 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.135970 10879 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.137975 10879 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.138729  6450 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:47.138924  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 757204us
I20260709 12:26:47.139920  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.140753 11113 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:47.142096  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2048) took 1979 ms. Trace:
I20260709 12:26:47.109973 11093 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.119546 10679 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.119736 10667 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.148644 10667 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.149196 10679 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.151029 10667 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.151381  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.153604  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 644) took 1488 ms. Trace:
I20260709 12:26:47.120262 11141 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.120740 10988 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.120791 11052 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.160259 11141 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.160601 11052 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.163048 11052 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.078357 11136 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.142257  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 645) took 1476 ms. Trace:
I20260709 12:26:47.101215 10977 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.167799 10977 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.169509  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 655) took 1684 ms. Trace:
I20260709 12:26:47.111322 11137 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.145051 10942 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.171178 10942 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.173496 10942 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.142539  6301 rpcz_store.cc:276] 0709 12:26:45.163813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.450678 (+286865us) service_pool.cc:224] Handling call
0709 12:26:45.451043 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:45.451303 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:46.127469 (+676166us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.138506 (+1011037us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.141828 (+  3322us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:47.141878 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.141946 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.142048 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1032427,"prepare.queue_time_us":311591,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":25,"raft.queue_time_us":740160,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":103,"thread_start_us":644770,"threads_started":2,"wal-append.queue_time_us":345712}
I20260709 12:26:47.176319  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:47.177613  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.123790 11051 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.178694  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.119022 11139 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.179351  6307 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:47.179684  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:47.180243  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:47.146832 11113 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:47.180814  6307 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.181173 11086 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.181680 11086 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.182443 11113 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.183068 11113 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.183830 11086 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.185184 11113 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.186854 10464 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.157903 11144 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.185228  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2264) took 1619 ms. Trace:
I20260709 12:26:47.188221 11144 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.188370  6304 rpcz_store.cc:276] 0709 12:26:45.565607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.247752 (+682145us) service_pool.cc:224] Handling call
0709 12:26:46.248121 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.248455 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.248500 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.248926 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 12:26:46.457751 (+208825us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:46.479760 (+ 22009us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.168877 (+689117us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800420ca0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:47.184827 (+ 15950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.184918 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.184965 (+    47us) raft_consensus.cc:1685] finished
0709 12:26:47.185041 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.185169 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":712524,"prepare.queue_time_us":47887,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":706530,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":190,"spinlock_wait_cycles":23187840,"thread_start_us":208344,"threads_started":1,"wal-append.queue_time_us":260089}
I20260709 12:26:47.212878  7320 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:47.212779  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.160322  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.213461  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:47.169950  7335 rpcz_store.cc:276] 0709 12:26:45.485238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.491406 (+  6168us) service_pool.cc:224] Handling call
0709 12:26:45.996343 (+504937us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.004124 (+  7781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.169253 (+1165129us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b54003bc4d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:47.169409 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110891904}
I20260709 12:26:47.171094 11137 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.179070  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 646) took 1513 ms. Trace:
I20260709 12:26:47.158144  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2262) took 1750 ms. Trace:
I20260709 12:26:47.215886  6307 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:47.144016  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.138931 10982 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:47.154153  6309 rpcz_store.cc:276] 0709 12:26:45.665160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.299630 (+634470us) service_pool.cc:224] Handling call
0709 12:26:46.299910 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.460454 (+160544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.460496 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:46.460558 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:46.460595 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.153288 (+692693us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:47.153348 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.153514 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":368747776}
I20260709 12:26:47.180267  6300 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.239729 11086 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.240178 11086 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.240568 11086 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.242026 10978 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.242631 10978 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.244658 10978 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:47.245563  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.206653 10679 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.207046 11136 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.246960 10679 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:47.248873 11147 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 12:26:47.249238 10679 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.207509 10896 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.249930 10896 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:47.251847  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 107883us
I20260709 12:26:47.213574  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.252344  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.252717  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.190935 11131 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.218989  7319 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:47.253925  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.254745 10893 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:47.255398 10912 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.144436 11042 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.220494 11144 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.221477 11051 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.227483 11008 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.228097  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.231900  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:47.232824  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2268) took 1501 ms (client timeout 1632 ms). Trace:
I20260709 12:26:47.232981 11137 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.233619  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.283623  6309 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:47.284173  6459 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:47.234323  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.234691  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.286737  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.286872  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.235316  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.287498  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.165259  6310 rpcz_store.cc:276] 0709 12:26:45.665466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.409212 (+743746us) service_pool.cc:224] Handling call
0709 12:26:46.409528 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.409746 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:46.422032 (+ 12286us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.141867 (+719835us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.141930 (+    63us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:47.141993 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.142069 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.142194 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":684845,"prepare.queue_time_us":19084,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":679290,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":41005,"threads_started":1,"wal-append.queue_time_us":485161}
I20260709 12:26:47.288710  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.289372  6310 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
I20260709 12:26:47.245725 11025 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:47.215853  6305 rpcz_store.cc:276] 0709 12:26:45.407763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.208880 (+801117us) service_pool.cc:224] Handling call
0709 12:26:46.209389 (+   509us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:46.210281 (+   892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.210325 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:46.211110 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 12:26:46.445270 (+234160us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:46.780690 (+335420us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.157683 (+376993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.157803 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.157876 (+    73us) raft_consensus.cc:1685] finished
0709 12:26:47.157892 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.158068 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":377225,"prepare.queue_time_us":19033,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":377653,"raft.run_cpu_time_us":1078,"raft.run_wall_time_us":44188,"thread_start_us":552671,"threads_started":2,"wal-append.queue_time_us":446452}
I20260709 12:26:47.251924 10896 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:47.292258  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.292518  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.206718 10977 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.287526  7317 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:47.211871 11010 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.307911 11010 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.253100  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.256685 11115 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.214366  6302 rpcz_store.cc:276] 0709 12:26:45.665784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.911679 (+1245895us) service_pool.cc:224] Handling call
0709 12:26:46.912048 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.912270 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:46.929364 (+ 17094us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.025016 (+ 95652us) spinlock_profiling.cc:243] Waited 6.57 ms on lock 0x7b1800423b20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:26:47.178707 (+153691us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.178771 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:47.178823 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.178900 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.179016 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99779,"prepare.queue_time_us":14665,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":97222,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":116,"spinlock_wait_cycles":15118720,"thread_start_us":61965,"threads_started":1,"wal-append.queue_time_us":169335}
I20260709 12:26:47.313575  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.314466  6302 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:26:47.257551  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:47.315873  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58399us
I20260709 12:26:47.316386 10893 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:47.261485  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.263104  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.264398  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.278301  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.280879  6454 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:26:47.280896  6308 rpcz_store.cc:269] 0709 12:26:45.730992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.742079 (+ 11087us) service_pool.cc:224] Handling call
0709 12:26:47.232657 (+1490578us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:47.232761 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534740864}
I20260709 12:26:47.318370  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.333307 10912 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:47.333561  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.333820 10994 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.276539 10978 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.334407 10994 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.334849 10978 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.335315  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.334781 10893 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:47.284744  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:47.290840  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.337025 10978 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.297137  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:47.337230 10912 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.337779  6310 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:47.279376 11052 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:26:47.338116 10994 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.288153 11051 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.338955 10994 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:47.339011  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 77622.2us
I20260709 12:26:47.340790  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.298434  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.341485  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.303539 11143 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:47.297755  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.304958 11144 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.342613 11144 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.336493 10893 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:47.304477 10667 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.322636 11139 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:47.283241 11024 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:26:47.302986 10876 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.321848 11141 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.337066  6309 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:47.344148  6450 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:26:47.324074  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.325346  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.282262  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:47.348134  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.363739  6457 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:47.364645  6307 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:47.362329  6274 connection.cc:256] RPC call timeout handler was delayed by -1.5165186229996834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17120000000000002s for a response
W20260709 12:26:47.347990  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85460.7us
I20260709 12:26:47.348582  6300 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:47.357201  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.354084  6310 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:26:47.373761  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.340790 11051 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.387763  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.323448 11042 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.395664  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.352456 11093 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:47.406250  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.410996  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.399963 11109 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:47.409461  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.414405  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:47.417528  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 308485us
I20260709 12:26:47.417757  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:47.418395  6351 leader_election.cc:400] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:26:47.418843  6351 leader_election.cc:403] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:47.387207 10994 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:47.321578 11086 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.434543  6309 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:47.435392  6309 raft_consensus.cc:606] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.404736 11097 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.356040 10667 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.373461 10876 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.447801  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.415846 11143 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:47.450902  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.451207 11078 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:47.451840  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.451959  6450 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:26:47.452075  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2252) took 2511 ms. Trace:
I20260709 12:26:47.453219  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:47.471182  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57579.3us
I20260709 12:26:47.472121  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:47.453787 11116 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:47.433144 10994 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:47.408949 11141 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:47.445982  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.483595  6305 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:47.460017  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.449872  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.432116 10896 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.432461 10998 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:47.464154  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45907.2us
I20260709 12:26:47.492859  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.422189 11127 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:47.470140 11093 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:26:47.477998  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.497895  6304 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:47.477839 11078 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.471179  6303 rpcz_store.cc:276] 0709 12:26:44.940779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.391363 (+450584us) service_pool.cc:224] Handling call
0709 12:26:45.413816 (+ 22453us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:45.469511 (+ 55695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:45.469558 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:45.765231 (+295673us) spinlock_profiling.cc:243] Waited 9.28 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:26:47.166201 (+1400970us) spinlock_profiling.cc:243] Waited 7.55 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:26:47.166796 (+   595us) log.cc:844] Serialized 52 byte log entry
0709 12:26:47.447247 (+280451us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:47.448052 (+   805us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.450321 (+  2269us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.451070 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.451822 (+   752us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:47.451868 (+    46us) raft_consensus.cc:1685] finished
0709 12:26:47.451881 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.452013 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10770,"prepare.queue_time_us":575,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":43728,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":13179,"spinlock_wait_cycles":38710016,"thread_start_us":279949,"threads_started":1,"wal-append.queue_time_us":273438}
I20260709 12:26:47.472607 11144 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.456470 11061 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.462692  6309 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:26:47.481793 10942 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:47.478777 11149 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:26:47.413862 11104 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.059s
I20260709 12:26:47.472157  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.412774 11109 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.507984  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.508281 10977 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:47.508713 10977 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.509066 10977 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:47.509641  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.511214  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.511970  6455 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:47.514354  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.514767  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.515144  6454 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:47.515357  6456 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:26:47.515830  6456 pending_rounds.cc:85] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:26:47.516219  6456 pending_rounds.cc:107] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 12.12
I20260709 12:26:47.516538  6456 raft_consensus.cc:2889] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.516605  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.517679  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.517802  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.518244  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.518630  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.519217  6459 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:47.519258  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.515926  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.415755 10464 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:26:47.506356 11061 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:47.521075 11061 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.494216  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:47.455587  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2346) took 1016 ms. Trace:
I20260709 12:26:47.522789  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:47.524351 11050 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:47.524608  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:47.482947 10925 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.491155 11141 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.520123  6455 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:47.525354 11141 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.525570 10925 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:47.483070 11143 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:47.526536  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.527341  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.527722 11097 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:47.527442 11143 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.528131 11097 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.528534 11097 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:47.522725  6451 rpcz_store.cc:276] 0709 12:26:46.438766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.438875 (+   109us) service_pool.cc:224] Handling call
0709 12:26:46.439130 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.439314 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:46.606094 (+166780us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.179135 (+573041us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b1800232ea0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a
0709 12:26:47.455230 (+276095us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.455292 (+    62us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:47.455340 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.455411 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:47.455525 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27850,"prepare.queue_time_us":509467,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":28663,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":48198400,"thread_start_us":766923,"threads_started":3,"wal-append.queue_time_us":416433}
I20260709 12:26:47.482182 10876 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.524770 11050 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.522643 11123 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:26:47.532454 10942 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:47.532984  6455 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:47.532485 11050 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.533488 10942 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.496999 10988 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:26:47.516240  6453 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:26:47.538686  6453 pending_rounds.cc:85] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 10
I20260709 12:26:47.538909  6453 pending_rounds.cc:107] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:26:47.539121  6453 raft_consensus.cc:2889] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:47.539917 11127 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:47.541333 11127 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.540043  6454 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:47.501786  6450 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:47.515413  6452 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:47.520097 10913 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.490893 10896 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.551306  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.460137 11148 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:26:47.518285  6456 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:47.567391  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.564442  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:47.572021  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.569386 10893 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:47.520191 11104 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:47.501627 11152 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:47.506775 10667 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.567152 10896 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.585762  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.571467 10998 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:47.550530 11050 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.577307 10876 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:26:47.578003 10912 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:47.544495 11094 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:47.579921 11104 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:47.576841 11112 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:26:47.589687 10998 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.608018 11050 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.609156 11148 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:47.583873 11068 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:47.614753 10876 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:47.615594 11086 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.588006 11042 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:26:47.569643 10925 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:26:47.575439 11001 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:26:47.542797 11097 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.597211 10912 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.574277 10994 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.630824  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.626650 11148 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:47.640923  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.637876 10912 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:47.641826  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.642771  6450 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:47.641866  6456 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:47.649250  6205 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 264) took 23956 ms (client timeout 15000 ms). Trace:
I20260709 12:26:47.653378 10912 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.654918 10876 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.657903 11097 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.658620  6450 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:47.661682  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.662707  6450 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:47.627578 11068 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.622622 10998 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:47.674461  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43731.1us
I20260709 12:26:47.670534  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:47.675146  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 167069us
I20260709 12:26:47.676020  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.678196  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:47.690027  6304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:47.690683  6304 raft_consensus.cc:606] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:47.686654  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.699069  6450 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:47.701290  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.697193  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.704829  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.709097  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:47.674541 11068 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:47.716606  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:47.717398  6385 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.682094 10925 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:47.731439  6451 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:47.711119 11093 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:47.707901 10696 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.743134  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.722729 11131 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.721477 10978 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:47.738881 10925 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:47.749147 10912 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.749662  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:47.742753 10876 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:26:47.750962  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.755000  6378 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:47.755270  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:47.734735 11097 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:47.756163  6382 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:47.756932  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.758440 11163 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:47.758916 11156 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:26:47.755358  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:47.763818  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92404.1us
I20260709 12:26:47.755038  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.786473  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:47.772600  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:47.785319  6377 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:47.788355  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:47.789052  6385 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:47.735708 11152 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:47.741204 11093 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.782058  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.670430 11104 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.733170 11141 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.797459 11097 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:47.798712 11097 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.745005 10696 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.802258 10696 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.755575  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.765527 10978 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.795450  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.809549  6382 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:47.782332 11163 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.806850 10876 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:47.812691 11163 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.812834 10876 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.813697  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.814709 11153 raft_consensus.cc:2764] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:26:47.810616 10912 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.815551  6378 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:47.815714  6452 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:47.815928 10912 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.816385 11054 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:47.816807 11054 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:47.724085 11148 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.817206 11054 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:47.817883 11148 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.825856 10896 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.811146 10978 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.816962  6378 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:47.834780  6385 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:26:47.834923  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:47.838961  6450 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:26:47.808154  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:47.793361 10893 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.771875 11131 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.796854 11093 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:47.854218  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.827991 11144 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:47.895824  6457 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:47.897483  6304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:47.901173  6304 raft_consensus.cc:606] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:47.901795  6304 raft_consensus.cc:614] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:47.856199 10925 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.863003 11002 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:47.856375 11010 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:47.884495 10619 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:47.649659  6205 rpcz_store.cc:269] 0709 12:26:23.709247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:23.718644 (+  9397us) service_pool.cc:224] Handling call
0709 12:26:25.091763 (+1373119us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:47.649036 (+22557273us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:38.257735 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:38.258322 (+   587us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:38.258356 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:26:38.258374 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:38.260138 (+  1764us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:38.260230 (+    92us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:38.260258 (+    28us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:26:38.287945 (+ 27687us) tablet.cc:701] Row locks acquired
0709 12:26:38.292030 (+  4085us) write_op.cc:260] PREPARE: finished
0709 12:26:38.334839 (+ 42809us) write_op.cc:270] Start()
0709 12:26:38.349804 (+ 14965us) write_op.cc:276] Timestamp: P: 1783599998301968 usec, L: 0
0709 12:26:38.359363 (+  9559us) op_driver.cc:348] REPLICATION: starting
0709 12:26:38.526833 (+167470us) log.cc:844] Serialized 6951 byte log entry
0709 12:26:39.963606 (+1436773us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b
0709 12:26:40.479201 (+515595us) op_driver.cc:464] REPLICATION: finished
0709 12:26:40.548386 (+ 69185us) write_op.cc:301] APPLY: starting
0709 12:26:40.548467 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:42.459549 (+1911082us) spinlock_profiling.cc:243] Waited 9.8 ms on lock 0x7b18002f2180. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2fe60ec 00007f6fa3032c0a 00007f6fa30324f7 00007f6fa3032490 00007f6fa2fe9f48 00007f6fa6cfe1bb 00007f6fa6cfe532 00007f6fa6d45b93 00007f6fa6bf2de3
0709 12:26:42.602011 (+142462us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:42.602061 (+    50us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:43.277635 (+675574us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:43.278068 (+   433us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:26:43.278114 (+    46us) write_op.cc:312] APPLY: finished
0709 12:26:43.437620 (+159506us) log.cc:844] Serialized 164 byte log entry
0709 12:26:43.670239 (+232619us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b
0709 12:26:43.787577 (+117338us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:43.891564 (+103987us) write_op.cc:454] Released schema lock
0709 12:26:43.939285 (+ 47721us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:44.087076 (+147791us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:44.091293 (+  4217us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":858291,"reported_tablets":50,"spinlock_wait_cycles":838251776,"child_traces":[["op",{"apply.queue_time_us":68867,"apply.run_cpu_time_us":75927,"apply.run_wall_time_us":3457223,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":237200,"prepare.run_cpu_time_us":33223,"prepare.run_wall_time_us":1939090,"raft.queue_time_us":199908,"raft.run_cpu_time_us":5986,"raft.run_wall_time_us":85542,"replication_time_us":2116133,"spinlock_wait_cycles":34206592,"thread_start_us":2141609,"threads_started":5,"wal-append.queue_time_us":2101187,"wal-append.run_cpu_time_us":43313,"wal-append.run_wall_time_us":2358277}]]}
I20260709 12:26:47.863176 11123 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:26:47.856753 10893 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:47.934785  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.935456  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:47.932403 11163 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:47.947187  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:47.974063  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.911480 11040 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:26:47.910465 10971 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:26:47.985932 11175 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:47.970573 11116 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.986858 11116 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:47.987412 10896 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:47.912647 11144 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:47.992264  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 485853us
W20260709 12:26:47.997074  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69041.7us
I20260709 12:26:48.041426  7318 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:26:47.945185 10619 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.005568  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.094017  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:48.020968  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.027993  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.091152  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:48.037339  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.019021  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.011658  6304 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:26:48.008422  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.078649  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.016533  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.073001  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:48.064319  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:48.040566  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.042090 11052 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:48.036088  6300 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:47.974428 11137 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:26:48.098888  6455 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:26:47.988984 11116 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.066779  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:48.054028  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:47.989270 11122 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:47.995013 11061 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.109948  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:48.116142  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.092725 11109 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.112421  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.127905  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.128230  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.125106  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.131265  6303 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:48.133881  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.102680 11052 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:48.146097  7316 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.151437  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:48.152951  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.138743  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.155375  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:48.155658  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63744.3us
I20260709 12:26:48.156158  7320 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:48.118225 11177 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:48.158828  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:48.173261  7317 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:48.168967  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.170639  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:48.152837 11172 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
I20260709 12:26:48.133126 11143 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:48.138474 11094 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:48.150501 10879 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:48.187721  7314 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:48.188732  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:48.186784  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.203022  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.203989  6303 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:48.183058 10913 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:48.211196  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.182816 10679 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:48.190059 11143 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:48.288659 11143 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:48.292584 11061 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.245608  6307 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:48.295115  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.295697  6307 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:48.296267  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.296905  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.297340  6307 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:48.297832  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.194236 11163 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.150039 11160 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:26:48.253345 10893 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.253887 10925 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.116365 11122 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.191717 11177 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:48.309085 11177 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.309966 11122 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:48.263208 11068 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.001s
I20260709 12:26:48.310760 11068 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.312875 11068 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.265923 10994 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:48.315218 11051 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:48.315630 11051 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:48.272208 11025 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:48.317543 11025 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:48.243635 10913 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:48.318701 10913 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.318647  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.322956  6382 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:48.323503 10925 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.325567 10925 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.327836  6206 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:48.328994 11168 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.467s	user 0.000s	sys 0.002s
I20260709 12:26:48.153537 11131 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.300846  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.329960 11052 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:48.330828  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.331218 11052 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.331787  6206 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 6, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.333659  6206 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:48.335114 10619 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:48.335283 11171 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.182s	user 0.002s	sys 0.000s
I20260709 12:26:48.335723 10893 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.353710 10893 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:48.354317 10998 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 12:26:48.354830 10998 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.356056  7319 raft_consensus.cc:2368] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11.
I20260709 12:26:48.356755 10893 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.358456  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:48.358738  6451 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:26:48.359560  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.360169  6302 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:48.318338 11025 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.366505 11025 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:48.366834  6302 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:26:48.367272  6453 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:48.339735  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.368171  6302 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:26:48.368366  6301 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:48.303220 11184 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:48.346997  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.369784 11184 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:48.339802 10977 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.830s	user 0.012s	sys 0.000s
I20260709 12:26:48.370793 10977 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.352048 11078 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.370867  6206 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 6 to 7, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 7 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:48.375818  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.360126  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:48.378835  6383 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.380054  7316 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:48.380517  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.381886  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:48.382184 10978 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.005s
I20260709 12:26:48.382840 10978 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.383201  6206 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:48.348510  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.341915 10696 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:48.354909  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:48.373308 11054 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.000s
I20260709 12:26:48.373390 10977 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.384081  6381 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.371279  6302 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 12:26:48.350399 11148 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.532s	user 0.010s	sys 0.000s
I20260709 12:26:48.377981  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:48.372519 11184 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.399904  6383 raft_consensus.cc:2368] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:26:48.494354  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:48.393317  6453 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.599274  6382 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.423568  6455 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:26:48.604044  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 226144us
I20260709 12:26:48.603404  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.607717  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.608464  6302 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.601979  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:48.464393 11144 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.301270 10679 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:48.638182  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:48.610337  6206 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 7 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 10 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.380635 11138 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:48.615252  6275 leader_election.cc:400] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11.
I20260709 12:26:48.645922  6275 leader_election.cc:403] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:26:48.604106  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 109940us
I20260709 12:26:48.610960  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:48.648679  6206 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 11 to 14, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 14 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:48.473325 10893 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:48.620090  7316 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:26:48.472419 11078 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:48.466001 10998 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.610114 11109 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:48.610330 11097 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:48.663750  6376 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:48.644348 10679 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.674301 11052 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:48.672210  6206 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.679108  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 662) took 1008 ms. Trace:
I20260709 12:26:48.610536 10977 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:48.720283 10977 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.721383  6206 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 7, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 7 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.683696  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.685025  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:48.703187 11093 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.815s	user 0.005s	sys 0.005s
I20260709 12:26:48.703707  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.704624 11143 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.705099  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.706073  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.726965  6384 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:26:48.654922 10978 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.643806 11086 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:26:48.642021 11148 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.613070 11137 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:48.643671 11138 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.717192  6456 rpcz_store.cc:276] 0709 12:26:47.670481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.671074 (+   593us) service_pool.cc:224] Handling call
0709 12:26:47.671363 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.674863 (+  3500us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:47.982318 (+307455us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b180042ea40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa6c9f3c7 00007f6fa32b19a7 00007f6fa32c10bf 00007f6fa32c1038
0709 12:26:48.101708 (+119390us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.481819 (+380111us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.503424 (+ 21605us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.594550 (+ 91126us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.594785 (+   235us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:48.594839 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.594929 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.678828 (+ 83899us) raft_consensus.cc:1685] finished
0709 12:26:48.678853 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.679040 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32708,"prepare.queue_time_us":92746,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":74,"raft.queue_time_us":89780,"raft.run_cpu_time_us":350,"raft.run_wall_time_us":91502,"spinlock_wait_cycles":23980416,"thread_start_us":490269,"threads_started":3,"wal-append.queue_time_us":158463}
W20260709 12:26:48.745297  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.643447 11123 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:26:48.656280 11078 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.734978 11052 consensus_queue.cc:1243] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:26:48.736119  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.737394 11177 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.737511  6385 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:48.740801 11141 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.643842 11075 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:26:48.742767  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:48.728781  6206 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 5 to 10, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 10 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.748600 11051 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.758704  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.655184 10893 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.753850  6302 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:48.756976  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.619540 11054 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.787603  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.751806  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:48.730885 10977 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.787106  6302 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:48.789108  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.750075 11138 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.792709  6206 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 10 to 12, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 12 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:48.787760 11112 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11.
W20260709 12:26:48.804104 11188 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:48.805766  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:48.806552  6383 raft_consensus.cc:2368] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:26:48.761138 11177 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.807328  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.808501  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.809437 11177 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.810798  6380 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.811924 11054 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.750892 11148 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.818557 11148 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.819350  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.728775 11093 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.820638  6379 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.820536  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.823904  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.827735 11050 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:48.828106  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.828392 11050 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.828745 11078 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.829456  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.831804  6301 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:48.833062  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2234) took 1084 ms. Trace:
I20260709 12:26:48.833467  6376 rpcz_store.cc:276] 0709 12:26:47.749219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.749393 (+   174us) service_pool.cc:224] Handling call
0709 12:26:48.832836 (+1083443us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54003e7e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.832974 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913869696}
I20260709 12:26:48.833817 11137 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:48.835707  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.845038  6384 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:48.835728  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:48.835983  6206 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 5 to 7, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:26:48.847343  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 195927us
I20260709 12:26:48.835608  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.851671  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:48.854370  6450 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:48.854197  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.834689 10977 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.843704  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 237340us
I20260709 12:26:48.846194  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.798235 11075 consensus_queue.cc:1243] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 9.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:26:48.872179  6384 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.874277  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.875324  6381 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:48.876586  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.859500  6376 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:48.866366  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:48.821244 11141 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:48.868976  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:48.879787  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.887826  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:48.891186  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:48.854666  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 212473us
I20260709 12:26:48.835139 11051 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:48.895290  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:48.887728  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:48.869649 10913 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.005s
I20260709 12:26:48.895162  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 660) took 1335 ms. Trace:
I20260709 12:26:48.898360  6452 rpcz_store.cc:276] 0709 12:26:47.559705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.676232 (+116527us) service_pool.cc:224] Handling call
0709 12:26:47.708278 (+ 32046us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.816199 (+107921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.894921 (+1078722us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001e19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:48.895106 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":246530816}
I20260709 12:26:48.887032  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.895010  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.899667  6381 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:48.900554  6384 raft_consensus.cc:2410] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:48.902729  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:48.857486 11050 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:48.905347  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:48.887995 11185 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.908756  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.886824 11104 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:48.909147  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:48.901772 10876 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.896051 11137 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:48.911998  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:48.916337  6302 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:26:48.916267  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 627) took 1123 ms. Trace:
I20260709 12:26:48.925531  6206 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 2 to 6, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 6 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:48.921563  6384 rpcz_store.cc:276] 0709 12:26:47.792413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.915233 (+1122820us) service_pool.cc:224] Handling call
0709 12:26:48.915639 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.915873 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.915910 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:48.915964 (+    54us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:48.916010 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.916078 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.916202 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:48.899816 11144 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:48.951964 11144 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:48.951006  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 652) took 1441 ms. Trace:
I20260709 12:26:48.952548 11144 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:48.952983 11143 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.902750 10913 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.956176 10913 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.952723  6457 rpcz_store.cc:276] 0709 12:26:47.509281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.509394 (+   113us) service_pool.cc:224] Handling call
0709 12:26:48.950798 (+1441404us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54000eb354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.950949 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864940416}
I20260709 12:26:48.958467  6310 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:48.959301  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.963294  6274 leader_election.cc:400] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:26:48.964669  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.965713  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.918716 11149 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:26:48.967537 11149 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:48.967643 11181 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:48.930835 11075 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:48.927356 11137 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.971307 11137 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.972258 11001 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.976172  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 624) took 1221 ms. Trace:
I20260709 12:26:48.890815 11097 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:26:48.977130  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.978030 11097 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:48.978557  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.978965 11097 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:48.979404 11097 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:48.979997  6308 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:48.944711  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2238) took 1194 ms. Trace:
I20260709 12:26:48.980540  6379 rpcz_store.cc:276] 0709 12:26:47.750573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.750707 (+   134us) service_pool.cc:224] Handling call
0709 12:26:48.944538 (+1193831us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001f4fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.944661 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2398446464}
I20260709 12:26:48.980247 11097 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.944974  6378 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:48.981622 10696 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:48.945194  6377 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:48.947932  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:48.948140  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:48.949812 11186 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:48.983516 11186 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:48.983579  7314 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:48.981622  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:48.984457 11186 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.985621 11186 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:48.950248 11193 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.986809 11193 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.987473  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 617) took 1235 ms. Trace:
I20260709 12:26:48.950979 10971 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:48.987866  6377 rpcz_store.cc:276] 0709 12:26:47.752516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.790993 (+ 38477us) service_pool.cc:224] Handling call
0709 12:26:48.945037 (+1154044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.983107 (+ 38070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.987279 (+  4172us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001f4fd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.987406 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653448576}
I20260709 12:26:48.988464 10971 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:48.989321 10971 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:48.987391  6382 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:26:48.990397 11139 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:48.951246  6459 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:48.991287 11139 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:48.992159  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2832) took 1477 ms. Trace:
I20260709 12:26:48.992581  6459 rpcz_store.cc:276] 0709 12:26:47.514796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.519954 (+  5158us) service_pool.cc:224] Handling call
0709 12:26:48.951076 (+1431122us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.991401 (+ 40325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.991958 (+   557us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54000eb350. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.992097 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3290672896}
I20260709 12:26:48.967949 11149 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:48.937883  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:48.978299  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:48.944144  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2235) took 1194 ms. Trace:
I20260709 12:26:48.990474 10971 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:48.976747  6385 rpcz_store.cc:276] 0709 12:26:47.754399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.848670 (+ 94271us) service_pool.cc:224] Handling call
0709 12:26:48.737317 (+888647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.753332 (+ 16015us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.975865 (+222533us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b54001955d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.976083 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2042935680}
I20260709 12:26:48.886714 10977 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:48.951475 10912 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:48.942111 11142 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:26:49.003988  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.035785  6452 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:49.003854  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:48.920609 11093 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.009646  6380 rpcz_store.cc:276] 0709 12:26:47.749612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.754807 (+  5195us) service_pool.cc:224] Handling call
0709 12:26:48.943944 (+1189137us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:48.944083 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2355410304}
W20260709 12:26:48.934403 11025 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.002s
I20260709 12:26:49.015033  6385 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: [7.4-9.5]
I20260709 12:26:48.909703 10925 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.090s
I20260709 12:26:49.051173  6381 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:49.051769  6380 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:49.053814  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:49.054382  6459 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.055956  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:49.058319  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.058739  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.059280  6300 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.057890  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:49.061997  6310 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:26:49.033440  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95661.8us
I20260709 12:26:48.932260  6206 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 9 to 12, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.038901  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.064052  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.040164  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.066382  6309 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:49.066856  6307 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:49.043885  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 626) took 1261 ms. Trace:
I20260709 12:26:49.044858  6385 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:26:49.068252  6384 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:49.069708  6385 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:49.072180  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.072887  6309 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:49.074122  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:49.076699  6380 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:49.067233  6378 rpcz_store.cc:276] 0709 12:26:47.782194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.889692 (+107498us) service_pool.cc:224] Handling call
0709 12:26:48.944785 (+1055093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.981849 (+ 37064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.043708 (+ 61859us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400037850. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:49.043834 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2425820032}
I20260709 12:26:49.084188  6378 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:48.932840 10998 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.109s
W20260709 12:26:49.088559  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48459.9us
I20260709 12:26:49.089793  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.090615  6459 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:49.026146 11146 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:26:49.091099  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52263.7us
I20260709 12:26:49.092162  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.093796  6274 leader_election.cc:400] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:26:49.094136  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:49.099635  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.101153  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:49.035957 11142 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:49.104117  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.108841  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.048123 10896 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:49.116834  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.109841  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.117921  6308 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.043231 10912 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:49.118305  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.116362  6206 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 1 to 8, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 8 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.119041  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:49.119884  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.115402  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.112208  6300 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:49.127036  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:49.128386  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:49.081176 11086 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:49.129597  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.081465 11078 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:49.137213  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:49.137677  6458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.008s	sys 0.018s
W20260709 12:26:49.095904 11024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.008s	sys 0.043s
I20260709 12:26:49.139545  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.096462 11187 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:49.096815 11109 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:49.162470  6458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.009s	sys 0.021s
I20260709 12:26:49.166769  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:49.110440 10913 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:49.163488  6300 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:26:49.143115  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.154635  6206 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 8 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 10 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:49.110605 10879 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:49.116201 10896 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:49.123927 10978 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:49.113157 11050 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:26:49.123804 11148 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.183794 11109 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.215878 10679 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.216511 10667 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:26:49.159530 11093 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:49.136936 11008 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.224814 11008 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.225229 11008 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:49.158156 11024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.009s	sys 0.044s
I20260709 12:26:49.165298 11078 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.124105 11160 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:49.056931 11139 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.142354 11146 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:26:49.102463 11149 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.217998 11109 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.218525 10667 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:49.221446 11141 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:49.165292 10696 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.222193  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:49.130381 10988 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:49.177896 11042 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.143113 11194 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.223436 11050 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:49.161160 11086 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.227557  6453 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:49.228741  6450 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:26:49.230075  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:49.336109  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:49.336678  6459 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.136682 11054 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:49.234165  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.202003 11193 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.173043 10912 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:49.135169 11189 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.186041  6451 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:49.252522  6301 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:26:49.343897  6301 pending_rounds.cc:85] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 4
I20260709 12:26:49.179534 11187 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:49.254137  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.235612 11024 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.214772 10982 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:26:49.207446 10977 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.268020 11138 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.230111 10679 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:49.278651  6206 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 5 to 9, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 9 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:49.288226 11050 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:49.286283  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.367455  6206 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 4 to 7, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 7 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:26:49.287294  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 65197.7us
I20260709 12:26:49.336519 10988 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:49.292079  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 107524us
W20260709 12:26:49.292774  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 364332us
I20260709 12:26:49.296640 11075 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:49.308099  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69937.9us
W20260709 12:26:49.310045  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 198459us
I20260709 12:26:49.287452 11104 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
W20260709 12:26:49.378391  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.318403  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136004us
I20260709 12:26:49.341814  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.344178  6301 pending_rounds.cc:107] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:26:49.380195  6301 raft_consensus.cc:2889] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.253082  6308 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:49.284685 11109 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.284020 10696 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.284459 11139 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.354933  6459 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:49.334586 11149 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:49.361545  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 448359us
I20260709 12:26:49.289144 11143 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:49.188329 10893 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.210583 11142 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.368605  7285 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.369436  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:49.445362  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:49.344642 11054 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:49.370028  6206 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 6 to 9, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 9 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:49.374953  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.383437  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.454952  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.387590  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.457515  6456 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:49.346107 11189 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:49.399494  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 260027us
I20260709 12:26:49.287289 11141 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:49.416993  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.445724  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76359.4us
I20260709 12:26:49.216884 10896 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:49.450641  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58165.5us
I20260709 12:26:49.465499  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.457382  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.403761 10696 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.471465  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:49.485093  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:49.486696 11052 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:49.487983 11052 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:49.488425 11052 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.488834 11052 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.490679 11040 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:49.491194 11040 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:49.492228 11040 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.493974 11040 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.494733 11195 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:49.496191  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 118181us
W20260709 12:26:49.458361 11051 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.008s
I20260709 12:26:49.498317  6381 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:26:49.483134  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102222us
I20260709 12:26:49.483547 10912 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.521917  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:49.510824  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:49.459391 11054 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.517709  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:49.452808 11185 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.533s	user 0.013s	sys 0.002s
I20260709 12:26:49.454500 10667 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.480145  6455 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:26:49.530898  6383 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:26:49.468576 10679 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.544576  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 665) took 1478 ms. Trace:
I20260709 12:26:49.544991  6307 rpcz_store.cc:276] 0709 12:26:48.066598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.300656 (+234058us) service_pool.cc:224] Handling call
0709 12:26:49.544393 (+1243737us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:49.544521 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1660236544}
I20260709 12:26:49.459736 11189 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.477780 10971 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:49.562991  6300 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:26:49.566118  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 629) took 1576 ms. Trace:
W20260709 12:26:49.574986  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 206534us
I20260709 12:26:49.469866 11177 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:26:49.532306 10896 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.590730 11220 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.571434 11112 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:26:49.571209 11075 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:49.583900 11001 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.003s
W20260709 12:26:49.279676 10619 consensus_queue.cc:1175] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4268023ecea149cd83cba77db53c3100" responder_term: 10 status { last_received { term: 9 index: 5 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:49.595551 11205 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:49.545924 11185 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:49.576818 11144 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.624s	user 0.017s	sys 0.000s
I20260709 12:26:49.571616  6379 rpcz_store.cc:276] 0709 12:26:47.989964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.985291 (+995327us) service_pool.cc:224] Handling call
0709 12:26:48.985580 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.087987 (+102407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.088017 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.093386 (+  5369us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.294011 (+200625us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:49.294078 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.294176 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.565839 (+271663us) raft_consensus.cc:1685] finished
0709 12:26:49.565866 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.566045 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47689,"raft.queue_time_us":56521,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":15367,"thread_start_us":160692,"threads_started":1,"wal-append.queue_time_us":266644}
I20260709 12:26:49.615001 10998 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:49.616813 11187 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.565963 11050 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.567265 10971 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:49.471342 11214 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.136s
I20260709 12:26:49.536669 10912 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.619282 11138 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:49.589515 10876 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:49.672801 10876 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.673831 10876 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.624024 11142 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.624826 11112 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:49.659611 11078 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.626724  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.621191  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2292) took 1523 ms. Trace:
W20260709 12:26:49.571082  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 105643us
I20260709 12:26:49.579360 10679 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:49.650446 11186 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.664s	user 0.012s	sys 0.004s
I20260709 12:26:49.655853  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:49.656102  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 142312us
I20260709 12:26:49.544142 10988 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.657389  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.660004  6452 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:49.587563 11141 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.626969 11025 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.662310  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2656) took 1344 ms. Trace:
I20260709 12:26:49.663331  6376 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
W20260709 12:26:49.664703  6382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.002s	sys 0.007s
I20260709 12:26:49.707839  6376 pending_rounds.cc:85] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 7
W20260709 12:26:49.708087  6382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.002s	sys 0.007s
W20260709 12:26:49.666291  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 183871us
I20260709 12:26:49.672127 10912 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.710356  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.616813 11008 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.678387 11112 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:49.594259 11001 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.682713  6303 rpcz_store.cc:276] 0709 12:26:48.097620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.728149 (+630529us) service_pool.cc:224] Handling call
0709 12:26:48.733135 (+  4986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.733366 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:26:48.856901 (+123535us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.620657 (+763756us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b18004190e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287
0709 12:26:49.620770 (+   113us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.620836 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:49.620903 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.620984 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.621116 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5398,"prepare.queue_time_us":24389,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":59,"raft.queue_time_us":130079,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":32974,"spinlock_wait_cycles":10180352,"thread_start_us":699614,"threads_started":3,"wal-append.queue_time_us":504300}
I20260709 12:26:49.646330 11144 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.633108  6300 pending_rounds.cc:85] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:26:49.724694  6300 pending_rounds.cc:107] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:26:49.725028  6300 raft_consensus.cc:2889] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:49.663861 11137 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.692s	user 0.008s	sys 0.008s
I20260709 12:26:49.680150 10971 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.697302 11226 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:26:49.683077 11181 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.003s
W20260709 12:26:49.704896  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47629.4us
I20260709 12:26:49.703457  6385 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:49.708144  6376 pending_rounds.cc:107] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 14.8
W20260709 12:26:49.667150  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 209882us
I20260709 12:26:49.737355  6376 raft_consensus.cc:2889] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:49.737355  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.707247  6383 rpcz_store.cc:276] 0709 12:26:48.317945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.808282 (+490337us) service_pool.cc:224] Handling call
0709 12:26:49.662023 (+853741us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:49.662232 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659234048}
I20260709 12:26:49.685200  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.724875  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:49.659888 10998 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:49.656810  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 194381us
W20260709 12:26:49.732290  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 151200us
I20260709 12:26:49.708958 11193 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:49.742512  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:49.745368  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.748170  6451 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:49.731621 11224 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:49.750432  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65294.5us
I20260709 12:26:49.755949  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.751549  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.757189  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.758729  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:49.769605  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.743501 10994 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:49.780210 11192 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.325s	user 0.000s	sys 0.002s
I20260709 12:26:49.746829 11181 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:26:49.782641  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2382) took 1327 ms. Trace:
I20260709 12:26:49.736720 11226 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:26:49.760181 11217 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:49.755277 11221 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.798831 11226 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.786665 11224 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:26:49.767144 11220 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:26:49.770040 11078 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.804036 11220 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:49.757100 11201 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:49.804514 11220 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:49.805481 11220 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.785256  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:49.769596 11214 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:26:49.784246  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2645) took 1687 ms. Trace:
I20260709 12:26:49.737298 11137 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.790294  6310 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:49.791114 11190 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:49.792866 11144 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:49.794128  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2646) took 1697 ms. Trace:
I20260709 12:26:49.767781 11193 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:49.783459 11213 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.803128  7316 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:26:49.822386  7316 pending_rounds.cc:85] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 8
I20260709 12:26:49.822862  7316 pending_rounds.cc:107] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:26:49.823123  7316 raft_consensus.cc:2889] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:49.750475 11199 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.287s	user 0.001s	sys 0.001s
I20260709 12:26:49.750707 11051 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:49.774217  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:49.833334  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.833693  6457 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:49.790271 11187 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.783318  6458 rpcz_store.cc:276] 0709 12:26:48.455261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.455483 (+   222us) service_pool.cc:224] Handling call
0709 12:26:48.455986 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.456360 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.456406 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.457103 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.167434 (+710331us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:49.167480 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.167554 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.782304 (+614750us) raft_consensus.cc:1685] finished
0709 12:26:49.782336 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.782571 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":701451,"threads_started":1,"wal-append.queue_time_us":886118}
I20260709 12:26:49.789013  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.746694 11177 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:26:49.810124  6377 rpcz_store.cc:276] 0709 12:26:48.096496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.988769 (+892273us) service_pool.cc:224] Handling call
0709 12:26:48.989071 (+   302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:48.989455 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.989484 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:49.045703 (+ 56219us) log.cc:844] Serialized 52 byte log entry
0709 12:26:49.356837 (+311134us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:49.496856 (+140019us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.497287 (+   431us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.682343 (+185056us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.682612 (+   269us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.784002 (+101390us) raft_consensus.cc:1685] finished
0709 12:26:49.784022 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.784158 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":184664,"prepare.queue_time_us":385299,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":191083,"raft.run_cpu_time_us":382,"raft.run_wall_time_us":6533,"thread_start_us":334368,"threads_started":2,"wal-append.queue_time_us":627057}
W20260709 12:26:49.815836  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.819149  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50836us
I20260709 12:26:49.820380 11216 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:49.818220  6382 rpcz_store.cc:276] 0709 12:26:48.096728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.990888 (+894160us) service_pool.cc:224] Handling call
0709 12:26:48.991163 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.991459 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.991484 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.991924 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.709048 (+717124us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:49.734068 (+ 25020us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.734314 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.734394 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.793823 (+ 59429us) raft_consensus.cc:1685] finished
0709 12:26:49.793857 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.794051 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19313,"prepare.queue_time_us":25149,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":2374,"raft.queue_time_us":1505,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":10412,"spinlock_wait_cycles":20528000,"thread_start_us":716606,"threads_started":1,"wal-append.queue_time_us":780007}
I20260709 12:26:49.825529 11220 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.828656 11008 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.772675 10977 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:26:49.831204 10696 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.831625 11097 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:49.818542 11185 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:49.816932 11201 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:26:49.839365  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.760929 11141 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
W20260709 12:26:49.883244  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:49.827308  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58425.6us
I20260709 12:26:49.815981 11078 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 12:26:49.770684 11112 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.848935 11193 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:49.837266  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 48347.9us
I20260709 12:26:49.891690 11185 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:49.892592 11185 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.893543 11112 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.895519 11137 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.853260 11216 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:49.898181  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.906873 11078 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:49.907096 10913 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.907377 11078 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:49.907651 10913 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.908279 11078 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.887945 11123 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:49.891384 11226 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:49.910668 11181 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:49.911082 11181 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:49.911885 11181 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.913106 11181 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:49.918215  6456 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:49.887773 10982 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.899704 11216 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.888863  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.919718 10982 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.920483 11078 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.920677  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:49.921686 11193 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:49.921967 10982 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.922411 11193 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:49.823032 10896 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:26:49.925397  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:49.925938 11189 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.927196 11054 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.892968 10998 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.932668  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:49.933354 10696 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.909956 10913 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:49.933732  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:49.934820  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:49.935119  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:49.936281  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:49.937088  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:49.937274  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:49.958478  6378 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:49.958422  6381 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:49.866636  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.940567  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.942982  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.944159  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.960618  6307 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:26:49.961372  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94902.1us
I20260709 12:26:49.961716  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:49.946790  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61754.2us
I20260709 12:26:49.963382  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:49.964704  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:49.965356 11194 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:49.965831 11194 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:49.966748 10998 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:49.966811 11194 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:49.967746  6379 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:49.967689 11050 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:49.974228  6382 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:26:49.971138 11054 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.955641  6300 raft_consensus.cc:2955] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Committing config change with OpId 12.8: config changed from index 2 to 8, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:49.982420  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:49.975481 11216 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:49.983760 11194 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:49.985167 11216 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:49.983932 11219 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:49.986083 11086 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:49.986971  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:49.991797 11221 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:49.993415  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:49.993654  6450 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:50.035590  6307 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:49.994691  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.036716  6458 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:49.997834 10876 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:49.991714  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:50.001555  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.044118  6451 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:50.006569 11052 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.013281  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.052305  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.052996  6304 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:50.053638  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2258) took 1413 ms. Trace:
I20260709 12:26:50.054008  6385 rpcz_store.cc:276] 0709 12:26:48.639836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.070400 (+430564us) service_pool.cc:224] Handling call
0709 12:26:49.070670 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.737076 (+666406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.029326 (+292250us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b5400064854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:50.053579 (+ 24253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1454347520}
I20260709 12:26:50.055083  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:50.014077 10679 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.021122  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:50.027285  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2245) took 2182 ms. Trace:
I20260709 12:26:50.032212  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:50.033812  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.059514  6376 rpcz_store.cc:276] 0709 12:26:47.844812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.936233 (+1091421us) service_pool.cc:224] Handling call
0709 12:26:49.662835 (+726602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.738185 (+ 75350us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.027046 (+288861us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54001f40d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:50.027208 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1475169536}
W20260709 12:26:50.044133  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52488.2us
I20260709 12:26:50.003587  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 664) took 1026 ms. Trace:
I20260709 12:26:50.060931  6457 rpcz_store.cc:276] 0709 12:26:48.977081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.985862 (+  8781us) service_pool.cc:224] Handling call
0709 12:26:48.986138 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.834157 (+848019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.003133 (+168976us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b54001617d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:50.003387 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933342592}
I20260709 12:26:50.050081  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.056015  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:50.064210  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.064977  6307 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:50.065686  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44636.1us
W20260709 12:26:50.065974  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.060241  6301 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:50.070389  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:50.066357  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:50.066938  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.067754  6309 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:50.058110  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.068243  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.070477  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.066672  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.030660 10988 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.072232  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:50.078136  7314 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:50.058272 11143 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:50.090787  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.098224  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:50.099731  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:50.063315 11148 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.045312 11226 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:26:50.106662  6303 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:50.074103 11097 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:50.060750 11201 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:26:50.074616 11186 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:50.127415  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.071344 11008 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:26:50.128515  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.129731 11184 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:50.139520 11184 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.096614 10988 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.095050 11143 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.128902 11224 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:50.102849 10912 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:50.141144  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.142060  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.131636 11025 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:50.145949  7317 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:50.100324 11213 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:50.101219 11040 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:50.099670 11152 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:50.103996 11109 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:50.074894 11220 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:26:50.149998  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.089946 11112 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:26:50.130812 11097 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.149521 11186 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:50.144591 11008 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.148015 11143 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.173790  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.148698 11239 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:50.148304 11224 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:50.127962 11148 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.153506  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.151417 11025 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:50.176052  6385 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:50.157413 11152 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.184981  6381 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:50.182863  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.188571  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:50.154692 11040 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:50.195091  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:50.205325  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.206590 10912 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:50.207158 10912 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:50.208284 10912 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.209898 10912 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.162216 10982 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:50.164763 11109 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:50.210705 10982 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:50.211163 10982 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.211344 11109 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.171613 10977 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.215903 10876 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.192641 11177 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:26:50.171861 11097 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.222666 11189 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.224550  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.225378  6381 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:50.225239 11148 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.179682 11008 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.222782 11240 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:50.229043 11240 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:50.229415 11237 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:26:50.229967 11240 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:50.231065 11242 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:26:50.231563 11052 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.163970 11213 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:50.200397 11224 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:50.232833 11213 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.203572 11214 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.233999 11214 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.191902 11137 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:50.236804 11214 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:50.235428  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.239670 10988 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.191084 11239 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.240424 11239 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.205183 11152 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.240654 11040 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.243383 11186 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.245513  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:50.201086 11025 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:50.247879  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:50.248871  6385 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:50.250293  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.250542  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.251195  6307 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.251802  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.249521  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.182651  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.255941  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.256654  6303 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:50.257812  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:50.262336  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.264775  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.257021  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.290136  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.291051  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.291561  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:50.292807  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.294013 10667 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.294762 10667 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.310261  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.321319  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:50.295393 10667 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.321357  6376 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [9.6-13.8]
I20260709 12:26:50.329926 10679 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:50.339691  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.341563  6382 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:26:50.345072  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.345209  6383 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:50.349159  6307 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:50.350874  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.271291 10988 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.353477  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:50.353977  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:50.285689 11086 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:50.403924  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50261.5us
I20260709 12:26:50.410090  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.411154  6455 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:26:50.385365  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.414350  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74798.5us
I20260709 12:26:50.413489 10988 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:50.418776 11244 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:26:50.420487 10988 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.401098 11138 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:50.426059 10679 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:50.398383 11221 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:26:50.428607 11137 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:50.443557 11194 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:50.452508 11148 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:50.474998  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 121104us
W20260709 12:26:50.480013  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 278685us
I20260709 12:26:50.439538 11224 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.462598 11078 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.482697  6307 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:50.474822 11194 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:50.475555  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 120545us
I20260709 12:26:50.509024  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.510977 11254 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:50.511101  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:50.511600 11254 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.512393  7331 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:50.512805  6303 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:26:50.503189 11190 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:50.514864 10982 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.514935 11190 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.517251 10982 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.517678 11190 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.469758 11137 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.506229 11148 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:50.519335 11194 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.519574 11148 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.520363 11137 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:50.520995  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:50.521798  7331 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:50.519750  6303 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:50.523594  6303 raft_consensus.cc:606] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:50.524399  6303 raft_consensus.cc:614] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:50.525545 11254 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.527218 10667 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:50.528801  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:50.528746  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.530689  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:50.522040 11025 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.535935 11214 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:26:50.535928 11052 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.539189 11052 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.541376 11052 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.555795 10971 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.561059 11138 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:50.561559 11138 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:50.562878  6302 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [9.6-13.8]
I20260709 12:26:50.563371  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.499796 11075 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.570358 11184 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.570783 11075 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.572578 11025 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.573024 11075 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:50.573613 11250 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.347s	user 0.003s	sys 0.000s
I20260709 12:26:50.532997 11078 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.578868 11078 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.582661  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:50.583572  7316 raft_consensus.cc:2410] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:26:50.582587 11143 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.586916  6455 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:50.588133  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.592674  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:50.591302  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 82413.4us
I20260709 12:26:50.593235  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.593416  6302 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:50.593950  6457 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:50.600407  7316 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:50.603250  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.603967  7331 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.604809  6455 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:50.604743 11097 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.606156  6301 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:26:50.608184 11137 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:26:50.607079 11152 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.615693 11137 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:50.616120 11137 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:50.617024 11137 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.617791 11152 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.602530  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.585652 11143 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:50.620769  6302 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:50.613705 11008 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:50.650309  6351 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.655s (SENT) [suppressed 8 similar messages]
I20260709 12:26:50.665323  7331 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:50.650856  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:50.668103 11141 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:50.668500 11141 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:50.668891 11141 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:50.673106 11254 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:26:50.680274 11263 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.680570  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.680966 11263 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.595248 11193 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:50.607898 11234 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:50.588435 11239 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.703841  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.704242  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.707535  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:50.597755 11223 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 12:26:50.684361 11137 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.595618 11138 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.632668 11184 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:26:50.708729  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.711462  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.698874 11220 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:26:50.712882  6307 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:50.714691  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:50.735615  6307 raft_consensus.cc:606] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.736092  6307 raft_consensus.cc:614] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:50.716305  6310 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:50.719764  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.737182 10696 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.738159 11229 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:50.738622 11229 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.711620 11221 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:26:50.739013 11229 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:50.740485 11221 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:50.740832 11221 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:50.741597 11221 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.741816 11141 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.735149  6302 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:50.744932 11141 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.746716 11220 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:50.747165 11220 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:50.747937 11220 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.748816 11221 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.749979 11263 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.750656 11097 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.752633 11224 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.753914  6302 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:50.759042  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.760713  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:50.758934  6304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c413b06caa444496aecb13ac2142c246"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:50.762463  6304 raft_consensus.cc:606] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.762801  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.763995  6457 raft_consensus.cc:2410] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:26:50.766872  6309 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:50.709417 11226 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:50.767669  6309 raft_consensus.cc:606] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.768236  6309 raft_consensus.cc:614] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:50.709857  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59585.3us
I20260709 12:26:50.769253  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.712111 11234 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:50.762501  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:50.772336  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.732333 11143 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:50.774390 11143 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:50.775238 11143 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.776414 11143 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.780427 11149 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:50.726346  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.797693  6309 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:26:50.798784  6309 raft_consensus.cc:606] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:50.799942  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:50.800261  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.801730  6458 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:50.799362  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:50.805244  6452 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:50.813365  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:50.815795  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:50.817533  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:50.821805  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 114345us
I20260709 12:26:50.821811  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.824119  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.824841  6385 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:50.826793  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.827276  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:50.827510  6302 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:50.827840  6304 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:50.828727  6452 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:50.829532  6381 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:26:50.829794  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.837745  6451 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:50.824065  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:50.839315  6376 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:50.839704  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:50.843060  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.843686  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:50.848662  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.827852  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:50.849663  6382 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 9: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:50.849298  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:50.850389  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:50.817672 11220 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:50.851222  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:50.821882 11201 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:50.856619  6383 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:50.892234  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.899220  6451 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:50.897174  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44110.2us
I20260709 12:26:50.823652 10977 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:50.842203 11211 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:50.902343  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:50.905588  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.912624  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:50.856038 11263 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:50.854802 11224 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:50.857702 11177 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:50.913601  6382 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: [7.4-10.5]
I20260709 12:26:50.923553  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:50.925024  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:50.925844  7314 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:50.916564  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.902951 10619 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:50.913491 11075 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:26:50.938479  6303 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:50.921936 11112 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.967847  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:50.974344  6451 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:50.975641  6452 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:26:50.941751 10619 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.941185 10667 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:50.994983 10619 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:50.996392 11272 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:26:50.998828 11246 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.784s	user 0.000s	sys 0.003s
W20260709 12:26:50.999600 11269 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:26:51.005173 10619 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:51.027842 11266 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:26:51.030611  6376 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:26:51.031103  6376 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 5
I20260709 12:26:51.031378  6376 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:26:51.031652  6376 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:51.034029  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:50.993459 11257 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.576s	user 0.003s	sys 0.003s
I20260709 12:26:50.946863 11214 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:51.037709 11214 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.038137 11214 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.040822 11239 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:50.939098 11224 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:51.043294 11224 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:50.918898 10977 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.046171 10977 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.993901 11260 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:50.994654 11277 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:51.070626 11277 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:51.073462 11008 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.075448 11234 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:50.919370 11211 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:51.077009 11268 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.167s	user 0.002s	sys 0.000s
I20260709 12:26:51.078084 11211 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.078855 11138 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.079677  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.080853  6451 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:51.082569 11282 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:26:50.991216 11112 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:51.085038 11273 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.278s	user 0.002s	sys 0.000s
I20260709 12:26:51.085258 11112 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.046451 11141 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:51.087342  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:51.087733 11141 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.090767 11141 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:51.047134  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:51.091747  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43606.5us
I20260709 12:26:51.049222  6385 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:51.086100  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.064195  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:51.092764  6451 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:51.094380  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:51.066955 11040 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.068650 11239 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.068856 10619 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.047040 11260 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:51.069672 11226 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:50.942509 11177 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:26:50.960266 11186 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.016s	sys 0.016s
I20260709 12:26:51.092564  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:51.102902  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:51.104367  6451 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:26:51.112555  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.114063  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:51.118556  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 205761us
I20260709 12:26:51.132256  6452 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.119076 11260 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:51.105141 11139 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:51.138090 11186 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.017s	sys 0.017s
I20260709 12:26:51.112511 11211 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:26:51.164083  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 261813us
I20260709 12:26:51.166239  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:51.169510  6455 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:51.174239  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:51.175164 11078 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:26:51.175603  6452 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:51.175895 11078 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:51.177166 11078 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:51.179606  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 239654us
I20260709 12:26:50.994937 11202 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.304s
I20260709 12:26:51.155454  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.186651  6385 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:51.188002  6458 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:26:51.209218  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53881.6us
W20260709 12:26:51.156755  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 263877us
I20260709 12:26:51.107362 11239 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:51.195612 11249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.972s	user 0.005s	sys 0.027s
I20260709 12:26:51.163924 11211 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.204241  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:51.190330 11201 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.210932 10977 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:51.212574  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.162688 11139 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.219689  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.220495  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.222164  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:51.224578  7331 raft_consensus.cc:2376] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11. Re-sending same reply.
I20260709 12:26:51.212074 10667 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:51.215199 11207 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:26:51.239889  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:51.240864  6300 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.242748  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:51.243912  6307 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:51.194361 11233 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:26:51.205312 11184 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:51.250599  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:51.216830 11249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.005s	sys 0.028s
I20260709 12:26:51.265633  6308 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:26:51.268404  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47977.1us
I20260709 12:26:51.269222  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:51.270826  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66653.6us
I20260709 12:26:51.273576  6301 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:51.274199  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:51.277206  6300 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:51.244956 11211 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.246654 11234 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.246093 11275 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.216135 11216 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:26:51.285936  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.258276 11025 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:51.218904 11229 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.299392  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.300108  6376 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:26:51.300195  6381 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:51.252207 11008 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:26:51.299444  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.254980 11281 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:26:51.304522  6384 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:26:51.299376 11025 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:51.308851 11025 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.310755 11262 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:51.311935  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 923) took 1071 ms. Trace:
I20260709 12:26:51.312346  7317 rpcz_store.cc:276] 0709 12:26:50.240759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.240915 (+   156us) service_pool.cc:224] Handling call
0709 12:26:50.241273 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.241672 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.241701 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.242153 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.787864 (+545711us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b18004b1180. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1
0709 12:26:50.788072 (+   208us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:50.788574 (+   502us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.788874 (+   300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.788996 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.831082 (+ 42086us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:51.018367 (+187285us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b18004b1180. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:51.308712 (+290345us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b24002eee78. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f8fb0 00007f6fa2e6f488 00007f6fa2e6fbe8 00007f6fa2e47398 00007f6fa2e4276a 00007f6fa2e41bb6
0709 12:26:51.311722 (+  3010us) raft_consensus.cc:1685] finished
0709 12:26:51.311742 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.311865 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25872,"prepare.queue_time_us":15763,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":26370,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":26659,"spinlock_wait_cycles":124410496,"thread_start_us":48970,"threads_started":1,"wal-append.queue_time_us":155453}
I20260709 12:26:51.314744 10696 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:51.315608 11263 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:51.316354 11229 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.317086 11143 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.317687 11143 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.244391 11201 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.324520 11201 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.326647 11291 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:51.329625 11137 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.002s
I20260709 12:26:51.294094 11234 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.332511 11234 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.333295  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:51.335233 11184 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:51.335963 11184 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:51.336535 11139 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:51.291436 11275 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.337030 11184 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.339545 11275 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:51.340478 11263 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:51.340322  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:51.341797  6308 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.341727 11263 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.343518 11239 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.232347 11177 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.347474 11185 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:26:51.349053 11185 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:51.349465 11185 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:51.351003 11214 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.363533  6377 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:51.353374 11214 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.339715 11143 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.406988  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:51.408092  6305 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:51.409243  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:51.410964  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:51.350308 11185 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.428220 11185 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:51.442247  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:51.443269  7317 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:26:51.447903  6308 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:51.450907 11143 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.451643  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:51.451539 11143 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.452752  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.453428  6455 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:26:51.457108 11143 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.459020 11202 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:51.459452 11202 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.459802 11202 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:51.460585 11286 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.491s	user 0.000s	sys 0.006s
I20260709 12:26:51.462809  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:51.464854 11260 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.466166 10982 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:51.485822  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:51.471613  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:51.473709  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:51.484372  7316 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:51.373793 11211 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.490762  6309 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:51.494047  6307 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:51.500403  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:51.497735 11279 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.461s	user 0.000s	sys 0.016s
I20260709 12:26:51.533665  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:51.536440  7331 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
W20260709 12:26:51.536784  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 267620us
I20260709 12:26:51.541430  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:51.537920  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:51.497958 11143 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:51.548579  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:51.556653  6455 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:51.560122  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 220159us
I20260709 12:26:51.561062  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.532820 11221 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.533123 10925 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:51.544173 11181 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:51.546882 11216 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:51.584245  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:51.584914  6457 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:51.585580 11211 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.586786  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.587426 11112 raft_consensus.cc:993] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:26:51.588141 11112 raft_consensus.cc:1081] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:26:51.582963 11177 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:51.595336 11177 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.595765 11177 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:51.599300 11211 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:51.599550  6452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c413b06caa444496aecb13ac2142c246"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59632
I20260709 12:26:51.600168  6452 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:51.599900 11211 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.600551  6452 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.602061 11211 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.603358  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.611101  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.563668 11143 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.613797  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.625085  6378 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:26:51.584640 11181 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.612749 11216 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.630384 11143 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:51.578504 11141 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:26:51.613046 10925 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:51.582626 11221 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:51.661785  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100887us
I20260709 12:26:51.662788 11052 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:51.664592  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.671561  6274 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:26:51.685040  6274 leader_election.cc:403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.675160  6455 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:51.678984 11177 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.682224  6457 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:26:51.690620  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:51.691726 11194 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:51.692188 11194 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:51.692484  6452 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.693061 11194 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.656054 11181 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.697836  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.699038  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:51.699741  6301 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:51.700074  6301 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:51.700273  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.700894  6301 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.660820 11216 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:51.706677 11025 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
W20260709 12:26:51.708732 11301 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:51.711524  7319 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:26:51.717756  6452 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.723197  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:51.763999  6457 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:51.766320 11040 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.766798  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:51.767658  6457 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 15
W20260709 12:26:51.761155  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46047.7us
I20260709 12:26:51.770747  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.761847  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.713414 11078 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.794205  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.796278  6458 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:51.800406  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.802853 10998 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:51.766983 11040 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.760699 11138 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.702008 11299 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:51.811492  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:51.811861  6307 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:51.818929  7319 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.725929 11275 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.821223  6452 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:51.688012 11214 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 12:26:51.825471  6457 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:51.828895  6452 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:51.805646 11040 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.816704  6304 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:51.832854  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:51.687340 11211 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.114s
I20260709 12:26:51.815299  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:51.815492 10925 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.848371 11042 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:51.849125 11042 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:51.821235 11260 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.868963  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 652) took 1267 ms. Trace:
I20260709 12:26:51.869422  7316 rpcz_store.cc:276] 0709 12:26:50.601754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.602223 (+   469us) service_pool.cc:224] Handling call
0709 12:26:51.868811 (+1266588us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54006801d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:51.868908 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818908032}
I20260709 12:26:51.840413 10619 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:51.876864  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:51.827979 11285 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:26:51.878819  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:51.827723 11275 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.820817 11138 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.881574  6452 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:51.883746  7319 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:51.880438  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:52.060231  6457 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:51.839804 11226 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:51.826025 11299 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.840183 11078 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
W20260709 12:26:51.897872 11312 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:26:51.898710 11305 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:26:52.047533 11233 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.901633  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:52.080974  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 179444us
I20260709 12:26:51.906026 11220 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:51.906565 10998 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:51.878386 10982 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:26:51.893463 11042 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:51.887799 10619 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.919869 11138 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:51.920469 11202 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:51.931335  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 105035us
I20260709 12:26:51.909821 11143 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:51.939164 11177 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:51.973273 10896 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.976876  6303 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:51.978806  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:51.979909  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:51.983902  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2671) took 1455 ms (client timeout 1883 ms). Trace:
W20260709 12:26:52.117765  6377 rpcz_store.cc:269] 0709 12:26:50.528102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.528578 (+   476us) service_pool.cc:224] Handling call
0709 12:26:51.983736 (+1455158us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b54001f45d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:51.983832 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917675392}
I20260709 12:26:51.993618 11181 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.094787 11040 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.093508 11220 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:51.999687  6301 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:51.956357 11216 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.058604  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:51.904198 11275 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:52.083739  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.103016  6457 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 12:26:52.104349  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.104730  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.105858  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.104386 11226 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:26:52.107987  6451 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:52.109308  6456 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:52.110416  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:52.106901  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.112406  6307 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:52.115202  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:52.115962  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:51.998039 11221 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:52.135015  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 156328us
I20260709 12:26:52.137091  7317 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:52.100008 11054 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.132642 11289 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.095846 11233 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.136435 11040 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.140468 11297 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:52.134141 10896 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:52.143659  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.143803  6457 pending_rounds.cc:85] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 4
I20260709 12:26:52.182679  6457 pending_rounds.cc:107] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:26:52.144085  6453 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:52.151211  6305 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.154667  6308 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:52.151449  7319 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:52.154803  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:52.159844  6456 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:26:52.182935  6457 raft_consensus.cc:2889] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:52.152657  6376 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 9.
W20260709 12:26:52.147226  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 88726.2us
I20260709 12:26:52.191718  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:52.171212  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:52.195820 10988 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:52.163426 11289 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.200250 11289 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.169934 11304 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:26:52.160754 11239 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:52.203101 11239 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.203946 11202 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.205093 11239 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.205773 10977 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:52.206519 10977 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:52.206760 10896 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.207584 10977 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.208596 11233 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.209828 11181 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.211642 11143 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.211825 11216 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.182885 11177 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.212481  6458 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:52.212579 11177 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.217423  6380 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:26:52.218214  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:52.176584  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
W20260709 12:26:52.140828  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 317599us
I20260709 12:26:52.222446  6382 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:26:52.229029  6305 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:26:52.229046  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:52.231475 11304 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:52.225844  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:52.272192  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54057.4us
I20260709 12:26:52.161111 11040 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.173462 11297 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:52.279467  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:52.280216  6376 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:52.232935  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.234048  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:52.284926  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59170.5us
I20260709 12:26:52.284787  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:52.285993  6304 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:52.159972 11144 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:52.288188  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54194.6us
I20260709 12:26:52.289685  7331 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:52.283974  6456 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:52.231890 11304 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:52.231958 11328 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:52.282784 11216 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:26:52.159577 11148 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:26:52.175326 11141 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:26:52.298746  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:52.283397 11319 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:26:52.286451  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:52.300570  6382 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:26:52.305306  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:52.305374  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:52.377166  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49826.3us
I20260709 12:26:52.377482  7331 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.282552 11315 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:52.299360 11304 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:52.160774 11054 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.296458 10925 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.271371 11138 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:26:52.379307  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.298810 10896 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:52.382086  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.410101  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.275952 11025 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:52.401772  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:52.399706  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 113475us
I20260709 12:26:52.412869  7316 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.291970 11024 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:26:52.412221  7319 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:52.402717  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:52.299733 11328 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:52.374809 11152 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.420676  7316 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:26:52.378880 11075 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.424468  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:52.424945  7331 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
W20260709 12:26:52.431167 11320 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:26:52.409411 10925 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.433142 10925 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.437147 11177 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:52.438159 11328 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.439465 11328 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:52.441036 11181 raft_consensus.cc:993] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:26:52.441701 11211 raft_consensus.cc:1081] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:26:52.437449 11277 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.444782 11277 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:52.444561 11075 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.447327 11075 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.409765 11326 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:26:52.427795 11152 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.412220 11185 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:52.456837 11152 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:52.453346 11138 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:52.455171 11207 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.463274 11207 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.465633 11207 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.460264 11297 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.303740  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:52.303257  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:52.463325 11138 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.428572 11025 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.534297  6301 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:52.574044  6453 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:52.616551 10667 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:52.616998 10896 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:52.595104  7318 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:52.704301 10896 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.636920 11025 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.642277  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:52.626091 11001 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.643088 11112 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.485283 11304 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.590029  7314 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:52.615521 10998 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.545975  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:52.633719 11138 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:52.634456 11139 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.629595  6455 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:52.553084  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:52.583000  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.547825  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:52.575747  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:52.708453 11112 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:52.646534 11181 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:52.730557  6382 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.678828 11054 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.715404 10667 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:52.643965  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 340173us
I20260709 12:26:52.708051 11001 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.737460 11277 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:52.738888 11328 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:52.739555  6378 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:52.739751  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 97922.5us
I20260709 12:26:52.648685 11201 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:52.740972  6304 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:52.647836 11185 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:52.733259 11221 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.637785  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:52.747092 11221 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.747279 11185 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:52.748219  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 110830us
I20260709 12:26:52.749596  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2345) took 1062 ms. Trace:
I20260709 12:26:52.750851 11025 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:52.751101 11304 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.656015 11207 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.751523 11025 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:52.752025 11304 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:26:52.753751 11207 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.754163  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:52.753214 11025 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.755122  6383 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:52.756451  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:52.739761 11328 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.760627  6380 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:52.760888 11328 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.763396  6452 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:52.764832  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:52.764919  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:52.766008  6384 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:52.733206 11112 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.768623  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.769941  6453 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:52.773175 11220 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.689386  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 680) took 1227 ms. Trace:
I20260709 12:26:52.774641  7317 rpcz_store.cc:276] 0709 12:26:51.462223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.462458 (+   235us) service_pool.cc:224] Handling call
0709 12:26:52.688729 (+1226271us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b5400565754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:52.689057 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548552576}
W20260709 12:26:52.776926  7314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 928) took 1733 ms (client timeout 1913 ms). Trace:
W20260709 12:26:52.777498  7314 rpcz_store.cc:269] 0709 12:26:51.044085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.046790 (+  2705us) service_pool.cc:224] Handling call
0709 12:26:52.776624 (+1729834us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400566154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:52.776839 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3442731264}
I20260709 12:26:52.778963 11181 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.773798  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:52.749182 11201 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.780550  6383 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:52.750006  6308 rpcz_store.cc:276] 0709 12:26:51.687131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.688621 (+  1490us) service_pool.cc:224] Handling call
0709 12:26:51.688855 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.184981 (+496126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.749036 (+564055us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:52.749347 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070881664}
W20260709 12:26:52.784184  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 172708us
I20260709 12:26:52.764968  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:52.755117 11313 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.002s
I20260709 12:26:52.733939 11139 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.812106 11001 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.816279  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.817427  6452 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:52.819630  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:52.821475 11327 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:52.822005 11327 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.822434 11327 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:52.824910 10619 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.733381 10998 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:52.833068 10998 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:52.833760  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.837890  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:52.839833 11322 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.428s	user 0.000s	sys 0.003s
I20260709 12:26:52.784576 11277 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:52.842442  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.844314 11139 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:52.847220  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:52.849363 11328 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:52.850883  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:52.851866  7318 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:52.851328 11277 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:52.854758  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.855955  6452 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:52.868539  6453 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:52.893082  6382 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:26:52.902281  6455 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.916828  6378 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:26:52.919355  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.927227  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:52.924595  7314 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:52.934298  6377 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:52.946543  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:52.958107  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:52.948334 11220 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.960404 11317 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:52.961699  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:52.964854 10667 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:52.969990 11343 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:52.981178 11343 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:52.981642 10925 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.548s	user 0.019s	sys 0.000s
I20260709 12:26:52.970682 11267 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:52.982751 11267 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.985162 11267 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:52.966776  6301 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:52.980180  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:52.971112 11326 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:52.986773 11326 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:52.987238 11326 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:52.988078 11339 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:52.988937 11339 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:52.981705 11343 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.990235 11339 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:52.995347 11201 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.976008 11317 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.996284 11317 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:52.998768  6380 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:26:52.999437 11181 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:52.999622  6380 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:53.000195  6380 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:52.980109 10977 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:26:53.022176  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 505907us
I20260709 12:26:53.023988  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:53.031111  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:53.034166 11042 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.035144  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:53.035904  6301 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:26:53.036178  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:53.037115  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:53.034904 11042 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:53.035917 11040 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.740s	user 0.011s	sys 0.010s
I20260709 12:26:53.070631  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:53.006906 10925 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.078218  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:53.052512 11112 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.097586  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.104023 11260 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.104811 11260 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.106889  7331 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:53.110649 11054 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.111654 11054 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.112653  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:53.112984  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.113730  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:53.114720 11054 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.115667  7332 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:53.116612 11281 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.118788 11299 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.075927 11040 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:53.126809  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67351us
I20260709 12:26:53.142594 11267 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.128050 11149 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.131707 11042 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.134811 11008 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.135795 11260 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.136684 11138 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.174544 11343 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.139729 10925 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.176779 11343 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.177618 11281 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.178001  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.178337  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:53.179090  6454 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:53.179447  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:53.140055 11112 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.141404  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.181144 11297 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.181933 11187 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:53.181783 11297 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.182358 11187 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.106632  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:53.182814 11187 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:53.184175 11297 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.187879  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.188598  6385 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:53.126045 11327 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.165921  6455 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:53.168435  6376 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:53.168385  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.170676 11148 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.171845 11340 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:53.171432 11008 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.180258 11149 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.201802  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:53.185017  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.144502  6456 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:53.198336  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.200606  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:53.201472  7329 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:53.202900  6383 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:53.217732  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.226229  6309 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:53.226503 11291 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:53.227177 11291 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.226529  6381 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:53.174396 11221 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.237587  7316 raft_consensus.cc:2368] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15.
I20260709 12:26:53.232631  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:53.234308  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.245714  6385 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:26:53.220715 11340 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:53.196461 11201 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:26:53.251367  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:53.261682  6451 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:53.257445  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:53.263664  6454 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:53.258569  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:53.264644  6383 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:26:53.268175  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:53.250964 11138 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.228163 11267 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.371266  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.228543 11148 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.251397 11186 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:53.409693 11149 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.410683 11317 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.414008 11317 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.384617  6456 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:53.428763  6452 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
W20260709 12:26:53.443120  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 237633us
I20260709 12:26:53.372123 11281 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.453302  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:53.458102  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:53.459353  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.460405 11148 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.460944  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:53.461766  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:53.463245  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78829.2us
W20260709 12:26:53.464957  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 298662us
I20260709 12:26:53.504827  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.515436  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:53.516788 11220 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:53.517653 11220 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.518066 11220 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:53.518644 11326 consensus_meta.cc:287] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.531s	user 0.006s	sys 0.009s
I20260709 12:26:53.519205 11326 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.520416  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:53.522209 11352 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.275s	user 0.006s	sys 0.000s
I20260709 12:26:53.523168  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.524832  6300 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:53.527228  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 683) took 1222 ms. Trace:
I20260709 12:26:53.528159  7317 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:26:53.528604 11342 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:26:53.516868  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 660) took 1422 ms (client timeout 1764 ms). Trace:
I20260709 12:26:53.528898  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:53.536854 11040 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.537497 11340 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.539546  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:53.507159 11328 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:53.545127 11348 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 12:26:53.545284 11220 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.444847 11186 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.505963 11345 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:53.548877  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:53.549757  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:52.277240  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:53.554666 11181 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:53.596081 11220 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.596506 11181 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:53.597911  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49130.3us
I20260709 12:26:53.555939 11025 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:53.598791 11025 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.600963 11025 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.601087 11260 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:53.601516 11025 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.530185  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.564350  7317 pending_rounds.cc:85] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 9
I20260709 12:26:53.563537  7332 rpcz_store.cc:276] 0709 12:26:52.305888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.305996 (+   108us) service_pool.cc:224] Handling call
0709 12:26:53.526981 (+1220985us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:53.527161 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588446592}
I20260709 12:26:53.573513  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2898) took 1402 ms. Trace:
I20260709 12:26:53.574275 11008 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:53.566229  6381 rpcz_store.cc:269] 0709 12:26:52.094813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.095057 (+   244us) service_pool.cc:224] Handling call
0709 12:26:53.516592 (+1421535us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:53.516789 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2414828800}
I20260709 12:26:53.577054  6308 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:53.506134 11221 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:26:53.593003 11346 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.617s	user 0.004s	sys 0.008s
I20260709 12:26:53.590857  6453 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:26:53.610039  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81233.1us
I20260709 12:26:53.602033  6457 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:26:53.611897  6380 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.004s
I20260709 12:26:53.563057 11326 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.576381 11112 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:53.614178  6380 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.604858  7317 pending_rounds.cc:107] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:26:53.616780  7317 raft_consensus.cc:2889] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:53.578830 11267 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.624394  6382 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:53.642611 11342 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:53.643492 11342 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.643960 11342 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:53.646953 11112 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:53.648303 11112 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.650684  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:53.651930 11341 raft_consensus.cc:2764] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:26:53.652927  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.653056  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:53.654280 11341 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:53.654623 11328 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.655807  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:53.656594  6459 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:53.656746  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.657613 11328 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.659070  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:53.561058 11187 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.667102 11186 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.668017  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.670635 11181 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.676048  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:53.705876  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46891.5us
I20260709 12:26:53.605353  6450 rpcz_store.cc:276] 0709 12:26:52.164274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.165362 (+  1088us) service_pool.cc:224] Handling call
0709 12:26:52.165632 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:52.172154 (+  6522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:52.172191 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:52.172665 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:26:52.761721 (+589056us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b18004505e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:26:52.953741 (+192020us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:52.953820 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.957649 (+  3829us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:53.557279 (+599630us) raft_consensus.cc:1685] finished
0709 12:26:53.557314 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.567115 (+  9801us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1064,"raft.queue_time_us":739,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":12706,"spinlock_wait_cycles":48706432,"thread_start_us":160402,"threads_started":1,"wal-append.queue_time_us":1356470}
I20260709 12:26:53.676389  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:53.677978  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.677892  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.602121 11025 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.715600  6384 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:53.716463 11025 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.630908 11221 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:53.687469  6380 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.632330 11008 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.718232 11008 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.718297 11221 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.623724 11327 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.688978  6381 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:53.722911 11008 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:53.590061 11291 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.658260 11328 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.737037  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.763156  6451 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:53.763851  6455 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
W20260709 12:26:53.726678 11339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.045s	sys 0.063s
I20260709 12:26:53.774017  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:53.768162  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:53.782630  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:53.773450 11339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.046s	sys 0.063s
I20260709 12:26:53.788980  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:53.791868  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 556502us
I20260709 12:26:53.776372 11025 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.821172 11181 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:53.805567 11201 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:53.815110 11326 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:53.858330 11181 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:53.860117 11181 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.823887 11040 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:53.865831  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.821688 11149 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.867057  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.870852  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:53.841598 11233 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:53.805266 11328 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.856205 11201 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.875846 10982 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:53.884627 10982 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.832325 11025 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.888890 11025 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.866562 11040 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.864651 11326 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:53.829624 11281 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:26:53.877085 11233 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.903746 11187 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:53.904155 10982 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:53.907356 11149 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:53.890797 11201 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:53.909216  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 388869us
I20260709 12:26:53.895774 11040 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:53.911984  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 458752us
I20260709 12:26:53.896836 11326 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:53.891881 11339 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:53.922547  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.902683 11233 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:53.916498  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:53.922243  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 295962us
I20260709 12:26:53.926060  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:53.926330  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:53.927304  6455 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:26:53.984314  7332 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:53.947161  6450 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:54.003340 11328 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.004060 11328 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.004590 11328 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.941071  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:54.007308  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:54.008702  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67756.6us
I20260709 12:26:54.007149  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:53.977730  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:54.026772  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 110389us
I20260709 12:26:53.947387  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:54.030778  7318 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
W20260709 12:26:54.033142  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 494143us
I20260709 12:26:53.985420 11317 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.952070  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:54.057673  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:54.058777  6376 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:26:54.062685  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:54.059649  7316 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:54.070335 11370 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.071163  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:54.071434 11202 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:54.072181 11202 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:54.073530 11327 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:54.074335 11327 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:54.075380 11327 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.071135 11370 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.081570 11370 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:54.073486 11202 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:53.944783 10619 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:54.085909  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:54.063251 11317 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.089599  6351 leader_election.cc:400] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15.
I20260709 12:26:54.131523  6351 leader_election.cc:403] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:54.094877  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:54.101147  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2691) took 1102 ms. Trace:
I20260709 12:26:54.133014  6455 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.112884  6275 leader_election.cc:400] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:26:54.134987  6275 leader_election.cc:403] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:26:54.090780  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:54.093669  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:54.104944 11317 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.133368  6382 rpcz_store.cc:276] 0709 12:26:52.998343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.000535 (+  2192us) service_pool.cc:224] Handling call
0709 12:26:53.000808 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.625061 (+624253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.100799 (+475738us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b540023f0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:54.101033 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1433529088}
I20260709 12:26:54.111683  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 938) took 1261 ms. Trace:
I20260709 12:26:54.120677  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:54.090747  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:54.197649  7317 rpcz_store.cc:276] 0709 12:26:52.849790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.849924 (+   134us) service_pool.cc:224] Handling call
0709 12:26:52.850331 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.625497 (+775166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.111390 (+485893us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b5400439c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:54.111595 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557966208}
I20260709 12:26:54.202231  6457 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:54.206676  6383 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.825s	user 0.012s	sys 0.000s
I20260709 12:26:54.207145  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:26:54.208168  6450 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:26:54.209401  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 152287us
I20260709 12:26:54.212489  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:54.211122  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:54.210917  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:54.091080 11366 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
W20260709 12:26:54.220220  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 130859us
W20260709 12:26:54.218222  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 77774.7us
I20260709 12:26:54.221596  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:54.216456  7329 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:53.926389 11281 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.287s
I20260709 12:26:54.222951  7318 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:54.095480 11328 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:54.197904 11233 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.223624  7316 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:54.095746 11149 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.097538 11345 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.245086 11345 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.245838  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:54.246510  7314 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:54.100409 11040 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.248289  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:54.248904 11040 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.259119 11328 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:54.259812 11328 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.096324 11342 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.261988 11328 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:54.262760 11342 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:54.265412 11371 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.174s	user 0.000s	sys 0.002s
I20260709 12:26:54.266098 11281 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:26:53.945755 11339 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.242309  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:54.093050 11372 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:54.092785 11375 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.190s
I20260709 12:26:54.201462 11260 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:26:54.295670  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:54.314499  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45950.2us
I20260709 12:26:54.302853  6451 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:54.311743  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:54.329614  6459 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:54.304495  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:54.211762 11297 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:54.267694  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:54.271400  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:54.330941  7319 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:26:54.307964 11365 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.229s	user 0.001s	sys 0.001s
I20260709 12:26:54.342068  6455 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:54.269021  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2690) took 1271 ms. Trace:
I20260709 12:26:54.323143  7314 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:26:54.233489 11186 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:54.088778 10619 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.228937 11370 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:54.311233 11369 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:26:54.304317 11233 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:54.351752 11317 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.349349 11374 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:26:54.356503 11008 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:26:54.364686  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 329091us
I20260709 12:26:54.339028 11328 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:54.346863  6378 rpcz_store.cc:276] 0709 12:26:52.998032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.180221 (+182189us) service_pool.cc:224] Handling call
0709 12:26:53.180664 (+   443us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:53.216568 (+ 35904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.216604 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:53.258977 (+ 42373us) log.cc:844] Serialized 52 byte log entry
0709 12:26:53.620379 (+361402us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:26:54.208621 (+588242us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.224693 (+ 16072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.224777 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.268746 (+ 43969us) raft_consensus.cc:1685] finished
0709 12:26:54.268769 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.268942 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88482,"prepare.queue_time_us":126591,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":6024,"raft.queue_time_us":64003,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":19866,"thread_start_us":657313,"threads_started":2,"wal-append.queue_time_us":375851}
I20260709 12:26:54.379181  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:54.363899 11317 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.384369 11317 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.333283 11040 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:26:54.391224 11374 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:54.391994 11374 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.392408 11345 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:54.392436 11368 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:54.349342 11112 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:54.303602 11326 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.266408 11201 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.381021  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:54.415894 10619 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.422254  6457 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:26:54.425076  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44171.9us
I20260709 12:26:54.427259  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:26:54.477599  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50441.3us
I20260709 12:26:54.369707 11008 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:54.428237 11221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.019s	sys 0.007s
W20260709 12:26:54.495224  6384 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.013s	sys 0.008s
W20260709 12:26:54.499367  6384 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.014s	sys 0.008s
W20260709 12:26:54.497004  6302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.008s	sys 0.014s
W20260709 12:26:54.503723  6302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.008s	sys 0.014s
I20260709 12:26:54.431861 11342 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:54.511219  7314 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:26:54.490307 11340 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.016s
W20260709 12:26:54.495719 11221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.019s	sys 0.007s
I20260709 12:26:54.531836  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2365) took 1006 ms. Trace:
W20260709 12:26:54.502978  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.021s
W20260709 12:26:54.532876  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.001s	sys 0.022s
W20260709 12:26:54.511369  6385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.002s	sys 0.022s
I20260709 12:26:54.490675 11187 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:54.533493  6385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.003s	sys 0.022s
I20260709 12:26:54.394783 11370 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.532382  6302 rpcz_store.cc:276] 0709 12:26:53.525290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.540020 (+ 14730us) service_pool.cc:224] Handling call
0709 12:26:53.540356 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.540719 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:53.541078 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.264669 (+723591us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:54.504180 (+239511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.504254 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:54.504319 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.504410 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.531764 (+ 27354us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3768,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":39,"raft.queue_time_us":166825,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":43583744,"thread_start_us":962472,"threads_started":2,"wal-append.queue_time_us":550073}
W20260709 12:26:54.532596 11340 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.006s	sys 0.016s
W20260709 12:26:54.525978 11138 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.003s	sys 0.019s
W20260709 12:26:54.576886  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.010s	sys 0.015s
I20260709 12:26:54.499925 11374 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:54.604666  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.010s	sys 0.015s
I20260709 12:26:54.607483  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:54.612425  7319 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 12:26:54.613204 11181 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.010s	sys 0.027s
W20260709 12:26:54.614251 11181 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.010s	sys 0.027s
I20260709 12:26:54.500238 11328 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.568465 11291 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:54.616148  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.014s	sys 0.022s
W20260709 12:26:54.627756  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.014s	sys 0.022s
W20260709 12:26:54.576059  6456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.019s	sys 0.024s
W20260709 12:26:54.606050 11138 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.003s	sys 0.020s
W20260709 12:26:54.624408  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 424221us
W20260709 12:26:54.628664  6456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.019s	sys 0.024s
W20260709 12:26:54.688730  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60292.5us
I20260709 12:26:54.689996  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:54.616384 11374 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:54.616770 11025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.011s	sys 0.026s
W20260709 12:26:54.621812  6454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.011s	sys 0.023s
W20260709 12:26:54.691047 11202 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.013s	sys 0.063s
I20260709 12:26:54.627449 11375 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:54.707505  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 434684us
W20260709 12:26:54.702135  6454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.012s	sys 0.023s
W20260709 12:26:54.701390  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 382427us
W20260709 12:26:54.693352 11327 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.017s	sys 0.014s
I20260709 12:26:54.693171 11374 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:54.720786  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 509772us
I20260709 12:26:54.722714  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:54.707844 11202 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.013s	sys 0.063s
W20260709 12:26:54.701694 11025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.011s	sys 0.027s
I20260709 12:26:54.735707  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2461) took 1325 ms. Trace:
W20260709 12:26:54.712045 11327 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.018s	sys 0.015s
W20260709 12:26:54.758339  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68407us
I20260709 12:26:54.692270 11291 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.728163 11374 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:54.747506 11281 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.774914 11281 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:54.776430  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:54.777781 11315 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.167s
I20260709 12:26:54.726254 10619 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:26:54.780153 11291 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:54.798969  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 188447us
I20260709 12:26:54.800166  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:54.849707  6459 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.000s
I20260709 12:26:54.851059  6459 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
W20260709 12:26:54.855777  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55704.1us
I20260709 12:26:54.767524 11326 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:54.704643 11008 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.857551  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:54.725515  6204 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:54.727272 11375 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:54.777697 11281 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:54.771821 11201 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:54.916759  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:55.115437  6308 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:55.048586  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 257805us
I20260709 12:26:54.770427 11220 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:54.870358 11181 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.127180  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:55.127684  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.094863  6303 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:55.091305 11326 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:54.919147 11112 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
W20260709 12:26:55.097389  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 239913us
I20260709 12:26:55.111765 11317 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:55.123329  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.133188 11326 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.144135 11326 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:55.144349  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:55.144974  6385 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:26:55.145948  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:55.146266 11396 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:55.147287 11396 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.152336  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:54.782225 11343 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.131721  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:55.135866  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:54.987535 11393 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.129561 11138 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:55.146581  6379 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:26:55.152963  6383 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:55.122352 11008 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:55.131048  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:55.154083  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:55.122124 11281 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.176856  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:55.167721 11317 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:55.168695  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:55.169375 11408 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.177717  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:55.137961 11149 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:55.208427 11148 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:55.209216 11328 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.212025 11317 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.219224  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:55.130589 11220 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.227784 10619 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:55.190986 11375 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.198966 11343 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.220503 11138 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:55.185477 11201 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.199801 11369 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:55.142510 11181 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.236239 11112 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.288388  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58568.6us
I20260709 12:26:55.255487 11369 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:55.323511  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 187719us
I20260709 12:26:55.329807  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:55.215171 11281 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.253757 10619 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:55.229761 11393 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:55.317340 11395 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:26:55.356498  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:55.365746  7318 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:55.319669 11327 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:55.250946 11149 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.360997  6383 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:26:55.333335 11369 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:55.386677  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:55.334332 11233 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:55.267949 11201 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.249167 11408 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.395153  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2331) took 1483 ms. Trace:
I20260709 12:26:55.360541 11410 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.396979  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:55.397853  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:55.399358  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 222585us
I20260709 12:26:55.371944 11327 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.337086 11148 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:26:55.375505 11025 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:55.393638 11374 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.000s
I20260709 12:26:55.375631 11415 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.377280 11414 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.424237  6272 leader_election.cc:400] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:26:55.366583 11112 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.391377 11396 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:55.393980 11297 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:55.444357  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:55.395728  6384 rpcz_store.cc:276] 0709 12:26:53.911958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.912096 (+   138us) service_pool.cc:224] Handling call
0709 12:26:53.926385 (+ 14289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.500290 (+573905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.500337 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.574941 (+ 74604us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.800332 (+225391us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:54.800367 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.800438 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.394932 (+594494us) raft_consensus.cc:1685] finished
0709 12:26:55.394953 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.395095 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9834,"raft.queue_time_us":39194,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9166,"thread_start_us":733612,"threads_started":2,"wal-append.queue_time_us":623851}
I20260709 12:26:55.445612 11415 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.446074 11382 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.137s
I20260709 12:26:55.447324  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:55.447906  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:55.422353 11327 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:55.401986 11409 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.399559 11233 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:55.510200  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 173386us
I20260709 12:26:55.530239  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:55.481467 11342 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.443017 11374 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.543944  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:55.416805 11138 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.493448 11297 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:55.473189 11025 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.487429 11424 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:55.486799 11414 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.437008 11410 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:55.574584 11394 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:55.575151 11201 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:55.576393  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 2908) took 2198 ms. Trace:
I20260709 12:26:55.576746  6454 rpcz_store.cc:276] 0709 12:26:53.378146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.378296 (+   150us) service_pool.cc:224] Handling call
0709 12:26:53.430920 (+ 52624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.453849 (+ 22929us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:53.913441 (+459592us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.736338 (+822897us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.756236 (+ 19898us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.756418 (+   182us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:54.756467 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.756553 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.576142 (+819589us) raft_consensus.cc:1685] finished
0709 12:26:55.576168 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.576308 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27336,"prepare.queue_time_us":340872,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":70780,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":140,"spinlock_wait_cycles":13991808,"thread_start_us":999737,"threads_started":2,"wal-append.queue_time_us":1406230}
I20260709 12:26:55.578675 11416 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:26:55.545356 11220 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.546103 11187 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:54.745122  6456 rpcz_store.cc:276] 0709 12:26:53.410318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.597529 (+187211us) service_pool.cc:224] Handling call
0709 12:26:53.598011 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.598435 (+   424us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:53.965238 (+366803us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.735196 (+769958us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.735281 (+    85us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:54.735359 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.735463 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.735629 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9861,"prepare.queue_time_us":280259,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":29148,"thread_start_us":1041341,"threads_started":2}
I20260709 12:26:55.561334 11411 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:55.539319 10619 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:55.608917 11391 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.410s	user 0.000s	sys 0.002s
W20260709 12:26:55.609277 11381 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.343s	user 0.000s	sys 0.001s
I20260709 12:26:55.609540 11403 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.609930 11260 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:55.610136 11425 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:26:55.610721 11412 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 12:26:55.610997 11413 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.611894 11397 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.614971 11187 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:55.615526 11315 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:55.605535 11326 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.616716 11436 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.604020 11201 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:55.546826 11291 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:26:55.546249 11342 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.639822  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:55.642072 11402 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.868s	user 0.005s	sys 0.000s
W20260709 12:26:55.643592 11388 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.453s	user 0.000s	sys 0.002s
I20260709 12:26:55.643854 11281 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:55.645157 11374 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:55.645378 11385 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 12:26:55.645463 11393 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.526386 11409 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.676615 11425 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.833573 11409 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:55.833914 11372 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.494s	user 0.012s	sys 0.002s
I20260709 12:26:55.834453 11372 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:55.834556  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 679) took 1035 ms. Trace:
I20260709 12:26:55.835182  6307 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:55.641824 11297 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.815965 11415 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:55.826712 11025 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.835094  7314 rpcz_store.cc:276] 0709 12:26:54.799484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.799607 (+   123us) service_pool.cc:224] Handling call
0709 12:26:54.801915 (+  2308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.855673 (+ 53758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.855715 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.916511 (+ 60796us) log.cc:844] Serialized 26 byte log entry
0709 12:26:55.515690 (+599179us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:55.515769 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.515873 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.834273 (+318400us) raft_consensus.cc:1685] finished
0709 12:26:55.834301 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.834462 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21754,"raft.queue_time_us":147412,"raft.run_cpu_time_us":8387,"raft.run_wall_time_us":113291,"thread_start_us":369166,"threads_started":1,"wal-append.queue_time_us":721333}
I20260709 12:26:55.706501  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:55.837795 11326 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.839498  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:55.707948  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 328465us
I20260709 12:26:55.839944  6204 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:55.648883 11260 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:26:55.712246  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 264982us
I20260709 12:26:55.713380  6305 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:26:55.847988  6305 pending_rounds.cc:85] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:26:55.848340  6305 pending_rounds.cc:107] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:26:55.848649  6305 raft_consensus.cc:2889] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:55.711808 11315 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.610013 11386 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.145s
I20260709 12:26:55.706326 11149 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.711141 11201 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:26:55.780284  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 399365us
W20260709 12:26:55.785223 11426 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:55.700083 11413 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.760188 11397 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.647285 11411 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.907531 11411 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:55.796092 11437 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:26:55.797171 11429 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:26:55.797719  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:55.800362 11186 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.436s	user 0.010s	sys 0.003s
I20260709 12:26:55.658555 11403 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:55.909401  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2481) took 1791 ms. Trace:
I20260709 12:26:55.899645 11201 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:55.909396  6310 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:55.911386 11403 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:55.755607 11328 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:55.912195 11328 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.909922  6453 rpcz_store.cc:276] 0709 12:26:54.117778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.117940 (+   162us) service_pool.cc:224] Handling call
0709 12:26:54.198984 (+ 81044us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.695247 (+496263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.695282 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:55.267321 (+572039us) spinlock_profiling.cc:243] Waited 994 us on lock 0x7b2c0012cca8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:26:55.307256 (+ 39935us) log.cc:844] Serialized 26 byte log entry
0709 12:26:55.478177 (+170921us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:55.478237 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.478343 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.909021 (+430678us) raft_consensus.cc:1685] finished
0709 12:26:55.909052 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.909317 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32661,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":2285952,"thread_start_us":567129,"threads_started":2,"wal-append.queue_time_us":201236}
I20260709 12:26:55.754959 11436 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.909626 11186 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15.
W20260709 12:26:55.821123  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 376861us
I20260709 12:26:55.915874  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:55.915988 11315 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:55.909576 11008 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.717s	user 0.009s	sys 0.000s
I20260709 12:26:55.916967 11442 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:55.918572 11436 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:55.830466  6308 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.714s	user 0.010s	sys 0.002s
I20260709 12:26:55.830720  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:55.634871 11181 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.818058 11112 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.616173 11424 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:55.879388  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:55.901794 11260 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:55.833488 11425 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.934754  6376 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:55.973598  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:55.922542  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43607.5us
W20260709 12:26:56.011710  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95912.1us
I20260709 12:26:56.014288  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:56.012825  6308 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:26:55.968768  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2366) took 2443 ms. Trace:
I20260709 12:26:56.013959  6385 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:56.012169  6302 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:26:55.933460  6300 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:56.016002  6309 rpcz_store.cc:276] 0709 12:26:53.525651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.541633 (+ 15982us) service_pool.cc:224] Handling call
0709 12:26:53.542092 (+   459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.928482 (+386390us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:26:53.928529 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.963687 (+ 35158us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.556059 (+592372us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:54.556124 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.556225 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.644634 (+1088409us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:26:55.968346 (+323712us) raft_consensus.cc:1685] finished
0709 12:26:55.968375 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.968638 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17017,"raft.queue_time_us":39912,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":23737,"spinlock_wait_cycles":11373952,"thread_start_us":934932,"threads_started":2,"wal-append.queue_time_us":1326405}
I20260709 12:26:55.971433 11386 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.916008 11413 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.028368  6378 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:56.022557  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.026347  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 727) took 1171 ms. Trace:
I20260709 12:26:56.032547  6308 rpcz_store.cc:276] 0709 12:26:54.843983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.844117 (+   134us) service_pool.cc:224] Handling call
0709 12:26:56.015544 (+1171427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:56.033298  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.018136  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.036346  6303 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:26:56.023890 11054 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:56.037985  6305 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:56.011777 11424 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.026686 11339 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:55.919566 11328 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.026423 11386 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:56.049048 11369 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.653s	user 0.009s	sys 0.000s
W20260709 12:26:56.030663  6351 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.564s (SENT) [suppressed 1 similar messages]
I20260709 12:26:56.035352 11260 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:56.049970  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.049847 11370 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:56.051169 11399 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.929s	user 0.004s	sys 0.009s
I20260709 12:26:56.054988  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2490) took 1287 ms. Trace:
W20260709 12:26:56.059455 11327 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
I20260709 12:26:56.067827  6451 rpcz_store.cc:276] 0709 12:26:54.768002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.768126 (+   124us) service_pool.cc:224] Handling call
0709 12:26:54.775807 (+  7681us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.783571 (+  7764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.783603 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:54.783640 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:54.783673 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.054617 (+1270944us) spinlock_profiling.cc:243] Waited 7.57 ms on lock 0x7b54001e14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:56.054701 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.054901 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17419008}
I20260709 12:26:56.061655  6453 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:26:56.024029 11397 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.049292 11424 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:56.071557  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.036926 11054 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:56.073846  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.066656  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:55.969112 11201 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.068725  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.082314  6454 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:56.069214  6456 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:26:56.080142  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.064750 11413 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:56.066578 11291 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:56.092926  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:56.066545 11386 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.098910  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.099632  6305 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:56.098065  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.102607  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.104082 11341 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:56.104606 11341 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:56.105049 11341 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:26:56.109906  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.110934  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:56.112007  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.112936  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.084178 11185 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:56.136480  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:56.138165  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.066745 11369 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.074402 11327 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:56.146586  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50662.4us
I20260709 12:26:56.147423  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:56.149158  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118501us
W20260709 12:26:56.149343  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 987952us
I20260709 12:26:56.153101  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.154381  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:56.154808  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.157593  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:56.157881  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.161242  6308 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:56.163955  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.097522 11291 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.142812 11185 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:56.175863  6309 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:56.176235  7319 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:56.176050  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.179111  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.180680  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.179728  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:56.190593  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.193526  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.194345  6303 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:56.139242 11054 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.167989 11413 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:56.199642  6457 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:26:56.207186  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.207633 11185 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.209427  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.210233  6385 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:56.202245  6308 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:56.216249  6308 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:56.217120  6308 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.218003  6308 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:56.226886  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:56.230391  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.243314 11201 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.229938  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:56.239172 11413 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.246127 11413 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.226789  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.246869  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.247361  6302 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:56.247457  6304 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:56.249620 11342 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:56.250159 11342 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.230007  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.230464  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.251780 11408 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:56.253207  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.233290 11369 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.242074 11327 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:56.257822 11187 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.258335 11187 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.260432 11187 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:56.229346 11414 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.261781  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:56.262828 11377 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.558s	user 0.007s	sys 0.002s
W20260709 12:26:56.264492 11387 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.538s	user 0.002s	sys 0.007s
I20260709 12:26:56.226876 11291 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:56.272706  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.275556  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.276242  6301 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:26:56.219676 11418 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:26:56.231771  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.220269  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.284687 11341 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.286108  6458 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:26:56.288936  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.289770  6309 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:56.290474  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.231377  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.238867  6310 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:56.238207  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 731) took 1360 ms. Trace:
I20260709 12:26:56.294282  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:56.294926  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.296006  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.397782  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:56.626204 10896 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.346747 11341 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:56.640468  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.296242  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.639355  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.641261  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:56.639569  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 345393us
I20260709 12:26:56.627976  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.292395  6307 rpcz_store.cc:276] 0709 12:26:55.087041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.113141 (+ 26100us) service_pool.cc:224] Handling call
0709 12:26:56.238034 (+1124893us) spinlock_profiling.cc:243] Waited 660 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:56.238147 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1518928768}
I20260709 12:26:56.575709 11396 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:26:56.296903 11342 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.297402  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.617177  6302 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:56.640345  6303 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:56.633639  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:56.351289  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.298277  6304 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:26:56.300174  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.692480  6381 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
W20260709 12:26:56.405702  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 236738us
I20260709 12:26:56.639335  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 680) took 1839 ms. Trace:
I20260709 12:26:56.647917  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:56.649971  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.681674  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.655452  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.656324  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:56.296612 11380 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.555s	user 0.004s	sys 0.000s
W20260709 12:26:56.694954  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48300us
W20260709 12:26:56.682726  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.648453 11383 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:56.235441 11233 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:56.698894 11424 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.601s	user 0.002s	sys 0.010s
I20260709 12:26:56.699383 11383 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:56.699937 11383 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.700019 11403 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.700532 11403 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.701821  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:56.703300  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 549567us
I20260709 12:26:56.705670  6301 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:56.639534  7287 connection.cc:256] RPC call timeout handler was delayed by -1.097729124000125s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1776s for a response
I20260709 12:26:56.647210  6308 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:26:56.706887  6307 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:56.697710  6455 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:26:56.654632 10896 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:56.234334 11384 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.531s	user 0.012s	sys 0.021s
I20260709 12:26:56.712136  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.713294 11408 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.713555 11413 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:26:56.714076 11404 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.726420 11408 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.719666  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.727785  6304 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:26:56.727569 11404 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.728927  6304 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:56.729288  6304 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:56.729694 11408 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:56.715366 10896 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:56.715801 11375 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:26:56.715688  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.733932  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.712188  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.737262  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.730784 11404 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.693394  7317 rpcz_store.cc:276] 0709 12:26:54.799842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.852647 (+ 52805us) service_pool.cc:224] Handling call
0709 12:26:55.088748 (+236101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.268021 (+179273us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:55.318061 (+ 50040us) log.cc:844] Serialized 14 byte log entry
0709 12:26:55.443802 (+125741us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.630937 (+1187135us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b180023c020. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:26:56.631069 (+   132us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:56.631116 (+    47us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:56.631164 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.632137 (+   973us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b5400567554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:56.632177 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.639280 (+  7103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34105,"prepare.queue_time_us":59184,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":20175,"raft.queue_time_us":2388014,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":197,"spinlock_wait_cycles":419516032,"thread_start_us":118825,"threads_started":1,"wal-append.queue_time_us":329635}
I20260709 12:26:56.749248  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:56.749652  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 327758us
I20260709 12:26:56.730049  6304 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.751019  6304 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.751754  6204 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 8, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 8 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:56.682377 11414 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.757905  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.759056  6305 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:56.731043 11408 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:56.760684  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:56.765014  7314 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:26:56.717298 11410 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.748440 11340 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:26:56.771535  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.771566  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:56.772775  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55659.5us
I20260709 12:26:56.773741  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:56.745759 11403 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.774415  7317 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:56.756729 11453 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.699422 11424 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.755992 11368 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:56.784785  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:56.785676  6385 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:26:56.756158 11327 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:56.779098  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.764775 11408 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:56.786679  6307 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:56.789991  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.790108  6303 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:26:56.771525 11340 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:56.774925 10982 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.778787 11187 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.778998 11416 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:56.801304  6309 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:56.780578 11461 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:56.803013  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.805346  6303 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:26:56.807811  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.776006 11414 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.798559 11342 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:56.823606  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:56.814297 11389 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 2.034s	user 0.006s	sys 0.005s
I20260709 12:26:56.797199 11327 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.801013 10982 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.802460 11187 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.818665  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.819571  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.836683  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.838042  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:26:56.825145 11460 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.193s	user 0.002s	sys 0.000s
I20260709 12:26:56.839982  6377 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:56.841197 11327 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.820292  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.843490  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.843804  6301 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:26:56.836627  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.825210  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.828884  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:56.831492  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:56.821614 11260 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.755s	user 0.010s	sys 0.002s
I20260709 12:26:56.833977  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:56.849699  6307 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:56.851042  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:56.852003  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:56.852778  6450 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:26:56.855719  6310 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:26:56.900700  6452 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:26:56.904400  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:56.853411 11413 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:56.825299 11461 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.788309 11453 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.791241 11368 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.813120 11416 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.899498 11340 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.905944 11342 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.918418  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 174835us
W20260709 12:26:56.920593  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 211701us
I20260709 12:26:56.832535 11408 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.911940 11345 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:26:56.923794 11457 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:26:56.947853  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:56.949736  7319 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:56.951633 11383 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.953758 11008 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:56.926805 11327 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.954771 11327 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:56.957067 11327 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.954182 11383 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:56.926630 11342 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:56.973491 11342 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:56.928257 11345 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.974504 11345 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.978026 11008 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:56.979796 11187 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:56.927731 11340 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:56.991365 11414 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:56.928103 11297 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:26:56.995877 10982 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:56.950018 11408 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.004359 11416 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.007593 11368 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:57.015163 11465 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:26:57.027029 11470 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:57.027944 11470 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.032066 11461 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.034551 11453 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.035223 11467 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:57.035939 11467 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:57.045096  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78655.5us
I20260709 12:26:57.064705  6304 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:26:57.066205  6304 pending_rounds.cc:85] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:26:57.066721  6304 pending_rounds.cc:107] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:26:57.067113  6304 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.070307  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:26:57.084514  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.110968  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.111788  7319 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:57.115947  7332 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:26:57.120013  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.122812 11343 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:57.123890  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.130196  6204 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:57.132103  6204 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:57.102159 11467 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.137955  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:57.133944  6204 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 9, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:57.103659 11404 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:26:57.139456  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.111418 11470 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.147571  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:57.148365  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.151023  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 300372us
I20260709 12:26:57.151724  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.152768  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.153465  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.161190  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:57.164062  6451 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:57.154723 11369 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:57.175690  7329 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:57.177954  7332 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:57.162724 11414 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:26:57.186574  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 238799us
I20260709 12:26:57.154425 11221 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:26:57.189347  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.191176  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.194769  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.196676  6379 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:57.179509 11054 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.201205  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.204187  7329 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:57.218628  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.220933 11419 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:57.222630 11419 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:57.222728  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:57.223079 11419 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:57.224792  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.228087  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.229064  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.230052  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.199214 11414 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:57.233163  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:57.199576 10619 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:57.206067 11444 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:57.247571  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.252602  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.226835 11260 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:57.264113  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.264487  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:57.265106  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.265548  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.265910  7319 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:57.262825  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.253717  7329 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:26:57.254758  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:57.252377 11418 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:57.208920 11040 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:57.270713 11418 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:57.271325 11418 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.259725  6309 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:26:57.271446  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:57.259946  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.261667  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.229391 11469 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:57.270828  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.259727  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.262785  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:57.277311  6310 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:57.242421 11299 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:57.276846 11479 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:26:57.280607  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.281975  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.296442  6304 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:57.304920  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.282187  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:57.310268  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:57.321280  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.259799 11438 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.325733  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.325799  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.327320  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.327685  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.328289  6383 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:26:57.339761 11345 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:57.341017 11480 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:57.343170 11483 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:57.343751 11483 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:57.344200 11483 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:57.344563 11444 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:57.345232 11444 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.346674  7319 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:57.347389 11444 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.348379  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.349869  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:57.351999 11345 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:57.352165  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 674) took 3222 ms (client timeout 1776 ms). Trace:
W20260709 12:26:57.352639  6376 rpcz_store.cc:269] 0709 12:26:54.129985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.130183 (+   198us) service_pool.cc:224] Handling call
0709 12:26:57.351973 (+3221790us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:57.352069 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3914892672}
I20260709 12:26:57.354148  6376 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:57.249279 10619 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.356402 10619 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:57.357457 11481 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:57.357729 11413 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:57.359246 11413 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:57.359889 11343 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:57.363813  6384 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:57.364404  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2726) took 1065 ms. Trace:
I20260709 12:26:57.364697  6384 rpcz_store.cc:276] 0709 12:26:56.299415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.301270 (+  1855us) service_pool.cc:224] Handling call
0709 12:26:57.363444 (+1062174us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:26:57.363642 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.364185 (+   543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.364353 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35629312}
I20260709 12:26:57.375998 11427 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:57.376255 11472 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:26:57.376452 11427 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:57.377645 11427 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.378561 11438 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.380026  6456 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:57.381759 11438 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:57.382644  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 55482.4us
I20260709 12:26:57.382735 11054 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.383395 11054 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:57.384430  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:57.385654 11054 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:57.385774  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:57.386330 11054 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.387046 11054 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:57.389367  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.389396 11054 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:57.391166  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.379190 11396 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:57.406255  6377 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:57.327971  6378 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 10.
I20260709 12:26:57.270038 11419 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:57.402364  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:57.392359  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2193) took 1096 ms. Trace:
I20260709 12:26:57.413241 11299 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:57.413658 11299 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.413923  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.414017  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.414017 11299 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.415002  7314 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:26:57.415956  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:57.417452  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:57.418424 11392 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:57.419751 11392 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:57.420223 11392 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:57.421067 11392 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.411571  6302 rpcz_store.cc:276] 0709 12:26:56.296066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.712264 (+416198us) service_pool.cc:224] Handling call
0709 12:26:56.721908 (+  9644us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:56.762715 (+ 40807us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:57.188629 (+425914us) log.cc:844] Serialized 14 byte log entry
0709 12:26:57.391977 (+203348us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:57.392038 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:57.392104 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.392177 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.392299 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":308846,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":13852,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":141,"thread_start_us":573308,"threads_started":2}
I20260709 12:26:57.422703 11421 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:57.422360 11299 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.423174 11421 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:57.424201 11421 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.427423  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.428320  6305 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:57.425602 11421 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.430172  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:57.432020  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:57.425767 11453 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.412166 11415 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:57.435477 11453 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.435771 11415 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:57.426441  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:57.439086  7314 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.452443 11453 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.454873  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.468364  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:57.469074  6456 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.469925  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.470746  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:57.471427  6384 raft_consensus.cc:606] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.475726  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73423.8us
I20260709 12:26:57.481953  6376 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:57.486654  7314 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:57.425347 11299 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.494231  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.409361 11396 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:26:57.478638 11462 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.747s	user 0.004s	sys 0.000s
I20260709 12:26:57.507882  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.508392  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.510744  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.511554  6384 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:57.507091  6204 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 13, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 13 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:26:57.519297  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64534.6us
W20260709 12:26:57.525084  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45516.9us
I20260709 12:26:57.489621 11328 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:26:57.527895  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.529295  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.529553  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:26:57.503644 11408 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.005s
I20260709 12:26:57.533267  6376 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:57.542948  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.545298  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.546232  6300 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:57.515734 11427 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.549101  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:26:57.552510  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.563364  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:57.519218 11424 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:57.564383  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.565117  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 405837us
I20260709 12:26:57.556499  6384 raft_consensus.cc:2376] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15. Re-sending same reply.
I20260709 12:26:57.580819  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.585269  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.585945  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.540213 11383 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:57.588467  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.590126  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.531538 11328 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:57.594254  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.596297  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:57.597093  6384 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:57.529983 11423 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:26:57.598802 11414 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:57.599634 11469 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:57.568459 11421 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:57.600761 11421 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.604966  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.600172  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.603243 11421 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.584388 11008 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:26:57.606846  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.568234 11408 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.610355 11423 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:57.612994  6378 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:57.593509 11260 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:57.605973 11375 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:57.637989 11423 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:57.637996 11408 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.639421 11423 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.641031 11408 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:57.641037 11423 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.608830 11328 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.596014 11415 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:57.621706 11299 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:57.673979  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 363819us
I20260709 12:26:57.674692  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.678568  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:57.672551  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.686697  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.688158  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.689003  6376 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:57.689342  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.690264  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:57.690313  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.650779 11260 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:57.705518 11414 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:26:57.705219 11260 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:57.707672  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.694041  6384 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:26:57.697712  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.717270  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.718741  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.720444  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.721478  6384 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:57.646713 11479 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:26:57.629606 11469 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:26:57.659158 11421 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:57.723397 11421 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:57.725477 11414 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:57.725823 11421 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:57.726718  6451 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:57.727536  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:57.733134 11414 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:26:57.733651 11414 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:57.734150 11414 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:26:57.693517  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86672us
I20260709 12:26:57.699137  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.672972 11444 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:26:57.683264 11339 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:57.762914  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:57.750299  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:57.800526  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:57.803188  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.820616  6454 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:57.699337 11408 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:57.825373  6451 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:57.691293 11485 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:57.678190 11415 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
W20260709 12:26:57.831903  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 159426us
I20260709 12:26:57.847319  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.852389  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:57.840243  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75159.4us
I20260709 12:26:57.863826  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:26:57.848537  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 154398us
I20260709 12:26:57.872282  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.824389  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81010.4us
I20260709 12:26:57.873441  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.875087  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.876431  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:57.880369  6452 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:57.872975  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:57.911355  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:57.849718 11383 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:57.927667  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55439.5us
I20260709 12:26:57.885422  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.930836  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57931.7us
I20260709 12:26:57.951224  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:26:57.937547  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85314.3us
W20260709 12:26:57.952412  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67068.1us
I20260709 12:26:57.953857  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:57.902010 11483 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.541s	user 0.005s	sys 0.007s
I20260709 12:26:57.955121  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.957917  6456 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:26:57.965039 11413 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:57.951243  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.966923  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:57.967368  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:57.968063 11413 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:57.823544 11444 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:58.009915  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.960597 11477 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:57.960476 11410 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:58.033025 11493 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:26:58.074615  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:58.120369  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 147164us
I20260709 12:26:57.970412 11483 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:57.959404 11512 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:57.960036 11438 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:58.087139  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 119886us
I20260709 12:26:58.141620  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:57.974649 11341 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:58.142544 11495 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.293s	user 0.001s	sys 0.001s
I20260709 12:26:58.143097 11341 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:26:58.143715 11341 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:58.143985  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.144593 11341 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:58.145603 11491 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:26:58.146521 11419 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:57.951072 11343 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:57.929755 11040 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:57.931529 11485 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:57.898373 11408 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:26:58.034363 11467 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.139842  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.151176  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:58.022891 11297 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:58.092693  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 165552us
I20260709 12:26:58.155673  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.155982  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:58.097290  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.089629 11370 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.134837  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:26:58.141074 11490 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
W20260709 12:26:58.165548  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.108198 11138 raft_consensus.cc:2764] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:26:58.202844  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.242161  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:58.243052  6308 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15. Re-sending same reply.
I20260709 12:26:58.125614 11424 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:58.250507  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:58.253450  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 282888us
I20260709 12:26:58.256525  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.259085  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.270553  6452 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:26:58.217710 11423 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.563s	user 0.007s	sys 0.006s
I20260709 12:26:58.261828  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.274216  6307 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:26:58.272344  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.275377  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.276185  6309 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:26:58.279676  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.248471 11469 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:58.238147 11383 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:58.292847  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 591790us
W20260709 12:26:58.311056  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 359857us
I20260709 12:26:58.293877  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:58.288545  6381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.004s	sys 0.016s
I20260709 12:26:58.238749 11485 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:58.126227 11414 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.310199  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.314340  6304 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:58.303678  6204 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 12, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 12 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.312573  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.273027 11370 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:58.325743  6300 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:58.333016  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:58.294479 11220 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.006s	sys 0.013s
I20260709 12:26:58.281646 11415 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.192487 11512 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.341298  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.350455  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:26:58.313268  6381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.004s	sys 0.016s
I20260709 12:26:58.347815  6204 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:58.349666  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70090.5us
I20260709 12:26:58.349877  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.351125  6458 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:58.352274  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:58.356900  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2360) took 1162 ms. Trace:
I20260709 12:26:58.357626  6300 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:58.311321 11469 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:58.361187  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:58.367992  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 210260us
I20260709 12:26:58.367161  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2358) took 1173 ms. Trace:
W20260709 12:26:58.348081 11220 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.007s	sys 0.013s
I20260709 12:26:58.357834  6381 rpcz_store.cc:276] 0709 12:26:57.194747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.194906 (+   159us) service_pool.cc:224] Handling call
0709 12:26:57.201700 (+  6794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:57.201990 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:26:57.567492 (+365502us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.356525 (+789033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:58.356582 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:58.356635 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.356706 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.356839 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":548553,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":61,"thread_start_us":1137486,"threads_started":2}
I20260709 12:26:58.326120 11404 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:58.343770 11424 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.305013 11138 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:26:58.375061  6204 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 11, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 11 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.369591  6385 rpcz_store.cc:276] 0709 12:26:57.194002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.194131 (+   129us) service_pool.cc:224] Handling call
0709 12:26:57.216124 (+ 21993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:57.216355 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.216380 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:57.232963 (+ 16583us) log.cc:844] Serialized 26 byte log entry
0709 12:26:57.517671 (+284708us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:57.517708 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.517782 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.366448 (+848666us) raft_consensus.cc:1685] finished
0709 12:26:58.366476 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.367086 (+   610us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":256001,"threads_started":1,"wal-append.queue_time_us":406321}
I20260709 12:26:58.385937  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.406136  6308 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:58.358736 11054 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:58.336480 11343 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:26:58.370206 11469 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.416742  6204 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 11, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 11 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.331576 11485 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.411120 11040 raft_consensus.cc:993] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:26:58.421002 11529 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:58.435127 11418 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.163s	user 0.006s	sys 0.007s
I20260709 12:26:58.435669 11418 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.437915 11418 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:58.438524 11492 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:58.439221 11492 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.441434 11492 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.443919 11524 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:58.445562 11221 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:26:58.431239 11515 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:26:58.363590 11415 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.449769  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.451586  6204 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 12, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 12 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.459129  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:58.460683  6204 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 12, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:26:58.463109  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74872.6us
I20260709 12:26:58.366300 11520 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 12:26:58.471979  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.482173  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.484148  7314 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:26:58.488762  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:26:58.497227  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.476238  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.451041 11467 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.525415  6304 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:26:58.526443  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:58.401513 11404 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.529150  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.519794  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 12, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 12 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.464515 11524 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:58.543594  6308 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:26:58.451290 11461 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.074s
I20260709 12:26:58.511888 11423 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:26:58.527149 11503 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.391s	user 0.001s	sys 0.000s
I20260709 12:26:58.552721 11531 raft_consensus.cc:1081] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Signalling peer 4e0e521cb08c420f981a29576eee90a6 to start an election
I20260709 12:26:58.519165 11240 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.240s
I20260709 12:26:58.565693 11404 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.566663  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:58.567343  6304 raft_consensus.cc:2376] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15. Re-sending same reply.
W20260709 12:26:58.567427 11523 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:58.571225  6456 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:58.563153 11524 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:58.573751 11496 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
W20260709 12:26:58.574837 11339 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.762s	user 0.001s	sys 0.010s
I20260709 12:26:58.575407 11339 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:58.553782  6204 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.576718  6204 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 10 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.577894 11492 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.526363 11475 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:58.512518 11444 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.578406 11492 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.579210 11444 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.579663 11444 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.581809 11444 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:58.539464 11497 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.446s	user 0.001s	sys 0.001s
I20260709 12:26:58.583477  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2423) took 1157 ms. Trace:
I20260709 12:26:58.583838  6310 rpcz_store.cc:276] 0709 12:26:57.425723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.426041 (+   318us) service_pool.cc:224] Handling call
0709 12:26:57.426400 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:57.426793 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.426827 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:57.427519 (+   692us) log.cc:844] Serialized 26 byte log entry
0709 12:26:57.682434 (+254915us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:57.691445 (+  9011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.691522 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.583275 (+891753us) raft_consensus.cc:1685] finished
0709 12:26:58.583298 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.583419 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2927,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"thread_start_us":254422,"threads_started":1,"wal-append.queue_time_us":500635}
I20260709 12:26:58.519220 11415 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:58.562791  6206 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 271) took 36075 ms (client timeout 15000 ms). Trace:
I20260709 12:26:58.585610  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.586807  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.588932  6301 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:58.592808  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 957) took 1367 ms. Trace:
I20260709 12:26:58.593266  7332 rpcz_store.cc:276] 0709 12:26:57.225309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.225447 (+   138us) service_pool.cc:224] Handling call
0709 12:26:57.237631 (+ 12184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:57.490475 (+252844us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.490535 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:57.491191 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 12:26:57.683552 (+192361us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:57.683591 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.247518 (+563927us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b5400439c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:58.247576 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.592467 (+344891us) raft_consensus.cc:1685] finished
0709 12:26:58.592493 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.592740 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10587,"raft.queue_time_us":21558,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":4360,"spinlock_wait_cycles":543569536,"thread_start_us":143866,"threads_started":1,"wal-append.queue_time_us":1076310}
I20260709 12:26:58.595377 11492 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:58.585707  6206 rpcz_store.cc:269] 0709 12:26:22.454297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.465977 (+ 11680us) service_pool.cc:224] Handling call
0709 12:26:24.032409 (+1566432us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:26:58.472259 (+34439850us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:49.670430 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:49.671315 (+   885us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:49.671371 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:26:49.671396 (+    25us) tablet.cc:662] Decoding operations
0709 12:26:49.673291 (+  1895us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:49.673460 (+   169us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:49.673495 (+    35us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:26:49.676080 (+  2585us) tablet.cc:701] Row locks acquired
0709 12:26:49.676150 (+    70us) write_op.cc:260] PREPARE: finished
0709 12:26:49.676351 (+   201us) write_op.cc:270] Start()
0709 12:26:49.676524 (+   173us) write_op.cc:276] Timestamp: P: 1783600009676273 usec, L: 0
0709 12:26:49.676544 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:26:49.677446 (+   902us) log.cc:844] Serialized 7295 byte log entry
0709 12:26:50.126280 (+448834us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b18003ea640. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:26:50.732719 (+606439us) op_driver.cc:464] REPLICATION: finished
0709 12:26:50.976336 (+243617us) write_op.cc:301] APPLY: starting
0709 12:26:50.995283 (+ 18947us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:51.365841 (+370558us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:51.365866 (+    25us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:51.901316 (+535450us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:51.901689 (+   373us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:26:51.901743 (+    54us) write_op.cc:312] APPLY: finished
0709 12:26:51.902198 (+   455us) log.cc:844] Serialized 172 byte log entry
0709 12:26:52.180117 (+277919us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:52.180322 (+   205us) write_op.cc:454] Released schema lock
0709 12:26:52.180389 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:52.298280 (+117891us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:52.299527 (+  1247us) write_op.cc:454] Released schema lock
0709 12:26:52.436050 (+136523us) spinlock_profiling.cc:243] Waited 9.9 ms on lock 0x7b18003ea640. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
Metrics: {"mutex_wait_us":14934,"reported_tablets":49,"spinlock_wait_cycles":578800896,"child_traces":[["op",{"apply.queue_time_us":180533,"apply.run_cpu_time_us":45490,"apply.run_wall_time_us":1314651,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":73927,"prepare.run_cpu_time_us":51232,"prepare.run_wall_time_us":87842,"raft.queue_time_us":284890,"raft.run_cpu_time_us":6526,"raft.run_wall_time_us":150591,"replication_time_us":1035221,"spinlock_wait_cycles":48699904,"thread_start_us":578480,"threads_started":5,"wal-append.queue_time_us":507398,"wal-append.run_cpu_time_us":18071,"wal-append.run_wall_time_us":2435160}]]}
I20260709 12:26:58.599712  6204 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.602577  6300 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
W20260709 12:26:58.604954  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 261141us
I20260709 12:26:58.605168  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.548530 11483 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.634773  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.639936  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:58.640655  7332 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:26:58.617636 11424 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:58.646319  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:58.638424 11396 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.120s	user 0.013s	sys 0.000s
I20260709 12:26:58.636034  6204 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 2 to 8, term changed from 4 to 12, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 12 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.652848  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.653810  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:26:58.653000  6378 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.654913  6378 raft_consensus.cc:614] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:58.654809  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 497511us
I20260709 12:26:58.655652  6379 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
W20260709 12:26:58.644902 11427 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.075s	user 0.007s	sys 0.006s
I20260709 12:26:58.656903  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.623803 11522 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.661659  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:26:58.600682 11423 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.670673  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:58.671324  6307 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:26:58.669680  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.676433  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.677902  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:26:58.680053  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 440410us
I20260709 12:26:58.684008  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.656126 11040 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.684837  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.690187  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:58.690837  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.664937 11492 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:58.693364  6385 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:58.693156  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.695055  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.695200  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:58.696475  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:58.697718  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:58.697991  6302 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:58.698442  6204 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 14, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 14 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.698025  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:58.666231 11414 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.686653 11368 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:58.680940 11519 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:58.704438  7373 heartbeater.cc:646] Failed to heartbeat to 127.5.4.126:35813 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.4.126:35813 timed out after 15.000s (SENT)
I20260709 12:26:58.685437 11339 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:58.662992 11396 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.698721 11492 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.709444 11492 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:58.709854 11396 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:58.710697  6459 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:26:58.710935  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2381) took 1044 ms. Trace:
I20260709 12:26:58.711297  6379 rpcz_store.cc:276] 0709 12:26:57.666908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.718437 (+ 51529us) service_pool.cc:224] Handling call
0709 12:26:58.710769 (+992332us) spinlock_profiling.cc:243] Waited 936 ms on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:58.710878 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2153087360}
I20260709 12:26:58.686529 11547 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.712687 11547 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.715051 11547 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:58.716061 11328 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.033s	user 0.010s	sys 0.001s
I20260709 12:26:58.716607  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58396 (request call id 38) took 1118 ms. Trace:
I20260709 12:26:58.716485 11328 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.717054  6378 rpcz_store.cc:276] 0709 12:26:57.597815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.607199 (+  9384us) service_pool.cc:224] Handling call
0709 12:26:58.716386 (+1109187us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:26:58.716512 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333896576}
I20260709 12:26:58.692176 11483 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:58.657785 11485 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.719885 11485 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.722218 11415 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:58.725086 11415 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:58.726094 11415 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.729113  6308 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:58.730566  6302 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:26:58.736526  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.685904 11427 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:58.686252 11535 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.686192 11240 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:58.742708  6204 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 8, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 8 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:58.745350  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44591.6us
W20260709 12:26:58.735265 11392 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.314s	user 0.006s	sys 0.010s
I20260709 12:26:58.757987 11522 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:58.758287 11392 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.759480  6450 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:58.760514 11392 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:58.752749  6204 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:58.761767  6204 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 9, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.762953  6378 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402a1967982740658c70049fd62be44e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:58.763624  6378 raft_consensus.cc:606] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.763991  6378 raft_consensus.cc:614] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:58.764022 11328 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:58.769529 11469 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.770342  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:58.699929 11408 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:58.763358  6204 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 12, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:58.702072 11467 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:58.780380  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.752664 11512 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:58.785320  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:58.792335  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.793411  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.744488 11461 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:26:58.818030  6378 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:58.818712  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45375.3us
I20260709 12:26:58.818403  6204 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.819856  6380 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
W20260709 12:26:58.820363  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 168535us
I20260709 12:26:58.801952  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:58.699653 11040 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.790179  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.830149  6204 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.826360  6454 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:59648
I20260709 12:26:58.833590  6454 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:58.834057  6454 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.831943  6204 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 10, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 10 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:58.789312 11467 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:58.843318  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148337us
I20260709 12:26:58.846571 11467 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:58.846524  6204 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:58.846992  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.848886  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.848801  6204 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 10 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.850909 11557 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:58.851838  6204 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:58.850425 11475 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:58.868080  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 78007.5us
I20260709 12:26:58.868911  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.816186 11054 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:58.850312  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.863994  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.750581 11427 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:58.875968  6454 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.794811 11342 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:58.744524 11415 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:58.875293  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.829854 11522 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.912814  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48947.3us
I20260709 12:26:58.853766 11510 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:58.915422  6454 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:58.913867  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 170463us
I20260709 12:26:58.932869  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.934046  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.934545  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:58.874081 11544 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:58.901206 11427 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:58.901966 11557 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:58.942423  6204 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 14, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 14 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:58.949369  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.919261 11475 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:58.919780 11408 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:58.921849  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:58.922392 11556 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:58.965901 11556 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:58.950588 11557 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.969455 11557 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:58.924906 11414 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:58.969669 10619 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:58.970151 11414 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.970324 10619 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.972350 11414 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.972983 10619 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:58.930542  6380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:58.956866  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:26:58.974583  6380 raft_consensus.cc:606] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:58.975028  6380 raft_consensus.cc:614] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:58.914878 11240 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:58.960929  6204 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.915877 11342 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.901592 11054 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:58.965050 11392 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:58.965198 11545 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:58.955982  6308 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:26:58.950073 11550 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:58.917872 11424 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.006443  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:26:58.943456 11396 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.012192  6204 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 3 to 10, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 10 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:26:59.015815  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 144875us
I20260709 12:26:59.009183 11341 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:26:59.083304  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 156389us
I20260709 12:26:58.977089 11512 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:59.085745  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79426.7us
I20260709 12:26:59.084054  6204 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 15 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:26:58.936350 11522 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.004741 11544 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:59.092303  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.049847 11339 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:26:59.102927 11299 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:26:59.103436 11299 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:59.103856 11342 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:59.104626 11240 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:59.105221 11240 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:59.105690 11240 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:59.105921 11558 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.268s	user 0.000s	sys 0.003s
I20260709 12:26:59.106807 11315 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:59.107340 11315 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:59.108320 11315 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.028538 11392 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:59.104466 11299 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:59.030975 11491 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:59.095434  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.116916 11516 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:59.123592  6204 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 1 to 8, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 8 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:26:59.135983  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.135820  6204 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 13, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 13 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.052150 11423 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.123677  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.139822  6450 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:59.017021 11403 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:26:59.145362  6204 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 4 to 9, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 9 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:59.152612  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.010040 11475 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.099380 11535 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.096462 11546 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.127141 11392 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.125770 11418 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:59.177335  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.179168  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:59.124867 11315 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:59.173158  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:59.173475  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:26:59.183284  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.154134 11509 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.092739 11396 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.053383 11467 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.141469 11521 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:59.187969  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.165535 11533 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:59.194981  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.201076  6456 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:59.201772  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.207625  6457 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:26:59.197670  6382 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:59.194227  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70686.2us
I20260709 12:26:59.203498  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:26:59.220450  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.181663 11469 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.161391 11423 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:26:59.193349 11299 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:26:59.228211  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.193167 11408 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:59.237949  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:59.192942 11444 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.196339 11345 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:59.227267 11554 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:26:59.194481 11564 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:59.201906 11424 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.176538 11546 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.198189 11535 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:26:59.230031 11568 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:26:59.254809 11492 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 12:26:59.270834 11479 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:59.279284 11340 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:59.279320  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:59.297546  6379 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:59.297645 11408 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.283281  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.298354 11408 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.196276 11509 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.300611 11408 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.300961 11509 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:59.290001  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 98626.1us
I20260709 12:26:59.212409 11508 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:26:59.294828  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:59.303546  6310 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:59.304378 11342 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:59.305187  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.305145 11342 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.306284 11408 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:26:59.307078 11408 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:59.307435  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:26:59.307783 11551 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.396s	user 0.002s	sys 0.000s
I20260709 12:26:59.308701 11408 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:26:59.309544 11561 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:26:59.312383  6382 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:59.312806  6382 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:26:59.313710  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.313661  6382 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.314248 11516 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.314692  6379 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:26:59.314810 11516 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.315629  6382 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.316000  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:59.317132 11516 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:59.315522  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.320911  6307 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:26:59.321311  6307 pending_rounds.cc:85] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 8
I20260709 12:26:59.321555  6307 pending_rounds.cc:107] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:26:59.322744  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.325316  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.325611  6310 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:26:59.325945  6304 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:59.321451  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.325264  6307 raft_consensus.cc:2889] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.257103 11040 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.324728  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.296751  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.221568 11533 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.253741 11444 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.338800 11485 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.296924 11340 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:26:59.267633 11424 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.296176 11492 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.505743  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.338160  6382 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:26:59.235663 11544 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:59.462205 11383 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.140s	user 0.012s	sys 0.001s
I20260709 12:26:59.515322  6309 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 7.
W20260709 12:26:59.506958 11415 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.005s
W20260709 12:26:59.511927 11497 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:26:59.437526 11485 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.575734  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.578052  6304 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:26:59.578549  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 261865us
I20260709 12:26:59.507622 11475 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.575981  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:26:59.557376  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47140.6us
W20260709 12:26:59.563779 11567 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:26:59.508201 11522 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:59.518172 11299 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.518102 11518 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.586874  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.562943 11571 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:26:59.605391 11475 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.605219 11571 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.605916 11475 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.607187 11546 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.607710 11571 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.605249  6455 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:59.609169 11475 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.537154 11383 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.610067 11475 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:26:59.610435  6384 raft_consensus.cc:606] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.610818  6384 raft_consensus.cc:614] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:59.523254 10619 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.611737 11492 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.611718 10619 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.615564 10619 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.617614 11424 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.619204 11517 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.567014 11340 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.621079 11340 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:59.597592 11415 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:26:59.623831 11415 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.626153 11415 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:59.598450 11467 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.629799 11533 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.644220 11485 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:59.648133 11572 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:26:59.582651 11565 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.405s
I20260709 12:26:59.679731 11565 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:59.680145 11565 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:59.681262 11565 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:59.682829 11568 consensus_queue.cc:1175] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 12 index: 10 } }
I20260709 12:26:59.650717 11518 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.685588 11518 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.654078 11299 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.687955 11299 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:59.656728  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52678us
I20260709 12:26:59.694021  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.695880  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:26:59.696584  6382 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:59.697072  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:59.698372 11576 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:59.694047  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.699460  6379 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:26:59.700603  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.701272  6379 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 7.
I20260709 12:26:59.702735  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.703403  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:26:59.704722 11565 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.703330 11444 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:26:59.673175 11240 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.567s	user 0.004s	sys 0.007s
I20260709 12:26:59.708555 11240 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:26:59.716295  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.716861  6382 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:26:59.723337  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.724262  6300 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:26:59.676131 11427 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.000s
W20260709 12:26:59.730726  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.742376  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.746408  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:26:59.747399  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.760571 10619 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.761319 10619 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.761886  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:59.762869  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:59.760280  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.762949 11571 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:59.766769  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.766088 11485 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:59.765810 11571 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.768692 11571 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.743566 11444 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.770871 11444 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.773020 11444 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.743322 11240 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:26:59.773932 11553 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.923s	user 0.000s	sys 0.010s
I20260709 12:26:59.763787 10619 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.764397 11040 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:59.764581 11260 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:59.764849  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:59.765132 11577 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:59.779327  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:59.779808  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 175302us
I20260709 12:26:59.726320 11054 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.749959  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.788313  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:26:59.793919  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.790522  6300 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:59.801010  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.779100 11576 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:59.803630  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:59.781468 11423 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:59.783532 11040 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.805295  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:59.805583  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.766551 11427 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.837916  6300 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:26:59.840862  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:26:59.842097  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.839016  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:26:59.843536  6455 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:26:59.842886  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:26:59.847277  7319 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
W20260709 12:26:59.839682  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51433.8us
W20260709 12:26:59.803454 11557 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.767s	user 0.009s	sys 0.003s
I20260709 12:26:59.851830 11368 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:59.785019 11260 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.791217 11424 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:26:59.824205 11054 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:59.858906  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:26:59.836978 11040 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:59.808794 11423 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.836380 11517 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.863503  7317 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:59.862730  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:59.866176  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.840579 11299 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:59.842887  6300 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:26:59.851511 11408 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:59.837142 10619 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.879051 11467 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:26:59.887835  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:26:59.866472 11369 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:26:59.874271 11424 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.885073  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:59.874871 11260 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:59.886271 11576 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:59.894538  6300 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:59.881386 11423 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:59.877705 11054 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.895738 11577 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:59.897539 11427 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:26:59.903549 11424 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:26:59.869825 11557 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:26:59.930989 11583 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
W20260709 12:26:59.905418 11392 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.715s	user 0.013s	sys 0.000s
I20260709 12:26:59.914255 11054 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:59.940829 11584 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:26:59.974792 11538 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.218s	user 0.002s	sys 0.010s
I20260709 12:26:59.976476 11328 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:26:59.977048 11328 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:26:59.977237  6307 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:26:59.922207 11577 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:26:59.978991 11577 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:59.980799  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:26:59.980980 11054 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.982039  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.911615 11576 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:26:59.966681  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.987154  6307 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:26:59.970659 11327 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:59.873199 11571 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:26:59.973131 11414 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:59.976563 11557 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:26:59.949960  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:59.983374 11541 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.222s	user 0.001s	sys 0.007s
I20260709 12:26:59.983588 11054 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:26:59.985011 11342 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:26:59.986142  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.886700 11517 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:26:59.990334  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:26:59.900468 10619 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.012773  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.013929  6307 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:26:59.972945  6300 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:00.015662  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.016287  6300 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:00.017072  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:26:59.993669  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:00.008121  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.008460 11485 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.011932  6308 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:00.024355  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:00.017650  6300 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:00.026126 11240 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:00.026676 11240 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:00.028141 11517 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.027635 11240 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.030642  6308 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.031004  6383 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:59.993243 11327 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:26:59.995573 11427 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.009881 11414 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:00.031886 11328 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.039321  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.045526  6378 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:26:59.990798 11392 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.044876 11587 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:27:00.047899 11327 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.048910 11587 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:00.049319 11587 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:00.050252 11576 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.051585 11328 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:00.051941 11524 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:00.052359 11524 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:00.053244 11524 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.029000 11485 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.030000 11403 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:00.057963  6453 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:00.049484 11327 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.054353 11524 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.062525  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.064781  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.065287  6377 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:00.058267 11403 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.068712 11403 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.069075  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:00.072120  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.075974 11040 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.061023 11587 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.077258 10619 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.078146 11423 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.079201 11392 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.089004  6379 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:27:00.098030  6379 pending_rounds.cc:85] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 9
I20260709 12:27:00.098460  6379 pending_rounds.cc:107] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:27:00.096269 11587 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:00.086189  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.098869  6379 raft_consensus.cc:2889] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.101596  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.091382 11299 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.102242  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.102887  6378 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:00.102736 11299 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.103333 11260 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.104853 11040 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.085971 11368 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:00.105639  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.106688  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.107338 11571 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.108142  6383 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.109179  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.109360  6380 raft_consensus.cc:2368] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12.
I20260709 12:27:00.109992  6383 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:27:00.075974 11424 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.110821  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.112771 11424 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.113242 11423 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.116868  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.117841  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:00.121472  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.114473  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:00.126258  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:00.128283  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.129197  6382 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.132895 11535 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:00.133770  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.134385  7316 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.138461  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.139873  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:00.141484  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:00.143033  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.144686  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:00.146301  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.147040  7316 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:00.105556 11260 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.087007 11556 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.151494  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:00.154469  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.154589  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.156953 11565 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:00.157913 11565 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:00.158318 11577 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.047020 11427 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.104863 11299 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.167466 11492 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.114141 11392 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:27:00.129614 11368 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.129427 11485 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.161556 11342 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:00.202227  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.192387  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58396 (request call id 41) took 1021 ms. Trace:
I20260709 12:27:00.203047  6384 rpcz_store.cc:276] 0709 12:26:59.171286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.173259 (+  1973us) service_pool.cc:224] Handling call
0709 12:27:00.192185 (+1018926us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b54001955d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:00.192299 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985383680}
I20260709 12:27:00.205986  7332 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:00.161170 11423 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:00.114193 11403 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.208631  7318 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:00.177084 11582 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:00.199368 11342 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.178524 11556 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.187019 11392 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:00.182937 11577 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.099910 11414 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.185109 11492 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.224579 11396 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.228309  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.229220  7335 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:27:00.229758 11586 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.367s	user 0.001s	sys 0.000s
I20260709 12:27:00.221380 11392 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:00.231429 11392 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:00.232136  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2435) took 1744 ms (client timeout 1853 ms). Trace:
I20260709 12:27:00.232779 11427 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.233453  6303 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.235263 11392 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:00.232589  6302 rpcz_store.cc:269] 0709 12:26:58.487768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.487876 (+   108us) service_pool.cc:224] Handling call
0709 12:27:00.231974 (+1744098us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b54000b4854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:00.232080 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393017600}
I20260709 12:27:00.243391  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.244072  6453 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:00.246240  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:00.247191 11423 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.247303  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2390) took 1064 ms. Trace:
I20260709 12:27:00.247982  6381 rpcz_store.cc:276] 0709 12:26:59.183065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.220948 (+ 37883us) service_pool.cc:224] Handling call
0709 12:26:59.227692 (+  6744us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.464789 (+237097us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.464824 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:59.464871 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:59.464912 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.246957 (+782045us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b54003e7e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:00.247025 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.247221 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544689920}
I20260709 12:27:00.255492  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2389) took 1077 ms. Trace:
I20260709 12:27:00.256903  7331 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:00.255975  6376 rpcz_store.cc:276] 0709 12:26:59.178442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.180379 (+  1937us) service_pool.cc:224] Handling call
0709 12:26:59.180784 (+   405us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:26:59.443253 (+262469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.443299 (+    46us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:26:59.444309 (+  1010us) log.cc:844] Serialized 104 byte log entry
0709 12:26:59.757726 (+313417us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:26:59.757757 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.757835 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.229098 (+471263us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:00.255177 (+ 26079us) raft_consensus.cc:1685] finished
0709 12:27:00.255197 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.255423 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14116,"raft.queue_time_us":61785,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":2283,"spinlock_wait_cycles":18591488,"thread_start_us":478192,"threads_started":2,"wal-append.queue_time_us":280270}
W20260709 12:27:00.263131  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:00.265741  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.309s	user 0.027s	sys 0.018s
W20260709 12:27:00.266062  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.310s	user 0.027s	sys 0.018s
W20260709 12:27:00.266309  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 1.31223e+06us
I20260709 12:27:00.266891  7329 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:00.267365  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.269630 11575 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:00.219321 11342 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.270071 11575 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:00.247668 11423 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.270792 11423 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.271278  6275 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:00.273350 11556 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.276542  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.245929  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:00.276860  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 109308us
I20260709 12:27:00.275180  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:00.273819  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51206.6us
I20260709 12:27:00.277226  6310 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:00.274623  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.278172  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.280237  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:00.286990  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.290916  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.292744  6385 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.293092  6424 leader_election.cc:400] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12.
I20260709 12:27:00.293386  6424 leader_election.cc:403] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:27:00.293268  6302 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:52532
I20260709 12:27:00.293816  6302 raft_consensus.cc:606] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:00.294054  6302 raft_consensus.cc:614] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:00.274772 11575 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.295280  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.226274 11492 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.295905 11485 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.297828 11415 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.298262 11485 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.299319  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.300127 11415 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:00.297315 11520 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.295419  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.279421 11556 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.302126  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.303406  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.303433  6301 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.305804  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.307235 11415 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:00.308202  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:00.292411 11483 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.308976 11530 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:00.309442 11483 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.310022  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:00.311398  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:00.311439  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.279183 11578 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:27:00.312368 11536 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:00.312297  6302 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:27:00.312736 11578 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:00.313081 11536 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:00.313308 11578 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:00.301180 11512 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.314205 11536 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.315014 11512 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:00.315560 11512 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.319129 11556 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.321373  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.319231  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:00.321820  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.288275 11548 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:00.329854  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.334311  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.323657  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.324478  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.327188  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:00.322696  7331 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:00.341358  6382 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.296459 11240 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:00.348359  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46015.3us
I20260709 12:27:00.343631  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.349507  6308 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:00.344832  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.348279  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.350862  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.351543  6308 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:00.351680  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.352846  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.353396  6304 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.357726  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.358366  7314 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:00.362694  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:00.363624  6300 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:00.365051  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.366626  6300 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.304867 11403 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:00.329921 11492 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:00.368224  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.368500  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.358935  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.374387  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.368203 11555 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.379133 11555 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.381327 11555 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:00.364696 11524 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:00.329926 11424 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.385453 11424 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.330193 11396 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:27:00.314358 11578 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.388548 11578 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.389686  6304 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:00.331163 11530 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.390779 11530 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.391237  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.392882 11530 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.393338  6382 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:00.393682 11587 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:27:00.394115  6304 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:00.394760 11581 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.528s	user 0.002s	sys 0.000s
I20260709 12:27:00.395447 11576 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.396165  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 760) took 1113 ms. Trace:
I20260709 12:27:00.395917 11576 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.398558  6300 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:00.396533  6450 rpcz_store.cc:276] 0709 12:26:59.282433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.282578 (+   145us) service_pool.cc:224] Handling call
0709 12:26:59.282921 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:59.311653 (+ 28732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.311703 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:59.312518 (+   815us) log.cc:844] Serialized 52 byte log entry
0709 12:26:59.573213 (+260695us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:59.573253 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.573320 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.318426 (+745106us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:00.395879 (+ 77453us) raft_consensus.cc:1685] finished
0709 12:27:00.395901 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.396110 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28166,"raft.queue_time_us":29902,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":3613568,"thread_start_us":260232,"threads_started":1,"wal-append.queue_time_us":486884}
I20260709 12:27:00.353580 11240 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.415987  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.417805  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.418843  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.419744  6309 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:27:00.421286  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.365701 11557 raft_consensus.cc:2764] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:27:00.423399  6304 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:00.415990  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.424837  6300 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.407856 11342 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.435439  6451 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:00.435300  6453 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:00.437893 11342 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.437934  6300 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:00.438997  6304 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:00.440289  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:00.431887  6309 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.445263 11345 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:00.445778 11345 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:00.446789 11483 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.446972 11345 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.447906 11485 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.448570 11345 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:00.451143 11423 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.453500 11423 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.460877 11414 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.422942 11240 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.467835  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 746) took 1883 ms. Trace:
I20260709 12:27:00.468286  6303 rpcz_store.cc:276] 0709 12:26:58.584098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.584224 (+   126us) service_pool.cc:224] Handling call
0709 12:27:00.233188 (+1648964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.233993 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.467620 (+233627us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54000b4850. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:00.467770 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3791453440}
I20260709 12:27:00.466810  6309 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:00.476052 11576 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.476730  6307 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:00.477563 11424 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.488188  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.488577  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.489250  7319 raft_consensus.cc:2410] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:27:00.490521  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:00.489270  7318 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.493963  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.494704  6452 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.495855  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.501864  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.502667  6457 raft_consensus.cc:2410] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:27:00.502781  6425 connection.cc:256] RPC call timeout handler was delayed by -1.8594498440002099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:27:00.492820  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.503809  6459 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:00.505259  6301 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:00.509827  6310 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:00.515398  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.516036  6456 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:00.512244 11408 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.546710 11522 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.646833 11522 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:00.648569 11566 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.082s	user 0.000s	sys 0.003s
I20260709 12:27:00.648794 11240 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.649184 11522 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.649479  6300 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:27:00.633049 11555 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.633312 11345 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:00.516707  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:00.652040  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 135428us
I20260709 12:27:00.652391  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.640908  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:00.644438  7318 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:00.547693 11340 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:27:00.653046  6455 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:00.654237  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:00.664177 11576 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:00.651132 11408 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.664803 11576 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:00.665781 11555 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.665911 11576 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.660058 11571 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:00.667542  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.667632 11571 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.668354  6310 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:00.659953 11575 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.660725  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2439) took 1731 ms. Trace:
W20260709 12:27:00.671281  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 396161us
I20260709 12:27:00.672082  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.670924  6305 rpcz_store.cc:276] 0709 12:26:58.929099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.929796 (+   697us) service_pool.cc:224] Handling call
0709 12:26:58.930142 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.943584 (+ 13442us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.943617 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.944118 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.187605 (+243487us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:59.187664 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.187770 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.648003 (+1460233us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:00.650809 (+  2806us) raft_consensus.cc:1685] finished
0709 12:27:00.650831 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.660658 (+  9827us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29415,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"spinlock_wait_cycles":7418112,"thread_start_us":220469,"threads_started":1,"wal-append.queue_time_us":390762}
I20260709 12:27:00.672355 11570 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:00.662222 11557 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:00.673203 11557 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.673025  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.673704 11557 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:00.674013 11575 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.674738 11479 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:00.675447 11479 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:00.675874 11419 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.676437 11419 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.676923 11419 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.676597 11479 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.673039 11570 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.682015 11570 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:00.683212  6457 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:00.684161  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.692826 11537 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:00.693372 11537 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.693785 11537 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.694648 11577 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.695868  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.697315  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.699733  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:00.700981  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:00.708065  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67221.6us
I20260709 12:27:00.709846  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.715313  6452 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:00.712641  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:00.720441  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.722954  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.725730  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.637645 11512 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.730849  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.732515  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.735232  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.732002  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.737010  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:00.710760 11577 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.741171  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.724007 11383 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:00.727636 11483 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:00.717226 11392 raft_consensus.cc:2764] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:27:00.751295  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.724947 11414 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:00.768658 11520 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12.
I20260709 12:27:00.749186 11418 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:00.773303  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:00.773699  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:00.746814 11297 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:00.752648  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:00.760074  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59195.9us
I20260709 12:27:00.760551  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.762221 11240 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:00.742347 11345 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:00.765792 11571 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.754364 11383 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.723361 11369 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:00.774372  6379 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.752430  6450 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.806558  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.783742 11408 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.723801 11556 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.789407  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.793505  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.794291  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.794720  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:00.784652 11577 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.790617  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:00.753968 11327 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:00.754875 11483 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.798283  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:00.771121 11517 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.804467  6307 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:00.807539  6453 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:00.809600  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:00.773852 11418 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:00.770887 11557 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.827955  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.806389 11369 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.812978  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.818400  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.815838 11327 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.835898  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:00.815863 11555 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:00.821532  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.796833 11461 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:00.797405 11297 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:00.806161 11383 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:00.823966 11483 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.792693 11536 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:00.830209  6453 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:00.829277 11418 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:00.833614  6302 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:00.838975  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.839203  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.824265 11421 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:00.848434  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100465us
I20260709 12:27:00.841692 11415 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:00.839960 11575 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:00.850477 11297 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:00.856706 11419 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.858275 11512 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.858567  7317 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:00.837177 11369 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:00.897143  6385 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:00.874279  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.898854  6454 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:00.866117 11570 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.905580 11556 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.905712  6382 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.885686  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.866600 11345 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:00.892079  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52960.3us
W20260709 12:27:00.885295 11606 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:00.903247 11536 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:00.886327  6458 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:27:00.939476  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.928164  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:00.928977  6454 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:00.867369 11557 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:00.932858  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47328.9us
I20260709 12:27:00.906232 11556 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:00.937386  6301 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:00.940856  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.949937  6453 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:00.942291  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:00.941435  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:00.950737  6454 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:00.944116  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.953236  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.953969  6455 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:00.958353  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.961328  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.945223 11544 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:00.940444 11533 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.046756  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.179171  6310 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:01.079504  6452 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:00.996217 11418 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.182418 11418 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.038435 11328 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.029629  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.041541 11340 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:01.100389  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:01.033774  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.973516  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:01.063633  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:01.000528  6454 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
W20260709 12:27:01.194461  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94164.6us
I20260709 12:27:01.196089  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:01.197003  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 223575us
W20260709 12:27:01.198215  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 134672us
I20260709 12:27:01.200518  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.197837  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:00.957026 11556 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:00.954800 11536 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:00.964887 11040 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:00.969347 11555 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:00.969187 11608 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:01.220360 11608 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:01.220556  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:01.221411 11608 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.088800 11327 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.083016 11548 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:01.085981 11568 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:01.222941 11548 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.223150 11568 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.223840 11548 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.224905 11548 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:01.225580 11568 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:00.957533 11403 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:27:00.963007 11260 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:01.226809 11260 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.227196 11260 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:01.227546  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:00.953253 11424 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:01.230810 11424 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.231518 11536 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.233148 11424 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.094578 11485 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:00.969950 11544 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:01.234962 11040 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:01.235240 11544 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:01.235899 11040 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.072481  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:01.084164 11419 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.956292 11404 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:27:01.240075 11340 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:01.240604 11404 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:01.241025 11404 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:01.241237 11340 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:01.241622 11340 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.045142  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:01.212318 11461 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:01.242861 11461 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.243731 11461 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.089978 11570 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.127737  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.054349 11408 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:01.244962 11403 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:01.245425 11403 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:01.246268 11403 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.129755  6379 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:01.137611 11383 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.248409 11423 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.248593  6376 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:01.248953 11423 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.249773 11383 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:00.987848 11415 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:27:01.251019 11415 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:01.202648 11339 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:01.251399 11415 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.252556 11415 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.253959 11415 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:01.140976 11537 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.204360 11418 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:01.255611 11418 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:01.256233 11537 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.256615 11418 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.258661 11340 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.260205 11340 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:01.123745 11533 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.139530 11369 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:01.165968  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 227259us
I20260709 12:27:00.948627 11512 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:01.273197  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.077522  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.180065 11517 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:01.039644 11421 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:01.198087 11240 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:01.207233 11299 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:01.282712 11299 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:01.283900 11421 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:01.284511  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:01.285274  7318 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:01.205971  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:01.205427 11577 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:01.205267 11576 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:01.205662 11510 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:27:01.214934  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.215219  6307 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:01.216643  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:01.219525  6303 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:01.293318  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.214244  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.266691  6275 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:01.267710  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.295778 11576 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:01.296957  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 80384.2us
I20260709 12:27:01.297608 11537 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:01.297969  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:01.251430 11423 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.283839 11299 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:01.289723  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 83832.7us
I20260709 12:27:01.296782 11576 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.287189  6454 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:01.267941  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.298295 11537 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:01.305673 11510 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:01.280181 11555 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.284358 11421 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.327620  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.241981 11404 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.309301  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:01.311753  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:01.204780 11328 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.322841  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.323563  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.332543  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:01.333268  6309 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:01.334120  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.334868  6309 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 9.
I20260709 12:27:01.335633  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.336508  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.336953  6309 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 8: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:01.324483  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.327531  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.306697 11392 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:01.313832 11424 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:01.316825 11582 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.317830 11533 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:01.355437  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148784us
I20260709 12:27:01.365634  7319 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:01.327670 11570 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:27:01.345289 11568 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:01.320892 11383 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:27:01.347885 11240 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:01.367518 11610 raft_consensus.cc:2764] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:27:01.347110 11616 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.383663 11421 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.389885 11517 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.331880 11483 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:01.435858  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60866.7us
I20260709 12:27:01.353164 11602 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:27:01.377347 11568 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:01.447191  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:01.448009  6301 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:01.456957 11556 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.410320 11616 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.461054 11616 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:01.464371 11602 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:01.464826 11602 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:01.465583 11602 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.468490 11403 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:01.487210  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.491500 11610 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:27:01.493453 11621 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:27:01.493661 11383 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:01.494020 11383 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:01.470832 11568 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:01.503293  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:01.504956  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:01.509204  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 111071us
I20260709 12:27:01.516259 11299 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:27:01.518034  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.441260 11517 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:01.521627  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 84065us
I20260709 12:27:01.498842 11602 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:01.525445  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.529031  6454 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:01.407913 11537 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.532213  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.532441  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.537423  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:01.538439  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:01.540066  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.548623  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:01.549410  6379 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:01.558713  6382 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:01.561476  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.564488  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:01.520812 11524 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:01.569324  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.526005 11369 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.454875 11555 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.585549  6304 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:01.585549  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.590376  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:01.596422  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.598048  6454 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:01.598716  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.602074  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.579301 11587 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.607111  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:01.560371 11570 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:27:01.613663  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.586688 11240 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.594957 11040 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.621045 11461 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:01.622838 11328 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:01.625404  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:01.625409  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.609113 11419 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:01.631486  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.631783  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.632122  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.639082  7329 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:01.633494  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.639207  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.597801 11485 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:01.639899  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.596153 11512 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 12:27:01.657405  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43838.9us
W20260709 12:27:01.658130  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.658636  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.568011 11383 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.616823 11557 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.603766 11408 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:27:01.615211 11524 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:01.663341  7318 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:01.614634 11633 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:01.662866  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66509.3us
I20260709 12:27:01.674089  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:01.615761 11587 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.660380 11510 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.642536 11575 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:27:01.661456 11339 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:01.640578 11240 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.646945 11419 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.667685 11632 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:01.663916 11616 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.680791 11423 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:01.659508 11520 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 12:27:01.643424 11040 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.682941 11631 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:01.684473 11383 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:01.707332  6457 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:27:01.690562 11485 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:27:01.714231  6453 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:01.690766 11328 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.693154 11339 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.712944  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.713188  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:01.688279 11483 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:01.733027 11260 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.000s
I20260709 12:27:01.739427  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:01.733716 11260 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.746305  6376 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:01.693596 11510 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.710186 11423 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:01.727671 11339 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:01.761740  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.734803 11600 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.703433 11616 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.747365 11340 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.747812 11641 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.818585  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.812896 11600 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.814873  6459 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:27:01.736572 11328 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.768405 11517 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:01.716265 11631 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.835428  6385 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:01.835636  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:01.836717  6381 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:01.744572 11524 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.837011  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:01.838248  6381 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:01.839696  6381 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:01.843045  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:01.833261  6383 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:01.834940  6382 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:01.845371  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.770185 11556 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:27:01.889031 11522 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.889814 11522 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:01.890903 11638 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.224s	user 0.001s	sys 0.000s
I20260709 12:27:01.892352 11522 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:01.892645 11642 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:27:01.894243 11328 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:01.894833 11040 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:01.896842 11616 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:01.896956 11577 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.005s
I20260709 12:27:01.897634 11577 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:01.899649 11419 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:01.909848 11544 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.000s
I20260709 12:27:01.837035 11423 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.823035 11340 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.823685 11424 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:01.879750 11636 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 12:27:01.933681  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 90713.9us
W20260709 12:27:01.934271  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48936.5us
I20260709 12:27:01.942605 11512 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:01.929092 11328 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.954638 11392 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:01.919902 11571 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.970180 11557 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:01.970173 11328 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.916846 11340 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:01.977687  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.978427  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.922351 11631 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:01.982599  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.983220  6309 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:01.983559  6302 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:01.984579  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:01.985276  7329 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:01.985658  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.986368  6309 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:01.987758  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:01.988885  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:01.989949  6309 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:01.991408 11520 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:27:01.993022  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:01.994397 11634 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:01.994839 11634 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:01.995239 11634 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.000351 11512 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:02.004752  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:02.010717 11340 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:02.011181 11340 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.011581 11340 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:02.024725 11340 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:02.029862  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 184560us
I20260709 12:27:01.967408 11483 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:01.910459 11544 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.032207  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.830825 11641 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:01.969678 11392 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:02.042620  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 377499us
I20260709 12:27:02.043417  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:01.958238 11424 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:01.978250 11587 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.054992 11424 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.056295 11424 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:02.056824 11424 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:02.059957  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 305802us
W20260709 12:27:02.058314  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.066406  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.069259  6379 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:02.075380  6305 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:02.047539 11645 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:02.080665  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.029852 11557 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.082613  6453 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:02.046392 11652 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:02.086241  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:02.091549  6300 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:02.086128  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:02.092228  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:02.092563  6308 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:02.093838  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:02.069433 11260 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.073899 11240 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.076916 11485 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.040307 11483 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:02.105075  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61789us
I20260709 12:27:02.105803  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:02.100657 11652 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.102000 11575 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:02.072674 11634 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.125567  6304 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:02.122444  6308 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:02.133062  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.133905  6454 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:02.136190 11659 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:02.136873 11659 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.138971 11643 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:02.139452 11659 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.141150 11557 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.108891 11510 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.142866 11600 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.143594 11640 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:02.144111 11641 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.151150 11544 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.110736 11392 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.152518 11392 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:02.157407 11403 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.000s
I20260709 12:27:02.159466 11424 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:02.161953 11461 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.520s	user 0.010s	sys 0.000s
I20260709 12:27:02.111276 11485 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:02.163233  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.159637 11483 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:02.164819  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 111252us
W20260709 12:27:02.166944  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61220.6us
I20260709 12:27:02.170850  6452 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:02.113926 11660 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:02.128582 11575 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.142983 11328 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.184753  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.185968  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.186311  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:02.187851  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89812.5us
I20260709 12:27:02.195869  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.197261  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.139475 11643 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:02.207216  6453 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:02.141203 11340 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.212078  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:02.208416  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:02.213137  6459 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:02.214192  6310 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:02.214802  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.214893  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.215590  6378 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:02.169593 11479 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
W20260709 12:27:02.218766 11582 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.863s	user 0.012s	sys 0.000s
I20260709 12:27:02.219044  6382 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:02.219228 11582 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:02.219681 11568 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
I20260709 12:27:02.220064  7318 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:02.220315 11568 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.221726 11485 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:02.227276 11655 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:02.210537 11665 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:27:02.231041 11652 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.220225  6384 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:02.206672 11575 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:02.240260 11548 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.015s	user 0.010s	sys 0.002s
I20260709 12:27:02.245755 11643 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.203377 11557 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:02.247766 11557 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.203649 11641 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.251048 11641 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.253705 11641 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.169409 11658 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:02.269543  6456 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:02.271761 11461 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:02.273307 11461 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:02.273818 11461 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:02.274897 11577 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.275584 11577 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.277348  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:02.278043 11577 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.278735  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.279448  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:02.281715  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66824.2us
I20260709 12:27:02.286480 11421 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:02.289661 11608 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:02.290096 11608 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:02.290592 11327 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.291131 11327 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.291030 11608 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.292991 11634 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.215363 11662 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:27:02.301071  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.279578 11557 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.297966 11327 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.357434  7329 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:02.359397  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:02.360195  7331 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:02.363090  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:02.350144 11602 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.805s	user 0.009s	sys 0.004s
I20260709 12:27:02.368021  6308 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:02.369318  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:02.368600  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:02.375198  6450 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:02.322726 11582 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:02.377009  6300 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:02.300280 11339 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:02.362583 11633 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.004s	sys 0.022s
I20260709 12:27:02.391415  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.389978 11568 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.409693  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.393008  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.414593  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:02.406020  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.421809  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.407503  6381 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:02.423166  6377 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:02.392607  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.405969  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:02.407023 11633 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.004s	sys 0.022s
I20260709 12:27:02.383137 11645 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:02.354827 11461 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.433334  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:02.438685  6456 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2557) took 1663 ms (client timeout 1823 ms). Trace:
W20260709 12:27:02.458519  6456 rpcz_store.cc:269] 0709 12:27:00.775427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.775543 (+   116us) service_pool.cc:224] Handling call
0709 12:27:01.077365 (+301822us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:27:02.438449 (+1361084us) spinlock_profiling.cc:243] Waited 994 ms on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:02.438591 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295579136}
I20260709 12:27:02.439335 11571 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:02.440662 11674 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:02.447815 11536 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:02.449254  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 2986) took 1181 ms. Trace:
W20260709 12:27:02.451516  6348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.823s (SENT) [suppressed 26 similar messages]
W20260709 12:27:02.452989  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60494.7us
I20260709 12:27:02.465088  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:02.453603 11677 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:02.457329  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 321907us
I20260709 12:27:02.468896  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.469336  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.463930  6452 rpcz_store.cc:276] 0709 12:27:01.267521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.267666 (+   145us) service_pool.cc:224] Handling call
0709 12:27:02.449055 (+1181389us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54001c1454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:02.449181 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970535680}
I20260709 12:27:02.472535  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.409802 11669 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:02.468675  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.435182 11568 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:02.389050 11582 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:02.480587  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47319us
I20260709 12:27:02.487963  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:02.437790 11633 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:02.499102  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.500293  6304 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:02.474296 11677 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.463351 11571 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:02.503012  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.503664  6456 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:02.506016  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:02.513278  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61801.8us
I20260709 12:27:02.481010 11671 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:02.510113  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:02.488592 11641 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.515717  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.516883  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:02.643523  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 133437us
I20260709 12:27:02.518049  6380 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:02.549441 11485 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.463742 11645 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:02.550071  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.502084 11556 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:02.646329  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.644074  6451 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:02.481707 11536 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:02.567553 11670 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:27:02.515762 11568 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:02.636119 11419 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:02.575788  6304 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:02.585211  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:02.571939  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:02.581073  6458 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 8.
I20260709 12:27:02.489054 11657 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.085s
W20260709 12:27:02.638036  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.637909  6457 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:02.587821  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:02.664618  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76884.1us
I20260709 12:27:02.574388  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:02.665946  6454 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:02.591686  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:02.481436 11674 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.517125  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:02.668620  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 96767.1us
I20260709 12:27:02.669129  6383 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:02.669579  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:02.647262  6380 raft_consensus.cc:2368] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10.
I20260709 12:27:02.506888 11677 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:02.637032 11664 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:27:02.560704 11671 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.675235 11556 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:02.675719 11556 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.573657 11571 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.676859 11571 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:02.490258 11408 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.671901  6380 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:02.650034 11652 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:02.676043 11556 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.653766 11483 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.649354 11485 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.588899 11461 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:02.572988 11575 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.680667  6383 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:02.691565  6380 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:02.698068 11485 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.698691 11668 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:02.701782 11647 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:02.674141 11327 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:02.684847 11408 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.715307  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.692126 11652 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.652429 11641 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.722721  6453 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:02.714454 11327 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.716657  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.668040 11419 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.717796  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:02.738582 11632 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:02.739018 11632 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:02.739351  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:02.739362 11632 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:02.724917  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.726234 11408 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.726966  6300 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:02.706951 11483 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.744439 11483 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.707224 11647 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:02.715134 11668 consensus_queue.cc:1243] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:02.756340  6457 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:02.756500  6380 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:27:02.758811  6457 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:02.760982  6457 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:02.763260  6457 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:02.765110  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:02.765717  6302 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:02.767561  6424 leader_election.cc:400] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10.
I20260709 12:27:02.767908  6424 leader_election.cc:403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:02.769177 11662 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.702340 11568 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:27:02.774776  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.775449  6459 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:02.721534 11652 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:02.772085  6457 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:02.773490  6303 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:02.732306 11327 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:02.776798 11483 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:02.769398  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:02.771214 11582 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.793478  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:02.762749 11632 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.489719 11669 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.790441  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:02.791322  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:02.683290 11671 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.789177  6310 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:02.793843  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:02.777858 11392 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:27:02.802742  6302 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:02.802333  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:02.807013  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 141881us
I20260709 12:27:02.783535 11645 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.569123 11633 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.746023 11647 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.810915  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:02.779520 11510 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:27:02.780473 11040 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:27:02.811946  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:02.770151 11662 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.824323  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:02.810904 11616 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:02.829973  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:02.795763 11421 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
I20260709 12:27:02.834781  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:02.835866  7319 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:02.818409 11369 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:02.800343 11644 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:27:02.802963 11610 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:27:02.839617  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:02.843390  7319 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:02.812839 11570 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:27:02.831128 11616 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:02.805930 11658 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:27:02.828635 11602 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:27:02.807991 11537 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:27:02.801383 11483 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.843638  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:02.835016 11536 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.832931 11692 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:02.853356 11671 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.854157 11641 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.855610 11662 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.865341 11419 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.855970 11669 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.856585 11616 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.859158 11556 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.869027 11616 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:02.870676 11556 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:02.863500 11674 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.863314 11643 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:02.872165 11327 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.874282 11327 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:02.863662 11537 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:27:02.885381 11681 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.377s	user 0.000s	sys 0.002s
I20260709 12:27:02.874200 11677 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.891309 11652 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:02.892401 11677 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:02.893711 11652 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:02.895733 11676 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.345s	user 0.002s	sys 0.000s
I20260709 12:27:02.897799 11695 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:27:02.898056 11685 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.079s	user 0.002s	sys 0.000s
I20260709 12:27:02.898365 11695 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:02.928694  6309 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:02.947126  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 144988us
W20260709 12:27:02.951100  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 140897us
I20260709 12:27:02.952612  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2483d7801887403e873adc62429fe046"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:39682
I20260709 12:27:02.953526  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:02.954074  6454 raft_consensus.cc:606] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Received request to transfer leadership
I20260709 12:27:02.954972  6454 raft_consensus.cc:614] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:02.960290  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.961551  6455 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2573) took 1939 ms (client timeout 1709 ms). Trace:
W20260709 12:27:02.962095  6455 rpcz_store.cc:269] 0709 12:27:01.022462 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.123476 (+101014us) service_pool.cc:224] Handling call
0709 12:27:02.960754 (+1837278us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b54002c7c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:02.961202 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1855390080}
I20260709 12:27:02.967406  7331 raft_consensus.cc:2368] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:02.969038  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:02.990195  6455 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
W20260709 12:27:03.000464  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47387.2us
I20260709 12:27:03.005298  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:03.017380  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.024855 11536 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:03.054044 11392 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10.
I20260709 12:27:03.045408  6308 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:03.042016  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.091940 11634 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.092502 11644 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.093063 11634 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.093906 11644 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.097397 11582 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:03.097975 11582 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.043452  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:03.041563  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:03.050915  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.040227  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.036026  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.106280  6302 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
W20260709 12:27:03.107730  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 277629us
I20260709 12:27:03.108907 11369 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:03.110554 11536 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:03.051666 11423 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.112490 11423 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.112305 11536 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:02.880266 11632 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.056092 11421 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:03.075117 11678 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.520s	user 0.003s	sys 0.000s
I20260709 12:27:03.115938 11421 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.117995  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.112788 11645 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:03.030450 11483 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.060012 11544 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:03.064271 11299 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:03.046317  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:03.112112 11671 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.066179 11510 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.098415 11582 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:03.129693 11637 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.129624  7317 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:03.098438 11644 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.130944 11637 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.129554 11647 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:03.128032  6310 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:03.132946 11632 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.125384 11299 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:03.134621 11610 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:03.137992 11510 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.143855  7314 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:03.128552 11419 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:27:03.148273  7335 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:03.153183 11556 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:27:03.155827 11671 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.158674 11423 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.161931 11421 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:03.164268 11632 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.167740  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2494) took 1582 ms. Trace:
I20260709 12:27:03.168934 11634 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:03.170964  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.171240 11637 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.174574 11557 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:03.175444 11557 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.177798 11557 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.181155 11582 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.184146  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:03.062093 11537 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
W20260709 12:27:03.199195 11680 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.634s	user 0.005s	sys 0.000s
I20260709 12:27:03.199186 11483 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.200150  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:03.200390 11483 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.201373 11423 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:03.202360 11423 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:03.203655 11483 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.223968  7329 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
W20260709 12:27:03.225517  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84436us
W20260709 12:27:03.233790  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 219645us
I20260709 12:27:03.234742  7332 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:02.899264 11695 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.228649 11671 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:03.252221  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.254973  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.256211 11582 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:03.256696 11339 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.258128  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:03.259963  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.260485  6304 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:27:03.268357 11483 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:03.268944 11483 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.269363 11671 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:03.270017 11671 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.168368  6301 rpcz_store.cc:276] 0709 12:27:01.584949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.585957 (+  1008us) service_pool.cc:224] Handling call
0709 12:27:01.586264 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:01.592055 (+  5791us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:01.592092 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:01.593836 (+  1744us) log.cc:844] Serialized 26 byte log entry
0709 12:27:02.409454 (+815618us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b1800388600. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:02.692792 (+283338us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:02.692848 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.692947 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.897457 (+204510us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:03.167170 (+269713us) raft_consensus.cc:1685] finished
0709 12:27:03.167213 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.167485 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7212,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5682,"spinlock_wait_cycles":13907072,"thread_start_us":283078,"threads_started":1,"wal-append.queue_time_us":1193655}
I20260709 12:27:03.270439 11671 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:03.251072 11632 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.273591 11634 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.274000 11674 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:03.278281 11674 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:03.278887 11626 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.687s	user 0.000s	sys 0.003s
I20260709 12:27:03.278788  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.269429 11483 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:03.280081  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:03.280759  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.281594  7314 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:03.282671 11632 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.283099  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.283540 11632 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.284102  6455 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
W20260709 12:27:03.274502  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 269549us
I20260709 12:27:03.286944  6451 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:03.287228 11632 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.274479 11634 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.290428  6275 leader_election.cc:400] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:03.290832  6275 leader_election.cc:403] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:03.295307 11557 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:03.303788  7329 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:27:03.304370  7329 pending_rounds.cc:85] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 13
I20260709 12:27:03.314810  7329 pending_rounds.cc:107] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:27:03.314716  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.315416  7329 raft_consensus.cc:2889] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:03.316068 11423 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:03.316809 11423 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:03.317952  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:03.318540 11665 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.007s	sys 0.007s
I20260709 12:27:03.320572 11404 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:03.321662 11665 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.008s	sys 0.009s
I20260709 12:27:03.325659 11665 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.325965 11633 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.326555 11665 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.326663 11633 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.328145 11299 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.329566 11299 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:03.331353  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2582) took 1698 ms. Trace:
I20260709 12:27:03.331777  6450 rpcz_store.cc:276] 0709 12:27:01.633056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.639552 (+  6496us) service_pool.cc:224] Handling call
0709 12:27:01.639871 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.375808 (+735937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.330830 (+955022us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54001635d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:03.331152 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690209536}
I20260709 12:27:03.332738 11665 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.332815 11633 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.337680 11690 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.338389 11690 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.339501 11575 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.339908  7314 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:03.343859  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.344497  7317 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:03.345331  6457 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
W20260709 12:27:03.349579  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2577) took 1717 ms (client timeout 2230 ms). Trace:
I20260709 12:27:03.351720  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:03.353031  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:03.354247  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:03.360558  6458 rpcz_store.cc:269] 0709 12:27:01.631709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.631856 (+   147us) service_pool.cc:224] Handling call
0709 12:27:03.348968 (+1717112us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b540009a6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:03.349315 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978063616}
I20260709 12:27:03.355610  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:03.357396  6381 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
W20260709 12:27:03.338212  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 292320us
I20260709 12:27:03.362088  6382 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:03.371739  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50960.1us
I20260709 12:27:03.363350  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:03.379078  7331 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:03.393052 11665 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.393565 11665 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.393921 11665 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:03.394546 11556 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.395287 11556 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.396041  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:03.318338 11423 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.403504 11634 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.370198 11536 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:27:03.380684 11674 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:03.400902  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:03.411234 11556 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.062593  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:03.421350  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 119111us
I20260709 12:27:03.424708 11520 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:03.425241 11520 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:03.430141  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.430883  7317 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:03.430945  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.431421  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.437155  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.439447  6452 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:03.442091  7317 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:03.400315 11671 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.452884  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.419289 11404 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:27:03.454816  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.454380 11575 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:03.448334 11643 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.008s
W20260709 12:27:03.458806  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95691.6us
I20260709 12:27:03.449604 11557 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:03.462543  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:03.448558 11556 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:27:03.459591 11575 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.475827 11575 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.475919 11510 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.464125 11674 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:03.476761 11674 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:03.467602  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:03.467943 11632 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:03.478924 11632 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:03.480018 11632 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:03.482192 11699 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.070s	user 0.002s	sys 0.000s
I20260709 12:27:03.469782 11690 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.470522 11260 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:03.476403 11483 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.466832 11479 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:03.488665  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.461534 11634 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:03.489531  7335 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:03.490125 11634 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:03.491495 11634 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.461789 11299 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:27:03.493713 11299 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:03.494585 11556 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:03.495538  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.495393 11556 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.483748  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.426395 11520 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.485635  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.467680 11639 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:27:03.498498 11536 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.503362  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.502002  6450 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:03.511415  7331 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:03.511302 11536 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.511072 11423 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.514045 11536 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:03.509011 11520 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:03.509776 11557 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:03.513809 11423 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.508180 11556 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:03.518599  6254 auto_rebalancer.cc:893] Removing replace marker failed: Replica 4e0e521cb08c420f981a29576eee90a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:27:03.552366  6254 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:27:03.552863  6254 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:27:03.553292  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.554818 11522 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:03.555260 11522 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.555653 11522 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:03.558753 11695 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:03.563578 11608 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:03.569357  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.573136  7318 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:03.573906 11665 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.576540 11665 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.576884  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.579718  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:03.580607  6378 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:03.522763  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:03.477939 11674 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.536799 11423 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.541659  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2266) took 1562 ms. Trace:
I20260709 12:27:03.532723 11557 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:03.546708  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47292.9us
I20260709 12:27:03.548079 11671 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.583060  6307 rpcz_store.cc:276] 0709 12:27:01.979592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.982244 (+  2652us) service_pool.cc:224] Handling call
0709 12:27:02.940520 (+958276us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182eee 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:27:03.438748 (+498228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.451103 (+ 12355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.451135 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.451187 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:03.451223 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.451290 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.541292 (+ 90002us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400140d50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:03.541440 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1141400704}
I20260709 12:27:03.588809  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.591881  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:03.584930 11536 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:03.599812 11647 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:03.610766 11536 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.611070 11647 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.611382 11671 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.611660 11647 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:03.612201 11671 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.600391 11510 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:03.597290 11423 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.615479 11671 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.602386  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:03.584280 11557 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.605865 11297 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.617354 11665 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:03.611451 11536 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.621528 11665 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.622123 11665 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.616107 11423 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:03.624236  6379 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:03.625500 11297 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.626017 11297 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:03.606509  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 83889.1us
I20260709 12:27:03.632367  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:03.633352  6307 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:03.636168  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:03.637146  6304 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:03.639681 11423 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.640183 11665 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.635545  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:03.641497  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.643133  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.633992  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:03.652231  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:03.607543 11483 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.673110  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:03.674072  7329 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:03.673262  6301 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:03.686376  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.712301  6304 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:03.712644  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:03.687433 11483 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:03.734115  7316 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:03.735961  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.715009 11634 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.752939  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.743443  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:03.744867  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:03.746831  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.689431 11632 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.751242  6203 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:03.759625  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126563us
I20260709 12:27:03.732786 11510 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:03.668546 11522 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:03.765128  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.756417  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:03.779295  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.779621  6310 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:27:03.781605  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.774940 11674 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:03.782179 11632 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.784473 11632 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:03.761268 11633 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:03.787025 11690 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:03.787774 11690 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:03.787604 11633 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.788964 11690 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:03.790627  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.791159  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.792954 11557 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:03.798846  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60482.7us
I20260709 12:27:03.807442  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:03.761777 11634 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.770746 11510 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:03.816709  6304 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:03.808740 11633 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:03.820850 11510 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:03.859839  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.863782 11634 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:03.864895  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.858405 11297 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:03.896674  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:03.910969  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91876.9us
I20260709 12:27:03.912262  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:03.913460 11423 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:03.914127 11423 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:03.914711 11423 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:03.922888 11634 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:03.963073  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.935860 11520 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.935617 11557 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:03.976891  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.977243  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.977316  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.984267 11483 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.985119 11483 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:03.971391 11718 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:27:03.937237 11695 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:03.996894  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112972us
I20260709 12:27:04.011029 11695 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.972446 11634 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.015532 11652 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:04.016386 11632 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.015862 11695 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.017071 11634 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.017143 11632 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:03.996366 11536 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.018713 11643 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:04.008060  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:04.022914  6304 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:27:04.023178  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.029079  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:03.955739 11647 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.044550  6377 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:04.049958 11632 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:04.059087  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.059608  6454 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:04.060637  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54262.1us
I20260709 12:27:04.056298  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:04.006814 11520 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.080246  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:04.080521  6309 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:04.080799 11617 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:04.081127 11383 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:27:04.081730 11383 consensus_queue.cc:1243] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:04.067445 11643 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.089358 11643 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.093297  6203 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.094862 11520 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.067749 11671 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:04.097919  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:04.098645  6309 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:04.098872  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.080742  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:04.104599  6459 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:04.107064  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.103019  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:04.109774  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.117784  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:04.071592 11522 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.122248  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.123343  6455 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:27:04.129719 11345 raft_consensus.cc:2764] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:04.129597  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.129851  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.140789  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.147915  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.125983 11297 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.152770  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.103725 11652 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.167476  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:04.168747  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:04.170030  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 89395.4us
I20260709 12:27:04.172192  6309 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:04.147608 11634 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.169512  6303 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:04.174156  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.153218 11610 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:27:04.159607  6203 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 11, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 11 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:04.193114  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.193881  6303 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:04.195940  6379 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:04.199914  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.205122 11725 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:04.205284  6203 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:04.206029 11725 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:04.264839  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 189510us
I20260709 12:27:04.207098  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:04.215233  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:04.185047 11297 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.274027  6382 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:04.219530  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:04.271660  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 210165us
I20260709 12:27:04.272002  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.276604  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:04.284150  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69669.3us
I20260709 12:27:04.200721 11647 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:04.206933  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.202682 11610 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:04.294770  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.299958  6385 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:04.272275 11608 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.269038 11536 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:04.302717  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:04.307222  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:04.306506  6300 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:04.283797 11404 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:04.314153  7316 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:04.184846 11634 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.316506  6459 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:04.204097 11510 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.526347  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.317885  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.374650  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:04.376312  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.526803  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.526877 11668 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:04.378952  6204 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 265) took 19239 ms (client timeout 15000 ms). Trace:
I20260709 12:27:04.286074 11297 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.527618  6381 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:04.528728  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:04.529430  6452 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:04.529872 11297 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.318287 11404 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.530840 11404 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:04.530572  6204 rpcz_store.cc:269] 0709 12:26:44.964853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.966801 (+  1948us) service_pool.cc:224] Handling call
0709 12:27:04.199642 (+19232841us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:59.916853 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:59.917619 (+   766us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:59.917674 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:26:59.917697 (+    23us) tablet.cc:662] Decoding operations
0709 12:27:00.057584 (+139887us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:00.057684 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:00.057714 (+    30us) tablet.cc:685] Acquiring locks for 36 operations
0709 12:27:00.137162 (+ 79448us) tablet.cc:701] Row locks acquired
0709 12:27:00.137208 (+    46us) write_op.cc:260] PREPARE: finished
0709 12:27:00.137504 (+   296us) write_op.cc:270] Start()
0709 12:27:00.137720 (+   216us) write_op.cc:276] Timestamp: P: 1783600020137338 usec, L: 0
0709 12:27:00.137744 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:00.139129 (+  1385us) log.cc:844] Serialized 13770 byte log entry
0709 12:27:00.762198 (+623069us) op_driver.cc:464] REPLICATION: finished
0709 12:27:00.869894 (+107696us) write_op.cc:301] APPLY: starting
0709 12:27:00.887541 (+ 17647us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:01.183372 (+295831us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:01.183499 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:02.257424 (+1073925us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:02.258217 (+   793us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=72,key_file_lookups=72,delta_file_lookups=0,mrs_lookups=0
0709 12:27:02.258296 (+    79us) write_op.cc:312] APPLY: finished
0709 12:27:02.400207 (+141911us) log.cc:844] Serialized 308 byte log entry
0709 12:27:02.574154 (+173947us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:02.574439 (+   285us) write_op.cc:454] Released schema lock
0709 12:27:02.574562 (+   123us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:02.575030 (+   468us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:02.575059 (+    29us) write_op.cc:454] Released schema lock
0709 12:27:03.192035 (+616976us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
Metrics: {"mutex_wait_us":3403407,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":82562,"apply.run_cpu_time_us":96105,"apply.run_wall_time_us":1729901,"cfile_cache_hit":146,"cfile_cache_hit_bytes":154679,"num_ops":36,"prepare.queue_time_us":146215,"prepare.run_cpu_time_us":23992,"prepare.run_wall_time_us":264060,"raft.queue_time_us":363854,"raft.run_cpu_time_us":12614,"raft.run_wall_time_us":90481,"replication_time_us":624244,"spinlock_wait_cycles":173785472,"thread_start_us":391272,"threads_started":5,"wal-append.queue_time_us":304935,"wal-append.run_cpu_time_us":18138,"wal-append.run_wall_time_us":2880718}]]}
I20260709 12:27:04.527669 11668 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:04.542241  6457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.011s	sys 0.044s
W20260709 12:27:04.542793  6457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.011s	sys 0.044s
I20260709 12:27:04.289706 11575 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.544598 11575 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.545063 11510 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:04.547004 11575 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.538209 11668 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.503899  7331 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:04.504065 11345 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.550951 11345 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.553618 11345 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.554275 11297 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:04.555042 11297 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.557796 11297 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:04.566552  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.568135  6310 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:04.570397  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.571025 11647 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:04.571753 11647 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.572837  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.574088 11647 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.574316  6310 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:04.575031 11602 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.575631 11602 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.575781  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.576706  6300 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 7.
I20260709 12:27:04.577947 11602 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:04.503998 11423 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.589s	user 0.018s	sys 0.003s
I20260709 12:27:04.579810 11423 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.582760 11423 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.317708  7335 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:27:04.506781  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:04.602347  6453 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:04.507126  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.516109 11652 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.603437  6455 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:04.521015  6458 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:04.295926 11633 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.523902  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.524403 11571 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:04.525225 11634 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:04.525740 11719 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.356s	user 0.003s	sys 0.006s
I20260709 12:27:04.609274  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:04.609963  7316 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:04.525789  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.583401 11674 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.584602 11725 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:04.587939 11575 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.614580 11575 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.615456  6304 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:04.616885 11575 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.617096  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:04.589550  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.589957  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.618983  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:04.590794  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.591241  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:04.591632  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.591743  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:04.505136  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:04.376232 11608 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.603506 11652 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.519526  6203 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 9, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 9 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:04.608963  6301 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:27:04.625751  6382 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:04.612818  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:04.613318  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.627444  7335 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:04.613502 11674 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.619431  6376 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
W20260709 12:27:04.600528 11671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.011s	sys 0.005s
W20260709 12:27:04.623008  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 117958us
I20260709 12:27:04.623162  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.624651  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:04.625036  6455 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:04.625385 11633 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.632128  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.634436  6385 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:04.637920  6310 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:04.640872  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:04.645424  6452 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.620930 11571 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:04.659329  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.664259  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:04.636163 11671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.011s	sys 0.006s
I20260709 12:27:04.632723 11725 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.669186  6384 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:04.631266 11297 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:04.670991  7314 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:04.668514  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:04.677865  6382 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:04.663378 11734 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.600893 11610 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.674566  6302 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:04.669809  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:04.693486  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:04.674671 11671 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:04.674060 11634 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.664146 11575 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:27:04.699136  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.675766 11423 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:04.695199 11383 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:04.702157 11297 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:04.703003 11571 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.703924 11575 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:04.708637  7335 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:04.718586 11418 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.009s	sys 0.027s
I20260709 12:27:04.667080 11652 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.727229  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:04.619936 11404 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.725922  7316 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:04.737177  6452 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:04.736954  6379 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:04.678256 11725 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:04.726768 11556 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.007s	sys 0.030s
I20260709 12:27:04.744549  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.741758  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:04.734627 11418 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.010s	sys 0.027s
I20260709 12:27:04.697115 11643 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:27:04.753827  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:04.739506 11728 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:04.745615 11556 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.007s	sys 0.030s
I20260709 12:27:04.757802  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:04.745325 11627 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.747339 11647 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:04.723359 11634 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:04.763134  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 238435us
I20260709 12:27:04.697386 11734 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:04.768559  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.768493  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.735000 11423 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.765827  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:04.773025  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.776245  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.754689 11608 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:04.743263 11575 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.779824  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.782130  6310 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:04.785068  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.788952  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:04.776117 11383 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:04.791867 11383 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:04.792326 11383 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:04.783648 11647 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:04.795233 11732 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 12:27:04.795504 11647 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.797113  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:04.797851  7329 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:04.799290 11575 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:04.803716 11652 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:04.804436 11689 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:27:04.791318  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2438) took 1440 ms (client timeout 1575 ms). Trace:
W20260709 12:27:04.806905  6377 rpcz_store.cc:269] 0709 12:27:03.351034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.351194 (+   160us) service_pool.cc:224] Handling call
0709 12:27:04.791047 (+1439853us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400195ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:04.791206 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540205440}
I20260709 12:27:04.817445  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.779922 11728 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:04.834564  6300 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:04.876427  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70791.7us
I20260709 12:27:04.784590 11608 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.882901  6309 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:04.918220  6275 leader_election.cc:400] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:04.919543  6275 leader_election.cc:403] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:04.771155 11627 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:04.930382  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.778230 11671 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:04.953840  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.946775 11652 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:04.913578 11608 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:04.964021  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 263706us
I20260709 12:27:04.965271  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.967914  6310 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:04.974028  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86076.2us
W20260709 12:27:04.976925  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100820us
I20260709 12:27:04.965178 11571 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:04.976605  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:04.985430  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:04.994979  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.996053  6310 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:27:05.002914  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.990343 11404 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:05.011358  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.016678 11634 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:05.016897 11745 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
W20260709 12:27:05.019114 11715 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.252s	user 0.010s	sys 0.017s
I20260709 12:27:05.000805 11725 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:05.022960  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:05.025506  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 717) took 1574 ms. Trace:
I20260709 12:27:05.025924  7317 rpcz_store.cc:276] 0709 12:27:03.451328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.451558 (+   230us) service_pool.cc:224] Handling call
0709 12:27:03.457674 (+  6116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.485763 (+ 28089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.485851 (+    88us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.512447 (+ 26596us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.591392 (+ 78945us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:03.591449 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.591635 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.018375 (+1426740us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:05.025142 (+  6767us) raft_consensus.cc:1685] finished
0709 12:27:05.025167 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.025406 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3480,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":28640,"spinlock_wait_cycles":63846144,"thread_start_us":78349,"threads_started":1,"wal-append.queue_time_us":156866}
I20260709 12:27:05.027248  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:04.984043 11728 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.028409  6309 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.029515  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:04.998137 11689 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:05.035300  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:05.027602 11734 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:05.036486  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51155.5us
I20260709 12:27:05.037034 11674 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:05.039000 11738 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.122s	user 0.002s	sys 0.000s
W20260709 12:27:05.039512 11722 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.030s	user 0.001s	sys 0.010s
I20260709 12:27:05.039587 11739 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.040277 11739 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.040766  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:05.042013 11716 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.032s	user 0.000s	sys 0.004s
I20260709 12:27:05.042040  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.030376  6309 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.046939  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 814) took 1488 ms. Trace:
I20260709 12:27:05.047428  6450 rpcz_store.cc:276] 0709 12:27:03.558584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.558748 (+   164us) service_pool.cc:224] Handling call
0709 12:27:03.559112 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.559484 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.559518 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.560116 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.721009 (+160893us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:03.721059 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.721127 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.034251 (+1313124us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:05.046658 (+ 12407us) raft_consensus.cc:1685] finished
0709 12:27:05.046692 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.046887 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3456,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":84,"spinlock_wait_cycles":23878400,"thread_start_us":160223,"threads_started":1,"wal-append.queue_time_us":405327}
I20260709 12:27:05.057399  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.054952  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 734) took 1390 ms. Trace:
I20260709 12:27:05.062840  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.063657  6454 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:05.070310  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.060333  7332 rpcz_store.cc:276] 0709 12:27:03.664768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.664968 (+   200us) service_pool.cc:224] Handling call
0709 12:27:03.682207 (+ 17239us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:03.698019 (+ 15812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.698071 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:03.732676 (+ 34605us) log.cc:844] Serialized 52 byte log entry
0709 12:27:03.902340 (+169664us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:03.902595 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.902935 (+   340us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.038793 (+1135858us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:05.046903 (+  8110us) raft_consensus.cc:1685] finished
0709 12:27:05.046926 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.054883 (+  7957us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8355,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":85,"spinlock_wait_cycles":23489792,"thread_start_us":147566,"threads_started":1,"wal-append.queue_time_us":254648}
I20260709 12:27:05.070415  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.072021  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.071513  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.064287  6379 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:05.015209 11741 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:05.075481  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.074558  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.076689  6384 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:05.080596  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:05.083099 11557 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.083338 11608 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.027287 11423 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:05.084569  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.085421  6378 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:05.088114  6378 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.064956 11571 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.097077 11571 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.099412 11571 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.070724 11404 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:05.104295 11404 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.109473  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.110526  6454 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
W20260709 12:27:05.111598 11483 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.126s	user 0.008s	sys 0.009s
I20260709 12:27:05.112097 11483 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.091773  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.092386  6304 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.055116 11689 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:05.094084 11627 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.094370  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.094992  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.095446  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.056759 11510 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:27:05.125021  6454 pending_rounds.cc:85] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 7
I20260709 12:27:05.115067  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:05.129801  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.130252  6384 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.130586  6305 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.119167  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.131449  6452 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:05.132023  6302 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:05.132654  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.114778  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.133733  6382 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:05.129134  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.120684  6379 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.117147  6378 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:05.140647  6454 pending_rounds.cc:107] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:27:05.141074  6454 raft_consensus.cc:2889] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.122257  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:05.143013  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:05.142673  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.172677  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.160410  6381 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:05.183936  7319 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:27:05.162024  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.186082  6382 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:05.187299  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:05.185329  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.114466 11483 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.193066  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.193869  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:05.195427  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:05.198622  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 150961us
I20260709 12:27:05.202704  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.203555  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:05.205020  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:05.212402  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.128666 11632 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:05.201908  6383 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:05.143778 11745 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:05.144311 11739 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:05.223097  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100970us
I20260709 12:27:05.224035  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.224936  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:05.236315  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 140983us
I20260709 12:27:05.237135  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:05.243911  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.198724 11627 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.249588  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:05.246321  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:05.254397  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 488766us
I20260709 12:27:05.222115 11647 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:05.192057 11536 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:27:05.256045  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.255241  6456 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:05.234372 11404 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:05.268409  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.233649 11725 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:05.284138 11725 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.284701 11725 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:05.289675  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52646.2us
I20260709 12:27:05.291966  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.299450  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:05.263022 11610 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.257516 11647 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.310693  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.249213 11674 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:27:05.289119 11634 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:05.245409 11632 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.277879 11483 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:05.269670 11652 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:05.276434 11734 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.309209  6272 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
W20260709 12:27:05.342343  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 216568us
I20260709 12:27:05.265058 11745 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:27:05.304744 11297 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:05.295529 11762 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.267773 11627 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.349464  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.314527 11647 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:05.250787 11689 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.391800  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:05.306751 11747 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:05.392311  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.356722  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.336947 11634 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.304476 11418 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:27:05.388921 11745 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:05.361456 11741 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:05.392441 11297 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.400455 11652 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:05.354835 11734 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.459100  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:05.467952  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63192.9us
W20260709 12:27:05.459615  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 213356us
I20260709 12:27:05.469522  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.470335  6302 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.470899  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.471019  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:05.470425  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:05.419581 11634 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:05.405944 11747 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.348552 11610 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.430869 11745 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:05.427955 11483 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.463696 11674 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.471516  6308 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:05.461779  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.475677  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:05.479099 11765 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.178s	user 0.001s	sys 0.001s
W20260709 12:27:05.479529 11755 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.331s	user 0.001s	sys 0.000s
W20260709 12:27:05.481247 11763 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:27:05.481566 11749 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:27:05.466569 11652 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:05.518934  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 250554us
I20260709 12:27:05.466333 11297 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.474663  6305 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:05.483734 11571 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:05.468308 11739 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:05.493141  6453 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:05.495697 11734 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.477720 11747 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:05.497534 11610 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:05.503000 11780 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:05.493323  6304 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:05.427008 11768 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
W20260709 12:27:05.478529 11383 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.686s	user 0.013s	sys 0.001s
I20260709 12:27:05.457885 11556 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:05.491407  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 134778us
W20260709 12:27:05.522984  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45767.8us
W20260709 12:27:05.524325  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45708.9us
W20260709 12:27:05.507179 11690 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.496s	user 0.013s	sys 0.013s
W20260709 12:27:05.586261 11690 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.575s	user 0.014s	sys 0.014s
I20260709 12:27:05.587380  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3020) took 1394 ms. Trace:
I20260709 12:27:05.587744  6384 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
W20260709 12:27:05.588485  6459 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3011) took 1596 ms (client timeout 1857 ms). Trace:
W20260709 12:27:05.531247 11770 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:27:05.525692  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:05.533408  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 287379us
I20260709 12:27:05.521696 11652 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:05.579722  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:05.595140  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:05.583030 11634 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.575711 11780 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:05.584883 11345 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:05.585417 11556 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.420084 11762 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.526369 11571 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:05.587803  6458 rpcz_store.cc:276] 0709 12:27:04.193336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.193494 (+   158us) service_pool.cc:224] Handling call
0709 12:27:05.587167 (+1393673us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54001626d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:05.587310 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565251840}
W20260709 12:27:05.588901  6459 rpcz_store.cc:269] 0709 12:27:03.992468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.992939 (+   471us) service_pool.cc:224] Handling call
0709 12:27:05.588304 (+1595365us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b54001c1454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:05.588417 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":54127360}
I20260709 12:27:05.593782  6381 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.508194 11674 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.588994 11383 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.630646  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:05.583554 11774 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:05.601109  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:05.646015  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49158.4us
I20260709 12:27:05.649879  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:05.650624  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 125065us
I20260709 12:27:05.640120 11739 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:05.653447  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73831.2us
I20260709 12:27:05.654814  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.657847  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:05.599553 11627 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.675343  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:05.676791  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.679418  6377 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:05.630087 11780 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:05.683468  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.616361 11345 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:05.647244 11602 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:05.706874 11610 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:05.707315 11610 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.707669 11610 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:05.713685  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.631295 11634 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:05.688882  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2539) took 1055 ms. Trace:
I20260709 12:27:05.693461 11383 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:05.693859 11771 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:27:05.697042 11762 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.698652 11674 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:05.682699 11739 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:05.690050 11750 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:05.690130 11483 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:27:05.699596  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47660us
I20260709 12:27:05.700428 11345 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:05.724036  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48784.2us
I20260709 12:27:05.693547 11556 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.722570  6301 rpcz_store.cc:276] 0709 12:27:04.633486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.633991 (+   505us) service_pool.cc:224] Handling call
0709 12:27:04.668964 (+ 34973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.750117 (+ 81153us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.750148 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:04.750197 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:04.753515 (+  3318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.688627 (+935112us) spinlock_profiling.cc:243] Waited 80.9 ms on lock 0x7b54000b3954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:05.688692 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.688830 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":186131328}
I20260709 12:27:05.673458 11632 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:05.693533 11778 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
W20260709 12:27:05.799315  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 141553us
I20260709 12:27:05.564049 11745 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.720969 11536 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:05.684094 11665 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 12:27:05.801226 11780 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:05.804171  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.821359  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:05.806109 11671 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:05.825452 11345 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:05.829667 11792 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:05.829123  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.813987  6452 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.835760  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:05.800483 11750 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:05.801030 11634 raft_consensus.cc:2764] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:27:05.837836  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.836619  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:05.820519 11674 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:05.856195  6454 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:05.823083  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 243375us
I20260709 12:27:05.803077 11739 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:05.852531  6459 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:05.850106  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.853332  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.854455  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.860965  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.861856  6455 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:05.859668  6450 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:05.829139 11643 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.834106 11536 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:05.859712 11739 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:05.848621 11780 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.885407 11750 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.886197 11797 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:05.887291 11802 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:05.904163 11784 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:05.849113 11671 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:05.904808 11784 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.904928 11671 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:05.905205 11784 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:05.857606 11674 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.906874 11674 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.855561 11792 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.909969 11792 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:05.898813  6307 consensus_meta.cc:287] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 12:27:05.855783 11791 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:05.911167 11791 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.911586 11791 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.800733 11571 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:05.911811 11786 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:27:05.904304 11787 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:27:05.915879  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.916400  6307 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:05.918298 11790 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:05.919358  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.902148 11739 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:05.921061 11791 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.922550  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.923009  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:05.924044  6385 raft_consensus.cc:2368] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:05.924739  6376 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:05.925002 11780 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:05.925354 11791 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.926074  6384 pending_rounds.cc:85] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:27:05.928870  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.929422  6380 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:05.901229 11643 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.926357  6384 pending_rounds.cc:107] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:27:05.919469  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:05.930899  6384 raft_consensus.cc:2889] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.917073  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:05.932135  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 218518us
I20260709 12:27:05.933373  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.939448  6454 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:27:05.941497  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 258079us
I20260709 12:27:05.942719  6454 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:05.945186 11556 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.946056 11483 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.945777 11556 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.947124 11297 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.948796 11556 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.949327 11556 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:05.949802 11556 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.946581 11483 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.950978 11297 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:05.892629 11634 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:05.957284  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:05.902637 11536 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:05.961902  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.955451  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.966270  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:05.973963  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 131585us
I20260709 12:27:05.977317  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:05.979810  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:05.976156  6424 leader_election.cc:400] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
W20260709 12:27:05.969671 11725 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.684s	user 0.010s	sys 0.003s
I20260709 12:27:05.963312 11741 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:05.981678  6424 leader_election.cc:403] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:06.000813  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:06.000830  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:05.935091 11784 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:05.975564 11797 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:06.004403  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:05.964521 11634 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:05.980237 11665 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:06.008072  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:06.016543  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.981410 11808 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:06.039096  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:06.040189  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3026) took 1532 ms (client timeout 1546 ms). Trace:
W20260709 12:27:06.040576  6451 rpcz_store.cc:269] 0709 12:27:04.508200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.508323 (+   123us) service_pool.cc:224] Handling call
0709 12:27:06.038872 (+1530549us) spinlock_profiling.cc:243] Waited 6.26 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:27:06.040118 (+  1246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14391552}
I20260709 12:27:06.043411 11643 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.012660 11483 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:06.046934 11809 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:06.047108 11483 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:06.047668 11483 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.047773 11809 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:06.048062 11483 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.040939 11634 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:06.041308 11788 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:06.013491 11791 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:05.998868 11725 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.064478 11747 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.049723 11730 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:06.005740 11810 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:06.013903 11665 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:06.062592 11728 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.080489  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:06.061112 11788 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.066917 11556 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.069141 11791 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:06.112624 11665 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.046406 11671 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.059273 11610 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.116582 11790 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:06.124507 11788 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:06.129778 11791 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:06.135609 11652 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.527s	user 0.011s	sys 0.001s
W20260709 12:27:06.149894  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 149097us
I20260709 12:27:06.155205 11633 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.152822  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:06.157835 11633 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:06.158268 11633 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.158677 11633 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:06.161137  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2459) took 1107 ms. Trace:
I20260709 12:27:06.161561  6377 rpcz_store.cc:276] 0709 12:27:05.053486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.120179 (+ 66693us) service_pool.cc:224] Handling call
0709 12:27:05.128763 (+  8584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.684328 (+555565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.160869 (+476541us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400065254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:06.161044 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265888512}
I20260709 12:27:06.163105  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:06.141433 11647 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.001s
W20260709 12:27:06.169929  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 165580us
I20260709 12:27:06.172122  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:06.132851 11674 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:06.113350 11730 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:06.175850  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95851.8us
I20260709 12:27:06.130594 11792 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.130793 11780 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.181370  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:06.177455 11634 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:06.190410 11634 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.190779 11792 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.191918 11817 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:06.193339  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2293) took 1977 ms. Trace:
I20260709 12:27:06.193439  6453 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:06.182193 11811 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.193775  6303 rpcz_store.cc:276] 0709 12:27:04.216066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.221086 (+  5020us) service_pool.cc:224] Handling call
0709 12:27:06.193073 (+1971987us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54002d23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:06.193245 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3440107264}
I20260709 12:27:06.191084 11510 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:06.194589 11811 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.195096 11510 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:06.196661  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.196190 11510 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.198613 11510 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.197674 11811 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.167510 11652 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.145275 11797 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:06.173054 11643 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 12:27:06.207050  6452 raft_consensus.cc:2368] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
I20260709 12:27:06.179582 11725 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.173318 11647 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.190913 11634 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:06.186599 11808 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:06.219918  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:06.222061  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:06.222995  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:06.223766  6459 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
W20260709 12:27:06.217324  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45327.4us
I20260709 12:27:06.206936 11811 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:06.226384  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:06.226749  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:06.227093  6450 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:06.212944 11643 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:06.215952  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:06.230966  6300 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:06.231112  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:06.231447  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:06.233065  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.233227  6456 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:06.234001  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:06.234789  6302 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:06.231714  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:06.231885  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:06.239439  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:06.243485  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:06.244217  6304 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:06.239604  6310 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:06.238333 11345 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:06.251822 11824 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.251212 11345 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:06.252472 11824 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:06.254263 11345 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.254971  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:06.256372  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:06.256990  6300 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:06.257675  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 760) took 1046 ms. Trace:
I20260709 12:27:06.225865 11792 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:06.241173 11739 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.258344  6382 rpcz_store.cc:276] 0709 12:27:05.210985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.231108 (+ 20123us) service_pool.cc:224] Handling call
0709 12:27:05.231464 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.394358 (+162894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.394380 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.394446 (+    66us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:05.394477 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.257324 (+862847us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:06.257395 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.257585 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374228736}
I20260709 12:27:06.262969  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 790) took 1240 ms. Trace:
I20260709 12:27:06.260982 11739 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.263432  6307 rpcz_store.cc:276] 0709 12:27:05.022061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.034998 (+ 12937us) service_pool.cc:224] Handling call
0709 12:27:06.262795 (+1227797us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54002dfad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:06.262912 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":748997504}
I20260709 12:27:06.225538 11796 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:06.230072 11725 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:06.274427  6300 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:27:06.279071  6302 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:06.231781 11730 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.251210 11814 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:06.240317 11822 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:06.295702  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:06.299764  6308 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.297912 11816 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:06.298480 11819 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:06.293228 11810 raft_consensus.cc:2764] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:27:06.312157 11797 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:06.312904 11813 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.186s	user 0.000s	sys 0.001s
I20260709 12:27:06.337720  6302 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:06.316319 11779 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.836s	user 0.004s	sys 0.001s
I20260709 12:27:06.301324 11816 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.305778 11819 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.319700 11825 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:06.320598 11824 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:06.320817  6308 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:06.321357 11747 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:06.324126 11812 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
W20260709 12:27:06.335587  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 110360us
I20260709 12:27:06.336817  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:06.369061  6382 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:06.368997  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:06.370024 11747 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:06.370062  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.337646  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.370854 11674 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:06.339254 11808 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.370867 11747 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:06.371819  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:06.372418  6385 raft_consensus.cc:2376] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16. Re-sending same reply.
I20260709 12:27:06.372223 11747 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.373296  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.373895  6385 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:06.338428  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:06.348420  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.376047  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.305262 11345 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.310639 11610 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.286163 11823 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:27:06.346174 11816 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:06.353798 11780 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.357471 11791 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:06.350728 11819 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:06.372241 11652 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.358855  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.377418  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:06.774941  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.773s	user 0.021s	sys 0.016s
W20260709 12:27:06.867314  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.865s	user 0.022s	sys 0.016s
W20260709 12:27:06.867638  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 893959us
I20260709 12:27:06.868113  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:06.868446  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:06.869208  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:06.802459  7288 connection.cc:256] RPC call timeout handler was delayed by -1.5306536529997175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260709 12:27:06.797127  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:06.796293  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:06.802524  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:06.399394 11345 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:06.426205 11483 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:06.481585 11536 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.510s	user 0.008s	sys 0.001s
I20260709 12:27:06.498867 11633 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.370373 11634 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.469642 11647 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:06.756951 11671 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:06.468142 11728 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:06.581218 11757 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:06.302335 11665 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.494179 11788 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.805001 11790 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:06.475834 11791 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:06.293941 11796 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.524968 11814 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:06.544314 11816 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.527504 11819 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:06.402704 11825 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:27:06.866899  6383 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:06.869244  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:06.664753  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83795us
I20260709 12:27:06.878415  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 795) took 1854 ms. Trace:
I20260709 12:27:06.881165 11610 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:27:06.958972 11783 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:06.959959 11797 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:06.958331  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.012s	sys 0.038s
I20260709 12:27:06.984517 11522 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.009219  6305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.038s	sys 0.070s
I20260709 12:27:07.027072 11784 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:07.023979  7288 connection.cc:256] RPC call timeout handler was delayed by -1.1553483349998714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1829s for a response
I20260709 12:27:07.034946 11571 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:07.069432  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.009s	sys 0.016s
W20260709 12:27:07.082787  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.082659 11827 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.083614 11781 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:07.088421  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.090466  6456 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2638) took 1737 ms (client timeout 2014 ms). Trace:
I20260709 12:27:07.125923  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.127110 11602 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.129136  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.131212  6453 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:27:07.132714  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.133381  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:07.151796  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 773957us
W20260709 12:27:07.160473  6300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.861s	user 0.006s	sys 0.017s
W20260709 12:27:07.179759  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2: Illegal state: Replica 4e0e521cb08c420f981a29576eee90a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:07.185585 11799 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:07.192646  7319 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:07.199770 11750 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:07.200825  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.204454 11557 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.304026  6301 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:07.304284 11825 consensus_queue.cc:1243] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:07.304971 11825 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:07.305222 11665 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.305332 11825 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:07.306768  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:07.307658 11840 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:07.307749  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.308138 11840 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:07.308261  6455 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 7. Current term is 8. Ops: []
I20260709 12:27:07.308593 11840 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:07.309219  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.306250  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2549) took 1086 ms. Trace:
I20260709 12:27:07.310745  6301 rpcz_store.cc:276] 0709 12:27:06.220079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.220189 (+   110us) service_pool.cc:224] Handling call
0709 12:27:07.306083 (+1085894us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400206cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:07.306194 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2463768576}
W20260709 12:27:07.311233  6305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.017s	user 0.041s	sys 0.070s
I20260709 12:27:07.312114  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2552) took 1091 ms. Trace:
I20260709 12:27:07.306279 11825 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.312528  6305 rpcz_store.cc:276] 0709 12:27:06.221033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.221192 (+   159us) service_pool.cc:224] Handling call
0709 12:27:06.232540 (+ 11348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.232752 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:06.293777 (+ 61025us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.311753 (+1017976us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:07.311823 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:07.311880 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.311944 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.312055 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25873,"prepare.queue_time_us":26388,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"thread_start_us":1050678,"threads_started":2,"wal-append.queue_time_us":205119}
I20260709 12:27:07.314379 11799 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.314808 11797 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.316234 11627 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.316522 11799 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.316792 11627 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.316937 11788 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:07.317803  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.038s	user 0.010s	sys 0.021s
I20260709 12:27:07.318332 11790 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.313300 11602 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.319372 11819 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.320901  6303 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:07.320983 11602 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.321862 11483 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.322012  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2547) took 1102 ms. Trace:
I20260709 12:27:07.322574  6303 rpcz_store.cc:276] 0709 12:27:06.219102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.219237 (+   135us) service_pool.cc:224] Handling call
0709 12:27:07.321799 (+1102562us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002d28d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:07.321949 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2501742464}
W20260709 12:27:07.322592  6300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.009s	sys 0.017s
I20260709 12:27:07.326318 11345 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.327275 11784 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:07.327768 11784 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.328197 11784 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:07.327831 11757 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.328969  7314 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:07.324239 11627 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.327062  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2558) took 1103 ms. Trace:
I20260709 12:27:07.331317  6300 rpcz_store.cc:276] 0709 12:27:06.223818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.285336 (+ 61518us) service_pool.cc:224] Handling call
0709 12:27:06.285637 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.285844 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:06.299690 (+ 13846us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.325700 (+1026010us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.326642 (+   942us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:07.326703 (+    61us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:07.326768 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.326859 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.326993 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15790,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":3344,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":131,"thread_start_us":1025183,"threads_started":1,"wal-append.queue_time_us":152249}
I20260709 12:27:07.332845 11769 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:07.327093  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 457870us
I20260709 12:27:07.333423 11827 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.343725 11769 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.344795 11757 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.345808 11827 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.346216 11769 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.334672 11536 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.347069  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.347422 11769 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:27:07.218860  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.348687  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:07.324725 11522 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.350512 11536 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:07.350980  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.013s	sys 0.040s
I20260709 12:27:07.335525 11781 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.351567 11827 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:07.352128 11827 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:07.353169 11781 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.353822 11802 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.353049 11827 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.354432 11575 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.354373 11802 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.355137 11575 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.356606 11802 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.357169 11802 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.357407 11575 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.324548  6309 rpcz_store.cc:276] 0709 12:27:05.023581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.064683 (+ 41102us) service_pool.cc:224] Handling call
0709 12:27:05.134822 (+ 70139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:05.135252 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:05.135305 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:05.161240 (+ 25935us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:05.161331 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.161428 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.878160 (+1716732us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b54002071d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:06.878329 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1297,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":39424,"spinlock_wait_cycles":160466688}
I20260709 12:27:07.357647 11802 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.326398 11796 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.358017 11575 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.226948 11730 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:07.359675 11575 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.360059 11730 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.360476 11840 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.362066 11730 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.336580  6272 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:07.336302 11557 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.243927  7329 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:07.217296  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.338068  6454 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:07.219264  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:07.338469  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.05885e+06us
I20260709 12:27:07.210388 11823 consensus_queue.cc:1243] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:07.339553  6458 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:07.349092  6457 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:07.229301 11404 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:07.341454 11610 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:27:07.363946 11796 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:27:07.317039 11799 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.310276  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.387624 11689 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:07.348730  6302 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:07.388455 11796 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:07.388201 11689 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.388854 11796 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:07.389716 11796 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.342168  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.324120 11634 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.365904 11840 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.387578  6459 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:07.317623 11783 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.390872  6381 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.391980 11783 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.392257 11745 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:07.392231 11784 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.394797 11689 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.397480 11796 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.365672  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.237984  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:07.386719 11610 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:07.404037 11610 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.405342 11610 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:07.387370 11799 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.363190  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:07.307909 11633 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.340776 11816 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:07.409124  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46017us
I20260709 12:27:07.336393 11791 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.317111 11797 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.410164 11799 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.410630 11627 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:07.364953 11647 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.313109 11825 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.412091 11647 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.412578 11404 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.413687  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:07.415001 11404 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:07.416100 11647 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.417294 11627 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:07.406800 11784 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.400403  6381 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:07.391325  6304 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:07.419221  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 181330us
I20260709 12:27:07.384797  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 831) took 1168 ms. Trace:
I20260709 12:27:07.401968  6382 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:07.421566  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:07.422029  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2814) took 1083 ms. Trace:
I20260709 12:27:07.362263 11575 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.422639  6382 rpcz_store.cc:276] 0709 12:27:06.338440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.369834 (+ 31394us) service_pool.cc:224] Handling call
0709 12:27:07.421839 (+1052005us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400109fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:07.421971 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2370397952}
W20260709 12:27:07.334241  6456 rpcz_store.cc:269] 0709 12:27:05.353353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.353545 (+   192us) service_pool.cc:224] Handling call
0709 12:27:07.090190 (+1736645us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b54001e14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:07.090388 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1901725568}
I20260709 12:27:07.422439  7329 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:27:07.424664  6457 rpcz_store.cc:276] 0709 12:27:06.216604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.217160 (+   556us) service_pool.cc:224] Handling call
0709 12:27:06.223133 (+  5973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.383332 (+1160199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.384567 (+  1235us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001e19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:07.384738 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589065472}
I20260709 12:27:07.385236 11557 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.427014 11571 raft_consensus.cc:2764] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:27:07.418402  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103585us
I20260709 12:27:07.399746 11522 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.327059 11345 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.400188  6272 leader_election.cc:403] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:27:07.430202  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 93844.4us
I20260709 12:27:07.429989 11345 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.430683 11557 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:07.430940  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.431586 11689 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:07.431970 11689 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:07.432314 11689 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.432376 11345 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.432984 11345 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.410164 11791 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.433430 11345 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.435084  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.436764  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.437609 11345 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.438712  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.438892  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.439145  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.439972  6379 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.440642 11802 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.418318 11627 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.441063  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.441987  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.442764  6382 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.443297  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.444629  7329 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:41092
I20260709 12:27:07.444747  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.445731  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.433192  6451 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:07.448336  7329 raft_consensus.cc:606] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.450855  7329 raft_consensus.cc:614] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:07.448693  6274 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14.
I20260709 12:27:07.449522 11791 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.431990  6272 leader_election.cc:400] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
I20260709 12:27:07.445405  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:07.460672  6272 leader_election.cc:403] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:07.451486  6304 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:07.452687  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:07.452666  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.408353 11790 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:07.457864  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55593.6us
I20260709 12:27:07.459825  6307 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:07.467741  6302 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:07.471007  6383 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:07.473706  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.474643  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:07.475188  7332 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.465721  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:07.467581 11557 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:07.478643 11745 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:27:07.478727 11557 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.478986  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.479557 11557 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:07.481720  6450 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:07.482559  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.482746  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.483100  6459 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:07.483387  6450 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:07.478782  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.484141  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.497339  6450 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.500679  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.502429  6307 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:07.502795  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.502480  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.504622  6309 raft_consensus.cc:2368] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:07.511771  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.512210  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.515198  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.501291  6457 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:07.519522  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.519790  6300 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:07.520229  6383 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:07.520715  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.521417  6453 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:07.519578  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.520279  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.692245  6456 raft_consensus.cc:2368] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:27:07.636893  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.562852  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:07.671643  6203 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 297) took 42467 ms (client timeout 15000 ms). Trace:
I20260709 12:27:07.695348  6384 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:07.647892  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.685815  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.565057  6380 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:27:07.530388  6454 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:07.604883  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.698834  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:07.686759  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 221095us
I20260709 12:27:07.467533 11665 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.697569  6459 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
W20260709 12:27:07.684190  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 187814us
I20260709 12:27:07.708348  6454 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:27:07.708628  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.529934  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.706s (SENT) [suppressed 18 similar messages]
I20260709 12:27:07.710573  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.712399  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.523324  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:07.700519  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.605585  6308 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:07.694795  6453 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:07.700778  6301 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:27:07.714205  6301 pending_rounds.cc:85] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 13
I20260709 12:27:07.714579  6301 pending_rounds.cc:107] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:27:07.714835  6301 raft_consensus.cc:2889] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.715823  6305 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:27:07.716274  6305 pending_rounds.cc:85] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:27:07.716560  6305 pending_rounds.cc:107] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:27:07.701437  6384 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:07.716917  6305 raft_consensus.cc:2889] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:07.701996  6458 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.711515  6379 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:07.702220  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:07.712867  6272 leader_election.cc:400] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:07.498227 11791 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.718952  6272 leader_election.cc:403] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:07.719720  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.719067  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.574555  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.721230  6300 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.722445 11791 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.725019 11791 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.519783 11739 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:07.725896 11739 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.602952 11483 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:07.727130 11483 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:07.727531 11483 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:07.728237  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.606339 11826 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.734897  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.734596 11665 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.713850  6450 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:07.769294  6302 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 9: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:07.735443  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.769807  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.728662 11739 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:07.770423  6450 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:07.720887  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:07.695524  6203 rpcz_store.cc:269] 0709 12:26:25.204688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.205314 (+   626us) service_pool.cc:224] Handling call
0709 12:27:07.671504 (+42466190us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:05.262882 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:05.494320 (+231438us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:05.494387 (+    67us) write_op.cc:435] Acquired schema lock
0709 12:27:05.494413 (+    26us) tablet.cc:662] Decoding operations
0709 12:27:05.495284 (+   871us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:05.495407 (+   123us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:05.495447 (+    40us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:27:05.496144 (+   697us) tablet.cc:701] Row locks acquired
0709 12:27:05.496210 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:27:05.496427 (+   217us) write_op.cc:270] Start()
0709 12:27:05.496592 (+   165us) write_op.cc:276] Timestamp: P: 1783600025496353 usec, L: 0
0709 12:27:05.496616 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:27:05.497480 (+   864us) log.cc:844] Serialized 2029 byte log entry
0709 12:27:05.832857 (+335377us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b
0709 12:27:06.509134 (+676277us) op_driver.cc:464] REPLICATION: finished
0709 12:27:06.560366 (+ 51232us) write_op.cc:301] APPLY: starting
0709 12:27:06.560483 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:06.564634 (+  4151us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:06.564677 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:06.581406 (+ 16729us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:06.581693 (+   287us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:27:06.581757 (+    64us) write_op.cc:312] APPLY: finished
0709 12:27:06.582341 (+   584us) log.cc:844] Serialized 57 byte log entry
0709 12:27:06.583244 (+   903us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:06.583390 (+   146us) write_op.cc:454] Released schema lock
0709 12:27:06.583462 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:06.583940 (+   478us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:06.583992 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":797885,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":48652,"apply.run_cpu_time_us":9310,"apply.run_wall_time_us":23547,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":382952,"prepare.run_cpu_time_us":16034,"prepare.run_wall_time_us":748059,"raft.queue_time_us":135942,"raft.run_cpu_time_us":13478,"raft.run_wall_time_us":55600,"replication_time_us":1006358,"spinlock_wait_cycles":62974208,"thread_start_us":789997,"threads_started":4,"wal-append.queue_time_us":512998,"wal-append.run_cpu_time_us":49622,"wal-append.run_wall_time_us":1569512}]]}
I20260709 12:27:07.744275 11483 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:07.774582  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62267.4us
I20260709 12:27:07.775442  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.777509  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.744670  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.745707  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.746881  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:07.627393 11823 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:07.748127  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.749357  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:07.750074  6300 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:07.782244  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.752238 11802 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:07.791666  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.604727 11404 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.754266  6454 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:07.716337 11750 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:07.562224 11689 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.760749  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.763780 11790 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.764086  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.801640  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.801846  6308 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:07.706852 11824 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:27:07.720239  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402a1967982740658c70049fd62be44e"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:07.802942  6455 raft_consensus.cc:606] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.803436  6455 raft_consensus.cc:614] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:07.769083 11671 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:07.729086 11730 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:07.653273 11845 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:07.735423  6381 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.771973  6452 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 8. Current term is 9. Ops: []
I20260709 12:27:07.773778  6309 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.799239 11610 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.812772  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.813055 11610 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.813629  6309 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:07.780934  6385 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:07.753618  7332 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:07.795922  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.806737 11791 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.791555 11783 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.790752 11840 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:07.765942  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.816339  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.820508  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:07.802341  6302 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.817098  6377 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:07.767139  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54406.3us
I20260709 12:27:07.808928  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.810045  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.814479  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:07.800784  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.815248  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.823228  6309 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:07.815397  6301 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.815469  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.822078  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.815976  6456 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:07.826807  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.827487  6455 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:07.815161 11757 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:07.830912 11783 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.831607 11847 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:07.834466 11634 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:27:07.834749  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080022bfe0 after lost signal to thread 5140
I20260709 12:27:07.834942 11634 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:07.835907 11757 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:07.836330  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6500 after lost signal to thread 6240
I20260709 12:27:07.837312  7319 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:07.837857  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080034a620 after lost signal to thread 6334
I20260709 12:27:07.840307  6272 leader_election.cc:400] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:27:07.840706  6272 leader_election.cc:403] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:27:07.842691  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080034ad60 after lost signal to thread 6409
W20260709 12:27:07.844780  6480 debug-util.cc:398] Leaking SignalData structure 0x7b0800341c40 after lost signal to thread 6483
I20260709 12:27:07.849265 11757 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:07.849798 11757 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:07.815375 11610 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.841552  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.790239 11831 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:07.864802  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:07.864741  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84114.4us
I20260709 12:27:07.806638 11745 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
I20260709 12:27:07.873914  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.861876  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:07.876087  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.876323  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.877321  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.878050  6302 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:07.879339  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.851678  6457 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:07.847651  6458 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:07.880797  6454 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:07.861532  6480 debug-util.cc:398] Leaking SignalData structure 0x7b08002c0480 after lost signal to thread 7371
W20260709 12:27:07.881620  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080038c5e0 after lost signal to thread 7373
W20260709 12:27:07.859031  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 156929us
I20260709 12:27:07.882964  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:07.861541  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.853334  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:07.872503  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 151719us
I20260709 12:27:07.888314  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:07.889057  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 359155us
I20260709 12:27:07.798545 11823 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.889132  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:07.851697 11802 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.863404 11739 raft_consensus.cc:2764] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:27:07.903193  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.895069  6425 leader_election.cc:400] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:07.891748  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:07.801482 11750 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.899670  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.870878 11689 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:07.866606 11671 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:27:07.904687  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:07.908982  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.924887  6456 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:07.909843  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:07.916558  6300 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:07.923676  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.881258 11788 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:07.928100  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:07.815864 11791 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:07.933997  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:07.934926  6451 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:07.897548 11808 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:07.938329  6302 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:07.939579 11788 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:07.940661  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:07.941082  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.948858  6303 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:07.943358  6308 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:07.945436  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.947134  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:07.903236 11423 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:07.954025  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:07.959232  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 109833us
I20260709 12:27:07.962999  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:07.963663  6300 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:07.836074 11634 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.974462  6300 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:07.974531 11840 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:07.987468  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.908277 11802 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.951023 11689 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 12:27:08.004477  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94711.9us
I20260709 12:27:07.957183 11808 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:08.009202  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.006925  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:07.962632 11602 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:08.012806  6300 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:08.011389  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.004776 11423 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:08.000398 11840 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:08.001423  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:08.022639  6459 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:08.025339  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:07.969321 11797 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:27:08.004313 11860 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:08.032506  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78076us
I20260709 12:27:08.004057 11647 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:27:08.014983 11602 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:08.021323 11575 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:08.043220  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.021257 11784 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:08.049598  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91420.7us
I20260709 12:27:08.030802 11665 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.031162 11840 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.026664 11483 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.041213 11345 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:08.040016 11423 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:08.044237 11825 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.060175  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.067157  6450 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:08.069775  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.126142  6458 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:27:08.126781  6454 raft_consensus.cc:606] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.127157  6454 raft_consensus.cc:614] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:08.112354  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.047762 11602 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.110868 11730 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:08.056425 11784 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:08.151149 11864 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:27:08.154734 11823 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.154822  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.060820 11345 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:08.156404  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.157136  7317 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:08.157696  6272 leader_election.cc:400] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:27:08.158051  6272 leader_election.cc:403] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:27:08.158149  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.158677 11483 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12.
I20260709 12:27:08.159561  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.039486 11860 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:08.124084  6452 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:08.054199 11536 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:08.051378 11575 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:08.050944 11859 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:08.189949  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:08.194262  6303 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:08.195603 11575 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:08.223487  6309 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: [9.6-12.8]
I20260709 12:27:08.196022 11689 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:08.198072  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.195307  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 140278us
I20260709 12:27:08.038293 11788 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:08.235617  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.241793  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.238756  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 213472us
I20260709 12:27:07.820359 11845 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:07.775334 11826 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:08.250557  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.168771 11860 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:08.254751  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 253422us
I20260709 12:27:08.180852 11536 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:08.161607 11825 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.243659 11689 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:08.273172  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:08.258049 11871 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
W20260709 12:27:08.275403  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 232743us
W20260709 12:27:08.271768  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 211560us
W20260709 12:27:08.262611 11854 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:27:08.161417 11808 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.225047 11790 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:27:08.261226 11745 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.258175 11819 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:08.198655 11757 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.264592 11536 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.262810 11868 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:08.272208 11869 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.275836 11689 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:08.314218 11857 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.263s	user 0.001s	sys 0.000s
I20260709 12:27:08.279677 11791 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.270365 11861 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:27:08.303038 11819 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.338425 11877 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:27:08.369432 11877 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:08.334779 11862 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:27:08.338152 11557 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.002s
W20260709 12:27:08.335446 11853 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:27:08.322583 11750 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.269299 11423 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.332049 11826 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.378624  6456 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:08.286406 11745 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.369995 11877 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.388801 11877 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:08.394379  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 135546us
I20260709 12:27:08.333984 11602 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:08.388998 11404 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.000s
I20260709 12:27:08.375259 11802 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:08.391458 11845 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.396270  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.414299  6459 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:08.400681 11757 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.322741 11869 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.371591 11791 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:27:08.406458 11826 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:08.407449 11602 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.409832  6457 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:08.411168 11536 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.440064  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.412070  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.317992 11868 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:08.444406  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 167773us
I20260709 12:27:08.445416  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.446720 11877 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:08.432087 11840 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.434984 11575 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.436178 11745 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.436910 11634 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.379426 11557 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.431044 11869 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.376711 11892 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:27:08.413694 11404 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:08.403779 11889 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:08.454676  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 178074us
W20260709 12:27:08.494799  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47899.1us
I20260709 12:27:08.496740  6453 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:08.497285  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.456202  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:08.455309  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.456488  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.497102  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:08.500613  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44524.6us
I20260709 12:27:08.450409 11826 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:08.450351 11868 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:08.501933  6310 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 11: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:08.504597  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 245897us
I20260709 12:27:08.504632  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.454667 11784 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.513111  7332 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:08.514160  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:08.514396  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:08.515911  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.502259 11783 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:08.505551 11826 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:08.519358  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.496124 11889 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.534629 11889 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.543550 11840 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.514886 11884 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:08.551779  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.553975  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.555706  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.560000  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2318) took 1058 ms. Trace:
I20260709 12:27:08.519670 11790 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:27:08.562860 11802 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:08.563060 11884 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.564263 11884 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.560508  6304 rpcz_store.cc:276] 0709 12:27:07.501417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.505863 (+  4446us) service_pool.cc:224] Handling call
0709 12:27:07.506236 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.807107 (+300871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.807153 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.815704 (+  8551us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.962199 (+146495us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:07.999012 (+ 36813us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.012674 (+ 13662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.548294 (+535620us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54002d14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:08.548348 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.548392 (+    44us) raft_consensus.cc:1685] finished
0709 12:27:08.559733 (+ 11341us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.559913 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5086,"prepare.queue_time_us":36797,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":23005,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":8170,"spinlock_wait_cycles":687216640,"thread_start_us":146095,"threads_started":1,"wal-append.queue_time_us":240270}
W20260709 12:27:08.565383 11851 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.666s	user 0.004s	sys 0.009s
I20260709 12:27:08.564643 11689 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.567894 11689 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.563728 11790 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:08.567970  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.569486 11802 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:08.568821 11790 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.571784 11790 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.572769  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.574519 11802 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:08.575166 11802 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:08.574476 11790 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.576360 11602 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:08.576066 11802 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.518898 11797 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:08.579833 11886 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:27:08.584298 11536 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.579674 11892 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:08.588826 11847 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:08.589272 11866 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:08.594813 11866 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:08.595969  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.005s	sys 0.021s
W20260709 12:27:08.596473  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.005s	sys 0.021s
I20260709 12:27:08.597297 11866 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.520453 11877 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:08.518723 11750 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:08.598774 11750 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.530211 11826 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:08.599953  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 581018us
I20260709 12:27:08.591033 11610 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:08.600615 11610 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.600514 11847 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.564302 11557 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.602784 11847 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.515679 11891 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:08.604737 11797 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.601275 11610 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.605863  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.606405 11797 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.610565 11891 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:08.611066 11891 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:08.611950 11891 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.614642 11819 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.599732 11750 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.603262  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:08.617961 11757 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:08.621487  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.625665  7332 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:08.630874  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.631667  6307 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:08.629942  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.635447 11892 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.635793  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.636238 11892 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.638805 11877 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:08.639305 11877 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.639968 11877 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.646181 11892 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:08.651907  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.652714  6307 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:27:08.656693  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.661499  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:08.666137  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 152052us
W20260709 12:27:08.669595  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.675239  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.678582  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.663988 11892 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:08.682857  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 162752us
I20260709 12:27:08.676641  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:08.684576  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.687132  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.694062  7335 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:08.685078  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:08.696305  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.697024 11750 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:08.683149 11892 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.698132 11892 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.705129 11892 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.707858 11892 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:08.713248  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:08.713908  6302 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.705552 11610 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.716126 11877 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.716747 11610 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.722254 11831 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.694511 11895 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:08.722877 11831 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.723500 11895 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:08.723912 11895 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:08.725188 11831 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.729223  6302 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:08.729718  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:08.730578  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.731186  7314 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:08.731961 11888 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:08.733276 11888 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:08.733649 11888 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.735532 11860 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.738193 11860 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.738829  7319 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:08.738538 11888 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:08.739132  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.740216 11888 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:08.740783 11886 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:08.741372 11886 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:08.741091 11345 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.742209 11345 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:08.754999 11889 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:08.748515  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52254us
I20260709 12:27:08.756093  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.756492  7314 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:08.757856  7335 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:08.758358  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:08.759043 11889 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.759287  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.759675  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.760569  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.761663  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.696554 11871 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:08.761946 11889 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:08.728365 11826 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.763239  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:08.742532 11750 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.784758  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.785943  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.764731  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.787395  7318 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:08.788100  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.742347 11886 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.743775 11872 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:08.790696 11872 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.772259 11627 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:08.792330  6459 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:08.792618 11627 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:08.793359  6300 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:27:08.793452 11627 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.798895  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.791692 11872 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.786855  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.822044  6458 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:08.823429  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:08.824395  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.781248  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.783535  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2327) took 1271 ms. Trace:
W20260709 12:27:08.814910  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53531.7us
I20260709 12:27:08.764273  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:08.825956  6310 rpcz_store.cc:276] 0709 12:27:07.511706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.511851 (+   145us) service_pool.cc:224] Handling call
0709 12:27:08.783317 (+1271466us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54002e7a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:08.783473 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2265253120}
I20260709 12:27:08.742760 11831 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.834052 11831 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.834833 11750 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.800845 11575 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:08.773036  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:08.830554  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.839265  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:08.832211  6302 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.840116  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.841547  6455 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:08.829789 11871 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:08.772730 11855 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.727s	user 0.000s	sys 0.007s
W20260709 12:27:08.852634  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79956.3us
I20260709 12:27:08.855168  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.763320 11902 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.853649  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:08.857653  7318 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:08.858237  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.820906 11610 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:08.773914 11826 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.781198 11877 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:08.859877  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:08.861339  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:08.858649  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:08.861853  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.859918  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:08.864879  6458 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:08.865748  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.866369  7318 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:08.867969  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:08.868659  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.872385  7329 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:08.872809  6307 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:08.868443  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:08.873215  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.873776  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:08.872756  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.873714  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.869168  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:08.870333  6300 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:08.875094  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:08.876951  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.878837  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:08.875911  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.784710 11797 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:08.855420 11831 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:08.909966 11826 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:27:08.903590  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.904779  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.924418  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.973965  7329 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:08.925652  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.926772 11885 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:08.876329 11873 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:08.884704 11872 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:08.878136 11892 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:08.876215 11845 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:08.882524 11823 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:08.855463 11627 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:27:08.934907  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.936338  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:08.939787  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.858531 11901 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:08.903322 11783 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:08.870160 11745 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:08.879977  7316 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:08.953410  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 771) took 1232 ms. Trace:
I20260709 12:27:08.940917 11610 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:09.003463 11610 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.003902 11610 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:08.964962  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:08.887194  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:08.780709 11868 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:08.879508 11884 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:08.986608  6451 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 8: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:08.989068  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 110197us
I20260709 12:27:08.990649  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:08.944391 11877 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:27:08.996625  7331 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:08.998363  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.011722  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:08.999179  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:09.007133  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:09.001541  6379 rpcz_store.cc:276] 0709 12:27:07.721531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.721656 (+   125us) service_pool.cc:224] Handling call
0709 12:27:07.721980 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.724132 (+  2152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.724165 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:07.724703 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.907215 (+182512us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:07.907273 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.907379 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.772155 (+864776us) spinlock_profiling.cc:243] Waited 4.37 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:08.946756 (+174601us) raft_consensus.cc:1685] finished
0709 12:27:08.946778 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.953329 (+  6551us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10046208,"thread_start_us":182112,"threads_started":1,"wal-append.queue_time_us":283795}
I20260709 12:27:09.014096  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.014374  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.015132  6450 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
W20260709 12:27:09.009424  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 122372us
I20260709 12:27:09.017378  6450 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:08.966015 11557 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
W20260709 12:27:09.019348  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 116953us
I20260709 12:27:09.021144  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:09.021514  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:08.997252 11901 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.024286  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:09.022912  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:08.996225 11885 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:09.027933  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:08.934971 11871 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.023670  7329 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:09.026197  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.026907  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:09.027161  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.066715  6458 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:09.009990 11845 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:08.997514 11783 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:09.027876  6452 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:09.065040  7331 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:08.969391 11889 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:09.067205  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47202.3us
I20260709 12:27:09.082244  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.028000  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.083425  6301 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:09.071369 11788 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.031599  6303 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:09.078164  7314 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:09.079337 11797 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.019671 11819 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:09.087129 11819 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:09.088111 11819 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.072898  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:09.095660  6451 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: [10.5-10.5]
I20260709 12:27:09.029165 11796 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:27:09.025808 11860 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:09.084297 11845 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.034586 11634 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.084472 11783 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.107762  7335 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:09.109701  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.109057  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2334) took 1422 ms. Trace:
I20260709 12:27:09.123234  6309 rpcz_store.cc:276] 0709 12:27:07.686751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.951974 (+265223us) service_pool.cc:224] Handling call
0709 12:27:08.198441 (+246467us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:08.233393 (+ 34952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.108848 (+875455us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400020dd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:09.108990 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566018432}
I20260709 12:27:09.028697 11901 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.116804  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:09.107242  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2575) took 1322 ms. Trace:
I20260709 12:27:09.108922 11889 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:09.127916  6305 rpcz_store.cc:276] 0709 12:27:07.785227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.806990 (+ 21763us) service_pool.cc:224] Handling call
0709 12:27:09.107081 (+1300091us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b54002071d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:09.107185 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1030489344}
I20260709 12:27:09.024708 11575 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:27:09.098434 11819 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.102849 11797 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.064517 11891 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:27:09.099249 11910 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.121953 11745 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:09.120566 11860 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:09.119712 11916 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:09.161032  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44264.9us
I20260709 12:27:09.107496 11557 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:27:09.114792 11895 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.171936  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:09.137140 11824 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:09.171092 11860 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:09.162992 11745 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:09.171319 11916 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:09.184604 11823 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:09.194285 11845 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.199723 11557 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:09.168362 11910 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.167778 11885 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:09.216727  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44902.6us
I20260709 12:27:09.231685  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.233173  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.201653  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55284 (request call id 13) took 1270 ms. Trace:
I20260709 12:27:09.215636 11745 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.234728  6454 rpcz_store.cc:276] 0709 12:27:07.940223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.940529 (+   306us) service_pool.cc:224] Handling call
0709 12:27:09.201414 (+1260885us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b54002c9554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:09.201555 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410144128}
I20260709 12:27:09.240700 11823 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.242524 11610 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.243896  6200 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 8 to 10, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 10 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:09.216331 11916 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.247586 11916 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:09.208537 11895 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.215497 11824 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.252501 11824 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:09.253805 11902 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.204051 11868 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:09.200941 11423 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:09.254154 11783 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.278630 11783 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:09.278367 11522 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:09.263846 11910 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.272650  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 852) took 1008 ms. Trace:
I20260709 12:27:09.279721 11522 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:09.279690  6453 rpcz_store.cc:276] 0709 12:27:08.264442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.264633 (+   191us) service_pool.cc:224] Handling call
0709 12:27:08.275305 (+ 10672us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.501254 (+225949us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.272287 (+771033us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54002c8154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:09.272576 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508826496}
I20260709 12:27:09.280414 11522 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:09.263751 11881 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:27:09.281486 11881 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:09.282297 11881 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.283432 11797 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:09.276539 11892 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:27:09.284407 11892 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:09.284752 11892 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:09.285487 11892 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.239528 11788 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.261013  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:27:09.286969  6451 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.288408  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:09.223758 11810 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:09.237293 11826 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:09.289930 11826 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:09.290398 11826 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.276333 11634 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:09.276342  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:09.269251 11860 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.215889 11875 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:09.294080 11875 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:09.279945 11790 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:09.295235 11790 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:09.295213 11875 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.296348 11891 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:09.296298 11901 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.297250 11891 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:09.293421  6454 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.262567  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 850) took 1148 ms. Trace:
I20260709 12:27:09.298789 11901 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.298964  6457 rpcz_store.cc:276] 0709 12:27:08.114303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.123627 (+  9324us) service_pool.cc:224] Handling call
0709 12:27:08.123895 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.422583 (+298688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.262250 (+839667us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b5400345154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:09.262466 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657165824}
I20260709 12:27:09.287863 11522 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.301853  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:09.253916 11610 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.289531  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:09.314184  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:09.315007  7318 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:09.315295  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.272665  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:09.317080  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.305935  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:09.318773  7317 raft_consensus.cc:2376] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13. Re-sending same reply.
I20260709 12:27:09.311573  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:09.319653  7332 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:09.311906  7335 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:09.283469 11881 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:09.312211  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:09.312539 11892 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.323220  7335 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:09.288899 11423 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:09.293478 11860 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.320992  7331 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.213264 11825 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.317715  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
W20260709 12:27:09.324534  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52004.6us
I20260709 12:27:09.324708  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:09.263197 11575 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.166159 11913 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.330427  6457 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:27:09.315847  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.327495  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.341451  6454 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:09.332633  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.296718 11875 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.305011  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.345301  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.347923  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.348686  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.328430 11797 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:27:09.350142  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.352172  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.352531  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.357969  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.360973  6378 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:09.361124  6382 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:09.360991  6384 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17.
I20260709 12:27:09.358453  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.362815  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.365027  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.365898  6383 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:27:09.372514  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:09.373206  6300 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:09.378300  7331 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:09.380503  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:09.364300  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.382529  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.382447  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.410784  6301 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.395057  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.392896  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.338879 11909 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:09.322577 11819 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:27:09.419111  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.419833  6308 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:09.425864  6303 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:09.421259  6379 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:09.298194 11891 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.343318 11522 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:09.341555 11873 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:09.417394  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.331060 11876 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:27:09.296386 11790 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.348866 11867 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.348987 11757 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:09.441026  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:09.474563  6458 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
W20260709 12:27:09.444523  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.339591 11861 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:27:09.449121  6377 raft_consensus.cc:2368] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
I20260709 12:27:09.457604  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.348556 11345 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
W20260709 12:27:09.458433  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.458659 11790 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:09.461539  6274 leader_election.cc:336] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.546s (SENT)
W20260709 12:27:09.487481  6274 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.546s (SENT)
I20260709 12:27:09.432256 11634 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:09.469373  6381 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:09.477641  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.480952 11847 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.356961 11860 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.094s
I20260709 12:27:09.349377 11575 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.491555  6380 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:09.450639 11788 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.518224  6377 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:09.493394  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.463316  6300 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:09.498889  6308 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:09.427634 11869 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:09.350656 11913 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.466266 11810 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.430899 11557 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:27:09.430765 11797 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:27:09.528607  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.529313  6456 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.547803  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:09.559928  6309 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.442409 11909 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.479212 11745 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.246367  6200 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 14 to 15, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 15 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:09.572755  6300 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:09.513070 11819 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.576882  6308 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:09.547065 11739 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:09.427963 11423 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.535940 11757 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:09.570384 11797 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:09.536158 11869 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:09.581974  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.015s	sys 0.022s
W20260709 12:27:09.600785  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.015s	sys 0.022s
I20260709 12:27:09.504410 11634 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.588717  6310 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.502274 11877 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:27:09.590878  6377 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:09.595010  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 108976us
W20260709 12:27:09.591895  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102810us
I20260709 12:27:09.589974  6305 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:09.592343  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 234138us
I20260709 12:27:09.536542 11788 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.624719  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:09.616971  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.629535  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.618409  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.631037  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.486896 11873 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.633932  6309 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:09.589963 11757 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:09.611723 11869 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:09.642194  6303 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:09.643862  6200 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 12 to 15, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 15 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:09.648003 11847 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:09.652216 11788 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.653002 11575 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.653873 11634 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.547312 11810 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.660940  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.587276 11819 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.672091  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.672858  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.685801  6382 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:09.674592  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.675491  6376 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:09.688643  6380 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:27:09.685717  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.689787  6379 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.690786  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.691907  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.692975  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.693547 11941 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:09.694005  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.696776 11814 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:09.697256 11814 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:09.698462 11750 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:09.700642  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:09.700845  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.701488 11927 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:09.702199  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.702843  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.703480  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.703612  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.704164  6377 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:09.705015  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.706933 11927 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:27:09.707476  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:09.708148 11902 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:09.708573 11902 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:09.708876 11909 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.709074 11745 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:09.709408 11902 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.707477 11915 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.710759 11902 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:09.685321 11924 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.713296 11915 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.713994 11924 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.715991 11915 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.716203 11924 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.719717 11739 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:09.720332 11739 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:09.698253 11814 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.723132 11876 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:09.723757 11876 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:09.724804 11876 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.725914 11814 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.727375  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:09.635687 11823 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:09.624974 11797 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:27:09.737650  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:09.741775  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.743044  6381 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:09.742377  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.659981 11895 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.755548  6450 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:09.741580  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.745333  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.747071  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.760021  6456 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:09.750771  6457 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:09.755178  6454 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:09.760622  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:09.762392  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.764866  6379 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:09.767455  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.768900  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.770138  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.754596 11823 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:09.772359 11929 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:27:09.773408  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.825155  6381 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:09.826028  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.827126  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.780496 11810 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.773937 11895 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.772408  6378 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:09.679250 11847 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.780329 11819 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.726399 11876 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:09.714388 11757 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.711211  6200 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:09.686690 11860 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:09.791222  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.792098  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.792877 11536 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:09.793949 11935 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.802232  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.887356  6380 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:09.886921 11935 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:09.888509  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.889210  6380 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 8.
I20260709 12:27:09.889861 11935 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.804039  6376 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:09.891597  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.893066  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.894009  6376 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:27:09.805033 11627 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:09.805261  6310 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:09.807389  6274 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.753s (SENT)
I20260709 12:27:09.808897  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.810312 11423 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:09.810925 11940 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:09.895337 11627 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:09.811826  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56425.2us
W20260709 12:27:09.813599  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.814661  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.897217  6456 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:09.897938 11895 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:09.895686  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:09.899322  6456 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:09.900135  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.900592  6456 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:27:09.815711  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.901706  6451 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
W20260709 12:27:09.815976  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.816628  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.904058  6458 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:09.817298  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.807533  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:09.778228  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:09.819950  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 188980us
I20260709 12:27:09.821880 11902 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:09.823446 11913 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.823683 11823 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:09.828559  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.962167  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.829851  6382 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
W20260709 12:27:09.830744  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.828799  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.835731  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.664584 11861 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.847718  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:09.830065 11826 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.001s
I20260709 12:27:09.861775  6385 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:09.862703  6453 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:09.969913  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:09.867870 11847 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:09.876909 11869 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.890117  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:09.973330  6380 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:09.975898  6380 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:27:09.906311  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 98978.4us
I20260709 12:27:09.905143  6450 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:09.906941 11915 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:27:09.936372  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.905864 11750 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:09.939404  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:09.906153 11927 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:09.939697  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.008513  6457 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:09.905306 11917 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:09.940493  6455 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:09.941011  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:09.943063  6452 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:09.944628  6456 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
W20260709 12:27:09.957939  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 179778us
I20260709 12:27:09.963930  6376 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:09.961609  6377 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:09.967330  7317 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:09.968818  6382 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:09.966531  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:09.977764  6274 leader_election.cc:336] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.888s (SENT)
I20260709 12:27:09.942679 11819 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:10.016589  6380 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:10.016520 11819 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.017264  6385 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:27:10.017632  6385 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:27:10.017836  6385 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:27:10.018065  6385 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:10.018675 11861 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:10.018929 11819 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:09.983141  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.021279  6459 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:10.021476  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:10.022840  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:10.024562 11823 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.025007 11869 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:10.025187 11913 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:10.025571 11913 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.025897 11913 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:10.027522  6200 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 9 to 10, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 10 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:09.942704 11902 consensus_queue.cc:1243] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:10.029314 11902 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.029815 11902 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.938292 11881 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:27:10.031836 11902 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:09.965317 11892 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.626s	user 0.012s	sys 0.001s
I20260709 12:27:09.998935  6383 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:10.005944  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.006568  6379 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:10.025031 11823 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:09.895816 11627 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:10.039367  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.014570  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:09.965590 11895 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:10.014730  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73811.4us
I20260709 12:27:10.004755 11808 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:27:10.015574  6274 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.888s (SENT)
I20260709 12:27:10.019150 11861 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:10.018173  6380 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:27:10.045717  6380 pending_rounds.cc:85] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 5
I20260709 12:27:10.043834  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:10.046408  6380 pending_rounds.cc:107] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:27:10.046815  6380 raft_consensus.cc:2889] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:10.048934  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.050009  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:10.037038  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70613.2us
I20260709 12:27:09.969092 11935 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:10.051966  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.052964  7329 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:10.041074  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:10.014863  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 813) took 1144 ms. Trace:
I20260709 12:27:10.035653 11823 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:10.045037  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.052369  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:10.058156  6455 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:10.059140  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.059877  6455 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:10.053815  6302 rpcz_store.cc:276] 0709 12:27:08.870831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.026185 (+155354us) service_pool.cc:224] Handling call
0709 12:27:09.113125 (+ 86940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.113398 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.113435 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.113973 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.301221 (+187248us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:09.301273 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.301363 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.711828 (+410465us) raft_consensus.cc:1685] finished
0709 12:27:09.711859 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.991643 (+279784us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b54002071d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:10.014790 (+ 23147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62375808,"thread_start_us":141872,"threads_started":1,"wal-append.queue_time_us":510169}
I20260709 12:27:10.021220 11739 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:10.067256  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.043864 11936 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.338s	user 0.002s	sys 0.002s
I20260709 12:27:10.045650 11861 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:10.045791 11790 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.001s
I20260709 12:27:10.044735 11808 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:10.040714 11868 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:09.991863 11826 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.069517  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:10.070210  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54668.9us
W20260709 12:27:10.073177 11928 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.355s	user 0.001s	sys 0.001s
I20260709 12:27:10.075107  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.045122 11915 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:10.060760 11935 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.043749 11946 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.091212  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:10.091903  6310 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.061471 11947 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:10.096794  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:10.077245 11810 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:10.062129 11927 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:10.094139  6309 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:10.091253  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:10.107558  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:10.107647  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:10.074134  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.095157  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:10.110164  6384 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:27:10.082654  6274 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.969s (SENT)
I20260709 12:27:10.112010  6300 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:10.118085  6300 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:10.100543  7314 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:10.120728  6309 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.101302  6307 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:10.119589  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.102392  6301 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:10.124223  6459 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:10.125133  6308 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:10.102504  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:10.126829  7331 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.095855  7318 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:10.112928 11825 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:10.131315 11825 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.131807 11825 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.132648  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:10.133859 11891 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:10.134183 11867 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:10.134359 11891 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:10.054626 11536 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:27:10.082356  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.134887  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:10.113405 11869 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:10.077556 11917 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:27:10.116204 11947 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:10.116372  6302 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:10.087016 11627 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.148514 11944 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:10.149168 11627 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.150148 11757 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:10.118310 11826 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:10.154239  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.093022 11868 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:10.088728 11924 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:10.076040 11895 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:27:10.119580 11927 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.188146  6302 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:10.184162  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:10.119820  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:10.137354  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:10.076803 11823 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:10.142217  6310 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:10.131227 11921 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.433s	user 0.002s	sys 0.000s
W20260709 12:27:10.144455  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70393us
I20260709 12:27:10.148116  6309 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:10.149734 11891 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.151115  6453 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:10.151445  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:10.152527 11938 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.454s	user 0.000s	sys 0.003s
I20260709 12:27:10.118261 11814 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:10.143468 11936 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:10.180202  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:10.082387 11819 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:27:10.094478 11423 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.180984  6300 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:10.185580  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:10.186067  7331 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:10.134687 11867 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:10.191354  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.230343  6457 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:10.197310 11935 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:10.232234  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.232864  6451 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:10.197456  6381 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:10.198020 11947 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.200840 11902 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:10.203934  6385 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:10.235948  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.280436  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.206547 11915 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:27:10.283763  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:10.285162 11915 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.285596 11915 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.285980 11915 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.286247  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:10.210021  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55846.4us
W20260709 12:27:10.288266  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51097.6us
I20260709 12:27:10.288417  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.289402  6302 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:10.143332 11869 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.213852 11845 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.149137 11944 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:10.194722 11927 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:10.234545  6381 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:10.207315  6200 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:10.286653 11797 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.209750  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.209470 11873 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.281392  6300 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:10.229420 11924 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:10.297039  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.214630 11814 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:10.306857  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.327163  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:10.306365  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:10.310633 11936 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:10.334862  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:10.311483 11930 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.612s	user 0.003s	sys 0.000s
I20260709 12:27:10.312317  6200 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 2 to 9, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 9 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:10.299378  6381 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.306537 11826 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:27:10.238289  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:10.317057  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 107401us
I20260709 12:27:10.347293  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:10.223459 11941 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:27:10.315218  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.296221 11750 consensus_queue.cc:1243] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:27:10.319705  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 769) took 1018 ms. Trace:
I20260709 12:27:10.295504 11869 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:10.328931  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:10.227787 11575 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:27:10.235332 11788 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:27:10.302632 11940 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:10.318737  6300 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:10.360286  7332 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:10.365113  6381 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:27:10.327672 11931 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.630s	user 0.003s	sys 0.001s
I20260709 12:27:10.366142  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.316493 11885 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:10.368839  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.325093 11739 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:10.326053 11876 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:10.338717  6378 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:27:10.365258  7319 rpcz_store.cc:276] 0709 12:27:09.301471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.302461 (+   990us) service_pool.cc:224] Handling call
0709 12:27:09.302789 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.303400 (+   611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.303429 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.326399 (+ 22970us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.582409 (+256010us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:09.582534 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.582625 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.151906 (+569281us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:10.306211 (+154305us) raft_consensus.cc:1685] finished
0709 12:27:10.306232 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.319624 (+ 13392us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1866,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7881,"spinlock_wait_cycles":220427136,"thread_start_us":255027,"threads_started":1,"wal-append.queue_time_us":371707}
I20260709 12:27:10.303016 11919 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:27:10.412074 11750 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.437449 11750 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:10.437932 11750 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.372146 11892 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.338074 11886 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:27:10.429256  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63217.9us
I20260709 12:27:10.346293 11790 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:10.334710 11923 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.425475 11924 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.452358 11949 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.463415 11876 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:10.322954 11797 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.327124 11873 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.327839 11868 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.339915 11810 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 12:27:10.424867 11885 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:10.412473 11933 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.713s	user 0.000s	sys 0.005s
I20260709 12:27:10.451018 11941 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:10.428306 11944 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.430373 11913 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.453109 11861 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:10.451653  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82889us
I20260709 12:27:10.540803  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:10.447526 11950 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:27:10.474377  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2347) took 1845 ms. Trace:
I20260709 12:27:10.476012  7319 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:10.376619 11946 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:10.478818 11808 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.481353 11823 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:10.488039 11814 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:10.491011 11951 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.147s	user 0.000s	sys 0.002s
I20260709 12:27:10.491320 11610 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:10.492136  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:10.513645 11634 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:10.504073 11871 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
W20260709 12:27:10.521490  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 176973us
W20260709 12:27:10.531898  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 347784us
I20260709 12:27:10.538575 11750 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.538960  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:10.540957  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 239972us
I20260709 12:27:10.575024  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:10.543558  6304 rpcz_store.cc:276] 0709 12:27:08.628803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.628932 (+   129us) service_pool.cc:224] Handling call
0709 12:27:08.629258 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:08.658358 (+ 29100us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.658390 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:08.658968 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.267783 (+608815us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b1800401500. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:09.528511 (+260728us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:09.528604 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.528776 (+   172us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.295660 (+766884us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:10.474052 (+178392us) raft_consensus.cc:1685] finished
0709 12:27:10.474085 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.474292 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1264,"raft.queue_time_us":29095,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":12896,"spinlock_wait_cycles":249236224,"thread_start_us":259472,"threads_started":1,"wal-append.queue_time_us":1037859}
I20260709 12:27:10.548487  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:10.576514  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.556308  6457 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:27:10.577205  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 85215.1us
I20260709 12:27:10.566018  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:10.514772 11790 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:10.527866 11885 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:10.529874 11941 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:10.594741 11941 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.595218 11790 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.576658  7329 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:10.510715 11923 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.584182 11423 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:10.599221 11923 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:10.517673 11886 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:10.577634  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.601426  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.604584 11810 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:10.605026 11345 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:10.605363 11810 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:27:10.606048 11810 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:10.611550 11757 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:10.612187 11757 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.573058 11634 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:10.615962 11634 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.617460 11634 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.616715 11867 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.618786 11881 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:10.619220 11881 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:10.619987 11873 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:10.620100 11881 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.621426 11881 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.579596  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:10.623114  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 43618.3us
I20260709 12:27:10.601156 11915 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.605471 11345 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.585207  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:10.636219  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51125.6us
I20260709 12:27:10.637880  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.625092  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:10.634882  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.640010  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74075us
I20260709 12:27:10.641703  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.562952 11823 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:10.644466  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.568753 11610 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:10.569794 11946 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:10.674827  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.692807  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:10.624662 11867 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:10.698364  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56769.8us
I20260709 12:27:10.625123 11757 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:10.595942 11797 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:10.641265 11345 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.630564 11810 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:27:10.625074 11952 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:10.672809 11923 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:10.641763 11961 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:10.695814 11960 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 12:27:10.696056 11925 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:27:10.692059 11802 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.756008  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 768) took 1454 ms. Trace:
I20260709 12:27:10.690434 11814 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:10.699990 11873 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.719205 11423 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:10.701164 11826 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.766844  7316 rpcz_store.cc:276] 0709 12:27:09.301193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.302347 (+  1154us) service_pool.cc:224] Handling call
0709 12:27:09.312455 (+ 10108us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.312845 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.312874 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.313375 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:27:09.487072 (+173697us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:09.487113 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.487178 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.195297 (+708119us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:10.731169 (+535872us) raft_consensus.cc:1685] finished
0709 12:27:10.731195 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.755926 (+ 24731us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1663,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":4324,"spinlock_wait_cycles":220330752,"thread_start_us":173294,"threads_started":1,"wal-append.queue_time_us":385345}
I20260709 12:27:10.730250 11886 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:10.752221 11917 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:10.721665 11536 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:27:10.754346 11961 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.700184 11891 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:27:10.755959 11739 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:27:10.795760 11919 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:10.796358 11919 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:10.767371 11927 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.797333 11919 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.798671 11927 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:10.803289 11913 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:10.815989  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 191797us
W20260709 12:27:10.819330  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 119876us
I20260709 12:27:10.667908 11876 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.824829  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.824643  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:10.813884 11968 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:10.832439  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:10.746927 11963 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:27:10.786850 11917 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:10.768082 11845 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:27:10.847463  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2483d7801887403e873adc62429fe046"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:10.853639  6454 raft_consensus.cc:606] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:10.847757  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:10.851768  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:10.805699 11886 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:10.810262 11961 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.834309 11557 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.873425 11964 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:27:10.875560 11970 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:27:10.833812 11739 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:10.887509 11861 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:10.895027 11915 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:10.827950 11877 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 12:27:10.777738 11610 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.860347 11923 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:10.926978  6451 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:10.926782 11886 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:10.859248 11962 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.928936 11955 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:10.929783  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 291969us
I20260709 12:27:10.930675 11871 raft_consensus.cc:2764] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:27:10.932859  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.775956 11873 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.935508  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.618840  6200 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 2 to 9, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 9 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:10.849318 11826 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:10.939407  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:10.940289  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:11.009532  6455 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:11.011518  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 266351us
I20260709 12:27:10.857045 11917 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:10.950695  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260709 12:27:10.951710  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 132858us
W20260709 12:27:11.027191  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.027379  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:11.027664  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.857992 11919 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.028306  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:11.029045  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:11.029642  6450 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:10.902865 11940 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:10.956506  6310 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:10.780336 11823 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:10.953697 11808 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:10.950248 11634 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:10.982232  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 291505us
I20260709 12:27:11.003475  6454 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:11.003885  6457 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
W20260709 12:27:11.042851  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45105.3us
I20260709 12:27:11.044119  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:11.045629  6307 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 12:27:10.966866 11869 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
W20260709 12:27:11.030742  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.022006  6455 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:10.996071 11875 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:10.944581 11557 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.007597 11750 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.006256 11962 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.036535 11952 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:11.079002  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.079247  6385 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:11.073808  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.070694  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.069144 11869 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:11.095849 11885 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.001s
I20260709 12:27:11.067449 11634 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.079084 11826 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:11.032178 11940 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:11.063809  6382 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:11.072568  6452 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:10.860877 11946 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:11.090548 11971 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:27:11.040688 11927 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.075915  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.064839  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:11.119468  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75448.9us
I20260709 12:27:11.031912 11423 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:11.172755 11423 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.175529 11423 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.150102 11757 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.126164  6384 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:11.139921  6450 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:11.066452 11871 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 12:27:11.110991 11825 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.979s	user 0.009s	sys 0.005s
I20260709 12:27:11.148643  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:11.108774  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.146713  6378 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 8.
I20260709 12:27:11.150593  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:11.147442  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:11.069993 11924 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:27:11.159101  6310 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:11.126624 11902 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:11.118310  6451 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:11.160442 11891 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.191351  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:11.195674  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:11.197736 11976 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
W20260709 12:27:11.203662  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56326us
W20260709 12:27:11.204672  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54199.2us
I20260709 12:27:11.216979  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:11.218858  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:11.220244 11944 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:11.220930 11944 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:11.234757  6382 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:11.238440 11944 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:11.163221 11536 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:27:11.164752 11826 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.183501 11927 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.162309 11885 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:11.248481  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76864us
I20260709 12:27:11.187472 11845 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.249737 11941 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.000s
I20260709 12:27:11.315899  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:11.242882 11866 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.246161 11947 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:11.246555 11876 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:11.287209 11873 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.304968 11913 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:11.192221 11757 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.267263 11955 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:11.334549  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.199991 11825 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.338404  6456 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:11.337337  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 140924us
I20260709 12:27:11.303826 11944 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:11.281111 11750 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:27:11.314999 11871 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:11.243338 11891 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.361657  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:11.342749 11913 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.331800 11876 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:11.317523 11902 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:27:11.349848 11873 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:11.385362  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 189748us
I20260709 12:27:11.281592 11952 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.322311 11941 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.379212 11750 consensus_queue.cc:1243] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 13.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:11.326571 11845 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.382779 11913 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:11.376621 11999 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:11.382987 11876 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:11.385738 11634 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.388931 11869 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.386719 11557 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.411286  7332 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:11.418831 11996 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.419693 11984 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 12:27:11.419862 11996 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:11.421440 11990 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:27:11.422535 11991 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.423125 11991 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.425359 11991 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.426705 11999 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:11.431301 11995 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:27:11.431937 11757 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:11.436848 11814 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.000s
I20260709 12:27:11.400168 11536 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:11.394416 11873 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:11.354965 11955 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.419220 11845 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.406397 11750 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:27:11.415668 11881 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.794s	user 0.010s	sys 0.007s
I20260709 12:27:11.385344  6200 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:11.405741 11962 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.372057 11998 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:27:11.393292 11994 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:27:11.471218 11345 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.713s	user 0.011s	sys 0.003s
I20260709 12:27:11.471768 11994 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:11.471679 11345 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.472244 11994 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:11.473181 11994 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.416851 12000 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:11.467908 11988 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:27:11.474735 12000 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.478348  6382 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:27:11.467994 12001 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:27:11.489022  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.489653  6304 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:11.490644  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.494848  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:11.495446  6305 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:11.497537 11876 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.464985 11750 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:11.504632  6305 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
W20260709 12:27:11.510094  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 148537us
W20260709 12:27:11.495208 11810 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.728s	user 0.014s	sys 0.000s
I20260709 12:27:11.479856 11345 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.520309  6308 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:11.408579 11944 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.462620 11536 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.516177  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.527122  6302 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:11.528769  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:11.533912  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.534188  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.535279  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.535615  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.536326  7335 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:11.536556  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.537735  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.539865  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.540575  7335 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:11.507968 12003 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.513521 11814 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.525161 11802 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.003s
I20260709 12:27:11.513834 11927 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.517503 11886 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.546540  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.551713  6379 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 12:27:11.573098  6379 pending_rounds.cc:85] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 4
I20260709 12:27:11.573437  6379 pending_rounds.cc:107] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:27:11.573799  6379 raft_consensus.cc:2889] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:11.523788 11885 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.556555  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.556692  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.568145  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:11.569478  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.569550 11941 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.570677  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.532032 11991 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:11.553869  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.575680  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.576700  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.578274  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.580574  7316 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:11.543895 11634 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.585743  6308 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:11.575433  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.675007  6302 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:11.522477 11810 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.677157  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.711606  6377 raft_consensus.cc:2376] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13. Re-sending same reply.
I20260709 12:27:11.711796  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.574390 11802 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:11.653419  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:11.713860  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60588.8us
I20260709 12:27:11.681574  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.570160 11996 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.682792  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.575894 11871 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:11.683892  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:11.579121 11886 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.691754 12006 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.678821 11845 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:11.732350  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 155247us
I20260709 12:27:11.694769 12004 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.228s
I20260709 12:27:11.695622 12010 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:11.695084  6200 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 9, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 9 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:11.743911  6302 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:11.745428  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:11.696245 11982 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:11.544051 11999 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
W20260709 12:27:11.655201  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.560967 11814 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.713109 11634 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.675608 11891 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.651626  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.764582  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:11.697839 11867 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.968s	user 0.009s	sys 0.005s
I20260709 12:27:11.714733  6378 raft_consensus.cc:2376] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14. Re-sending same reply.
I20260709 12:27:11.714993  6376 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:11.727839  7329 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:27:11.729211  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 341019us
I20260709 12:27:11.729564  6301 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:11.725006  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:11.730144  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:11.768407  7316 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:11.550912 11873 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.572161 12003 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.733645  6309 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:27:11.732206  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.734221  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:11.732649  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:11.731941  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.734648  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.758309 11886 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.739688  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:11.776367  6310 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:11.753665 11892 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:11.739989 11345 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.748121  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:11.783252  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.736436 11788 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:27:11.589254 12000 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.730822 11997 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:11.732216 11575 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:11.682109 11991 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.739655  6308 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:11.740450  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.778640  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:11.769311 11961 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.828s	user 0.012s	sys 0.001s
I20260709 12:27:11.780856  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.741534 11919 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:27:11.746417 11845 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.791968  6301 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:11.754762 12006 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.776443 11876 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.784884 11869 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:11.807750  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.827999  6378 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:11.808065  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:11.814688  7316 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:11.790364 11997 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.817341  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:11.795189  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47171.3us
I20260709 12:27:11.833694  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:11.836264  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.787715 11788 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.768507 11982 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.787259 11892 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.803803 11961 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.795341 11575 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.793231 11871 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:11.803339 12011 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:11.822342  6310 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:11.836923  6304 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:11.841365  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:11.836865  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.851018  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:11.872768  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:11.882299  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:11.882871  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49248.3us
I20260709 12:27:11.889137  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:11.788463 11345 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.895532  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:11.843681 12014 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:27:11.903853  6384 raft_consensus.cc:2368] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
W20260709 12:27:11.844954 12018 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:27:11.844256 12015 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:27:11.824473 11536 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.901758  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:11.901942  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:11.840404 11997 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:11.842656 12016 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.909050  6458 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:11.830917 12019 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.844401 11557 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:11.914968  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:11.940390  6457 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:27:11.838424 11634 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.846304 11845 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.839798 11991 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.841665 12025 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:11.882328 11788 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:11.917261  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:11.884955 12011 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.906982  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:11.974431  6459 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:11.926188  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:11.975621  6377 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:11.901695 11923 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:11.904569 11826 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:11.916086 11946 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:11.893558 11941 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:27:11.842572 11993 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.191s
I20260709 12:27:11.910758 11869 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 12:27:11.981515  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.966305 11867 raft_consensus.cc:2764] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:11.945299 11810 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.982685 11992 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:27:11.983835 12020 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 12:27:11.991743  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101292us
I20260709 12:27:11.977739 12028 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.988288 11814 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.975520 12011 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.001085  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:11.999126  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.006348  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:11.991655 11913 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.004s
I20260709 12:27:11.989326 11557 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.000200 11808 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.951s	user 0.007s	sys 0.008s
I20260709 12:27:11.991338 11861 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:12.028784 11575 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.028824 11861 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.030689 11557 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.032500 11861 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.033118 11557 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.010865 11913 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.036707 11913 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.037320  6300 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 12:27:12.011781  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 110113us
I20260709 12:27:12.007272 12028 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.014930 11797 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:12.023123 11955 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.023458 12011 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:11.998225 11946 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:11.968881 12019 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.024220 12006 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:11.964915 12016 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.042650  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:12.040899  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:12.094904  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54124.7us
I20260709 12:27:12.095096  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.095912  6385 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:12.047148  6459 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:12.096805  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.097234 12019 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.097759  7329 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:12.098737 11992 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:12.099247 11992 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.099632 11992 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.055823  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.102458  6455 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:12.103271  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.103686 12011 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.060549  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.105645 11892 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:11.977406 12025 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.090334 11961 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.044373 11797 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.010638 11885 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:27:12.043718 11873 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.092572 11345 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.105073 11946 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.091259 11982 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.106437  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.102459  7316 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:12.094396  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.123464  6377 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:27:12.067983 11997 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.113626  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:12.126978  7331 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.114705  7319 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:12.116145  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 198796us
I20260709 12:27:12.124554  6385 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:12.131366  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:12.132200  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:12.132339  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.133749  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:12.134707  6459 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:12.138144  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.141265  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.142050  6310 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:12.115527 11797 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:12.144769 11917 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.103s	user 0.015s	sys 0.000s
I20260709 12:27:12.144999  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:12.146338  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:12.157398  6385 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:12.157197 11917 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.160734  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.160811  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.161633  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.161947  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:12.161676  6310 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:12.166036  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.168932  7331 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:12.170288  6308 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:12.167430  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:12.144505 11819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.037s	user 0.000s	sys 0.038s
I20260709 12:27:12.178186  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:12.174114  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.176676  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:12.145462 11345 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.177258  6307 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:27:12.157817 11989 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:12.179268  6310 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:12.179132  6377 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:12.120329 11873 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.168412 12016 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.182808  6376 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:12.158663 11913 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:27:12.175670 12028 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.186138  6308 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:27:12.186722 12039 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:12.186626 11941 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:12.187392  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:12.189226  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:12.178969 11819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.072s	user 0.000s	sys 0.039s
W20260709 12:27:12.182225 11915 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.092s	user 0.014s	sys 0.023s
I20260709 12:27:12.172283 12040 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:12.191285  6304 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:12.183782 11575 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:12.206552 12040 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.206825 11575 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:12.207010 12040 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:12.187387 11986 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.028s	user 0.006s	sys 0.056s
I20260709 12:27:12.197496 11825 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.201246  6454 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:12.194294  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:12.207285 11575 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:12.183588 11345 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.213466  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 780) took 1578 ms. Trace:
I20260709 12:27:12.210202  6385 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:12.222051 11750 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:27:12.216344 11982 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:12.205353 12039 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.244554  7332 rpcz_store.cc:276] 0709 12:27:10.635096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.635280 (+   184us) service_pool.cc:224] Handling call
0709 12:27:10.678201 (+ 42921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.414519 (+736318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.213170 (+798651us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54003bc4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:12.213385 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685431040}
I20260709 12:27:12.251705  6385 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:12.218127 12004 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
W20260709 12:27:12.222416 11986 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.129s	user 0.007s	sys 0.056s
I20260709 12:27:12.212610 11823 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:12.254155 12039 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.267781  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:12.207731 11915 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.117s	user 0.014s	sys 0.024s
I20260709 12:27:12.257575 11917 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.259311  6385 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:27:12.243251 11913 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.254881  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:12.211036 12025 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.269951  6454 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:12.271698 11913 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.268225 11750 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:12.269344 11985 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.175s	user 0.001s	sys 0.004s
W20260709 12:27:12.298019 11987 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.000s	sys 0.007s
I20260709 12:27:12.286441 11913 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.295881 11345 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:12.317129 11987 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.000s	sys 0.008s
I20260709 12:27:12.371426 11797 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.523041 11797 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:12.524194 12051 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:27:12.519275 11788 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.004s
I20260709 12:27:12.529721  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:12.530661  6309 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.537955  6309 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:12.542781  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:12.548746  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.099s (SENT) [suppressed 19 similar messages]
I20260709 12:27:12.563145  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2382) took 1739 ms. Trace:
I20260709 12:27:12.563694  6303 rpcz_store.cc:276] 0709 12:27:10.823726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.823920 (+   194us) service_pool.cc:224] Handling call
0709 12:27:10.824325 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.032691 (+208366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.032736 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.033267 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.135100 (+101833us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:11.135145 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.546716 (+1411571us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b54002e8454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:12.546787 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:12.546849 (+    62us) raft_consensus.cc:1685] finished
0709 12:27:12.546870 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:12.563063 (+ 16193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31180,"raft.queue_time_us":37263,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":17265,"spinlock_wait_cycles":417751424,"thread_start_us":101525,"threads_started":1,"wal-append.queue_time_us":250414,"wal-append.run_cpu_time_us":2854,"wal-append.run_wall_time_us":1133595}
I20260709 12:27:12.576889 11610 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.528282 12040 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.555987 11819 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:12.560483 11797 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:12.584638  6385 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:27:12.533973 11788 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.578125 11917 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:12.601791  6455 raft_consensus.cc:2368] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
W20260709 12:27:12.605912 12054 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:12.576256 11992 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.599459 11797 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:12.591765 11819 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.601107 11917 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:12.619086  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70387.4us
I20260709 12:27:12.622016 11423 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.622452 11982 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:12.620877 11825 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:12.634793  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:12.639868  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 875) took 1715 ms. Trace:
I20260709 12:27:12.640081  6300 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:12.640324  6456 rpcz_store.cc:276] 0709 12:27:10.924056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.932262 (+  8206us) service_pool.cc:224] Handling call
0709 12:27:10.948015 (+ 15753us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.350059 (+402044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.635128 (+1285069us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54003c78d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:12.639801 (+  4673us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897146112}
I20260709 12:27:12.642827 11917 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.647046 11423 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.651983 12040 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.650266 11423 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.662891  6300 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
W20260709 12:27:12.665165 11739 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.596s	user 0.017s	sys 0.055s
W20260709 12:27:12.667632 12056 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:27:12.664512 11788 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:12.657131 11825 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.654917 11982 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.707214 11949 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:12.639377 11797 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.730876  6305 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:12.706964 11941 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.711910 11739 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.639s	user 0.017s	sys 0.055s
I20260709 12:27:12.741519 12039 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.742676 11345 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:12.743503 11949 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.746621 11996 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.753861 11992 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.754043  6378 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:27:12.782086  6378 pending_rounds.cc:85] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 8
I20260709 12:27:12.782397  6378 pending_rounds.cc:107] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 12.9
I20260709 12:27:12.782809  6378 raft_consensus.cc:2917] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Aborting config change with OpId 12.9: Aborted: Op aborted by new leader
I20260709 12:27:12.782696 11996 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.783308  6378 raft_consensus.cc:2889] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:12.783872  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 0) took 2547 ms. Trace:
I20260709 12:27:12.784176 11949 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:12.773689 11825 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:12.785368  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2389) took 1297 ms (client timeout 1695 ms). Trace:
I20260709 12:27:12.784211  6378 rpcz_store.cc:276] 0709 12:27:10.288773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.296613 (+  7840us) service_pool.cc:224] Handling call
0709 12:27:10.456169 (+159556us) log.cc:844] Serialized 485 byte log entry
0709 12:27:10.912776 (+456607us) spinlock_profiling.cc:243] Waited 38.5 ms on lock 0x7b540033f020. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3220142 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634 0000000000388479 00007f6fa01ba475
0709 12:27:10.928670 (+ 15894us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b18002fefc0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa717b58f 00007f6fa01ba571 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd
0709 12:27:11.099867 (+171197us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b18002fefc0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:27:12.783745 (+1683878us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131735,"raft.queue_time_us":49243,"raft.run_cpu_time_us":3948,"raft.run_wall_time_us":372682,"spinlock_wait_cycles":127965184,"thread_start_us":106200,"threads_started":1,"wal-append.queue_time_us":291551,"wal-append.run_cpu_time_us":3745,"wal-append.run_wall_time_us":1289394}
W20260709 12:27:12.785810  6307 rpcz_store.cc:269] 0709 12:27:11.488041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.490322 (+  2281us) service_pool.cc:224] Handling call
0709 12:27:12.785199 (+1294877us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54002062d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:12.785309 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1577356544}
I20260709 12:27:12.791450 11994 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:12.791920 12039 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:12.756846 11575 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.510s	user 0.006s	sys 0.003s
I20260709 12:27:12.757580  6381 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:12.758459 11814 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:12.793129 11575 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.793531 11814 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:12.794412 11814 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.795200 11575 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:12.795641 11814 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:12.759362 11819 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.761260 11982 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.757229 12007 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.766199  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:12.758203 12004 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.769201  6303 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:27:12.798247 12007 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.770241 11788 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.770319 11980 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.770871 11981 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.680s	user 0.010s	sys 0.001s
I20260709 12:27:12.771117 12000 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:12.798511 12004 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.804219 11980 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.804701 12007 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:12.805233  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3106) took 2255 ms. Trace:
I20260709 12:27:12.805593 12004 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.806659 11980 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.805651  6453 rpcz_store.cc:276] 0709 12:27:10.549827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.550790 (+   963us) service_pool.cc:224] Handling call
0709 12:27:10.551209 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.559327 (+  8118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.559372 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.566365 (+  6993us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.986572 (+420207us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:10.986631 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.986730 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:12.770236 (+1783506us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:12.804874 (+ 34638us) raft_consensus.cc:1685] finished
0709 12:27:12.804913 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:12.805142 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65759,"raft.queue_time_us":12405,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3991,"spinlock_wait_cycles":76188928,"thread_start_us":321240,"threads_started":1,"wal-append.queue_time_us":504468}
I20260709 12:27:12.807682 12000 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.809767 11873 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.771502  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.810684  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:12.810846 11980 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:27:12.765753 11941 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.811539 11980 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:12.813122 11997 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.813267 11980 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.813928 11757 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:12.814373 11757 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.814755 11757 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:12.798426  7332 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:12.811254  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.809823 12000 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.771787  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 795) took 1384 ms. Trace:
I20260709 12:27:12.818480  6382 rpcz_store.cc:276] 0709 12:27:11.387827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.388015 (+   188us) service_pool.cc:224] Handling call
0709 12:27:11.388543 (+   528us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.478944 (+ 90401us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.771507 (+1292563us) spinlock_profiling.cc:243] Waited 89.3 ms on lock 0x7b54003e6a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:12.771719 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205352704}
I20260709 12:27:12.821794  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:12.822695  7331 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:12.829504  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.830338  7331 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:12.837509  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.838189  6307 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:12.846568  7332 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:12.853683  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:12.771019 12009 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.854404  6301 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:12.854765  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.805709 11345 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:12.813127 11941 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:12.860495  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.802666 11944 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:12.860498  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.835314 12004 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:12.857393  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.863929  7331 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:12.863888  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:12.862823  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.855448  7332 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:12.861197  6300 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:12.865211  6304 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:12.874449  6304 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:12.881698  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.885040 11739 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.885704 11739 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.890213 12025 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.828574 11788 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.890877 12025 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.835585 11963 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:27:12.893360 11788 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.901388  6301 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:12.908175  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2708) took 1881 ms. Trace:
I20260709 12:27:12.908982  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2879) took 1371 ms. Trace:
I20260709 12:27:12.908658  6451 rpcz_store.cc:276] 0709 12:27:11.026566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.027720 (+  1154us) service_pool.cc:224] Handling call
0709 12:27:11.052970 (+ 25250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.193238 (+140268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.907923 (+1714685us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b54002c9550. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:12.908103 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56961792}
I20260709 12:27:12.909415  6376 rpcz_store.cc:276] 0709 12:27:11.537763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.789364 (+251601us) service_pool.cc:224] Handling call
0709 12:27:11.802760 (+ 13396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.185075 (+382315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.908724 (+723649us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b540023ebd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:12.908921 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":873540992}
I20260709 12:27:12.913242  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.913493  7319 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:27:12.915400  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.916632  6301 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:12.918691  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.919512  7316 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:12.910406 11739 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:12.858603 11345 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:12.858119 12009 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.863587 12004 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.930573  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.936053  6304 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:12.929404  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.937484  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.931665  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:12.934074  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.938854  7316 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:12.940788  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.940016  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.946408  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2876) took 1409 ms. Trace:
I20260709 12:27:12.946988  6379 rpcz_store.cc:276] 0709 12:27:11.537248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.777787 (+240539us) service_pool.cc:224] Handling call
0709 12:27:11.777998 (+   211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.817185 (+ 39187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.817212 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.817627 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 12:27:11.944236 (+126609us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:11.979958 (+ 35722us) log.cc:844] Serialized 14 byte log entry
0709 12:27:12.942901 (+962943us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b180039aba0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:27:12.943081 (+   180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.943157 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:12.943199 (+    42us) raft_consensus.cc:1685] finished
0709 12:27:12.946115 (+  2916us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:12.946308 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5273,"prepare.queue_time_us":35720,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":3027,"raft.queue_time_us":27290,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":404,"spinlock_wait_cycles":8050432,"thread_start_us":161852,"threads_started":2,"wal-append.queue_time_us":193442}
I20260709 12:27:12.945556  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.951141  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.951785  7316 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:12.953336  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:12.937500 12004 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:12.951656  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:12.964636  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:12.965704  6453 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:12.966959 11941 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.967730 11941 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.961213 12000 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:12.975155 12000 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:12.976329 12000 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.963598 12012 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:12.979187 12012 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:12.980307 12012 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.961478 11868 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.983984 11868 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:12.940806 11963 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:13.011140 11963 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:13.011277  6274 leader_election.cc:336] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.992s (SENT)
I20260709 12:27:13.011631 11963 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:13.011742  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:13.012959 11997 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:12.988958  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:13.015234  6300 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:12.987179 11790 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.990741  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:12.994052  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.016738 11790 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.017432  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:12.994339  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:12.986986 11788 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:13.018365  6450 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.018451 11788 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.018891 11788 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:12.995074  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:12.995069  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.019829 11940 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.021508 11940 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.023924 11978 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:13.024606 11978 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.996596 12045 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.027668 12045 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.029052  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2399) took 1147 ms. Trace:
I20260709 12:27:13.029554  6305 rpcz_store.cc:276] 0709 12:27:11.881687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.881840 (+   153us) service_pool.cc:224] Handling call
0709 12:27:13.028841 (+1147001us) spinlock_profiling.cc:243] Waited 847 ms on lock 0x7b5400205dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:13.028972 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1948847488}
I20260709 12:27:13.032213  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:13.000358 12027 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.000443 11924 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:13.033871 11924 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.033831 12027 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:12.998530 11989 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.001566 11998 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.035614 11989 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:13.035988 11998 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.036568 11977 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:13.036484 11989 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.000597 11345 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.003453 12025 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:12.963919 11955 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.006390 11873 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:13.039093 11955 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.003003  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2531) took 1130 ms. Trace:
I20260709 12:27:13.040596  6378 rpcz_store.cc:276] 0709 12:27:11.872336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.561012 (+688676us) service_pool.cc:224] Handling call
0709 12:27:12.561767 (+   755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.796201 (+234434us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.002695 (+206494us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:13.002928 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441085184}
I20260709 12:27:13.008733 11739 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:13.044100 11345 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.010538 11869 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.952600 11892 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:13.045641 11892 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.034700 11927 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:27:13.009959  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2522) took 1947 ms. Trace:
I20260709 12:27:13.007858 12003 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.045492 11869 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.102409  6381 rpcz_store.cc:276] 0709 12:27:11.062477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.062620 (+   143us) service_pool.cc:224] Handling call
0709 12:27:11.062921 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.793127 (+1730206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.009629 (+216502us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54001f54d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:13.009878 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3896609664}
I20260709 12:27:13.103237  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:13.032418  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2861) took 1498 ms. Trace:
I20260709 12:27:13.105525  6384 rpcz_store.cc:276] 0709 12:27:11.533568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.533680 (+   112us) service_pool.cc:224] Handling call
0709 12:27:13.032184 (+1498504us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b5400038c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:13.032345 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848564992}
I20260709 12:27:13.039191 11873 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:13.106673  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.107326  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.108580 11873 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.050763 12022 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:13.055511 11996 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.054280 11739 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.049204 11868 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.060130  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:13.060325  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:13.063793  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:13.124522  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 72774.9us
W20260709 12:27:13.125380  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 61708.9us
I20260709 12:27:13.125522  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:13.128170  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.129877  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.069640  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:13.071192  6450 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:13.072160  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:13.072225  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:13.131270  6307 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:13.131748  6310 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:13.074891  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.075372 11978 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.074900 11998 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.075995 11955 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.076982 12009 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.083307 12045 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.055536 11790 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.093178  6383 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:13.059204 11924 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.086133 11941 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.100747 11994 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.104296 11869 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.046216 11892 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:13.104893 11886 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:13.106367  6274 leader_election.cc:336] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.093s (SENT)
W20260709 12:27:13.174948  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68627.4us
I20260709 12:27:13.175778  6458 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:27:13.115144  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:13.037497 11989 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:13.132560  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.133378  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58619.6us
I20260709 12:27:13.134464 12022 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:13.131403  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.134341 11996 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:13.140808  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.172387  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.147235 11993 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:13.136600  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 76408.6us
I20260709 12:27:13.138041 11814 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:13.183012  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51683.6us
I20260709 12:27:13.142400  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.366132  6450 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:13.138079 11345 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:13.137923 11557 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:13.171838  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:13.184434  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69389.4us
W20260709 12:27:13.181519  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2865) took 1646 ms (client timeout 2097 ms). Trace:
I20260709 12:27:13.149449 11998 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.176404 11994 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.195250 11788 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.183686 11886 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:13.199697  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 58960.7us
I20260709 12:27:13.184795 11739 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.391124  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.392601  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.218170 11993 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:13.191411 11757 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.167491 11986 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:13.191370 11866 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:27:13.170588 11955 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:27:13.184516 12022 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.184032 11941 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:27:13.450042  7332 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:13.167793 12007 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:13.201822 11869 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:13.359661  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:13.361227  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.168186 11868 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:13.134724 11873 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:13.366858  6451 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:13.371198  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:13.372840  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.409775 11978 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:13.457918  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.458946  6450 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:13.376304  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:13.460111  7319 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:13.379201  6453 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:13.387161  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:13.461810  6456 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.413857 11940 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:13.386718 11998 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.449604 12022 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.456504 11869 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:13.464700 11978 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.457027 11986 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:13.371572 11944 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.507s	user 0.015s	sys 0.003s
I20260709 12:27:13.375183 11790 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:13.408051  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.364677 11885 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:13.363929 11924 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:13.367707 12071 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:13.427093  6459 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:13.382161 11996 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.385067 11875 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.376315 11557 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.404687 12027 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.476230 11940 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.463021 11873 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.475167 11868 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.492089 11869 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.496467 12004 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.520264 11940 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.520746 11873 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:13.520444 11996 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.521360 11875 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.526522 11557 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.527052 11885 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:13.528081 11963 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.018s
I20260709 12:27:13.496065 11978 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.518694 11924 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.512249 11790 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.533299 11998 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.535454 11941 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.536772  7335 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:13.576206 11924 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:13.579406 12073 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.368s	user 0.012s	sys 0.000s
I20260709 12:27:13.582020 11915 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:13.559186 11790 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.500154 11993 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.565660 11886 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.550662 11963 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.584833 11915 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:13.588032 11994 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:13.605580 12076 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:27:13.587863 12022 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:13.611320 12077 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:27:13.385846  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.374941  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 203234us
W20260709 12:27:13.657335  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.351451  6303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.004s	sys 0.007s
W20260709 12:27:13.658926  6304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.006s	sys 0.017s
W20260709 12:27:13.659417  6303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.004s	sys 0.007s
I20260709 12:27:13.659994  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:13.659602  6304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.006s	sys 0.017s
I20260709 12:27:13.372959  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:13.671799  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 298716us
I20260709 12:27:13.676111  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:13.371357  6383 rpcz_store.cc:269] 0709 12:27:11.534937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.535065 (+   128us) service_pool.cc:224] Handling call
0709 12:27:13.181202 (+1646137us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:13.181398 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3564636416}
I20260709 12:27:13.679857  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:13.682159  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:13.683439  6384 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:13.681519  6381 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:27:13.388064  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.690553  6381 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:13.691056  6381 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:13.692873  6381 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.697575  6381 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:13.695632  6272 leader_election.cc:400] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:27:13.698690  6272 leader_election.cc:403] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:27:13.695547  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:13.708393  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 337089us
I20260709 12:27:13.710040  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:13.715742  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:13.403890  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:13.717717  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.751478  6450 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:13.414146  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.456703  6455 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:13.754926  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.753185  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:13.755838  6455 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:13.756501  6450 raft_consensus.cc:2368] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:13.754967  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:13.733744  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:13.735998  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 376290us
I20260709 12:27:13.760838  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:13.741600  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:13.744830  6381 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
W20260709 12:27:13.753064  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 349074us
I20260709 12:27:13.762957  7316 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:13.764405  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:13.765954  6450 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:13.765854  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:13.770573  6381 pending_rounds.cc:85] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 15
I20260709 12:27:13.770908  6381 pending_rounds.cc:107] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:27:13.771286  6381 raft_consensus.cc:2889] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:13.771225  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:13.772586  6384 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:13.780884  6455 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:13.542006 11345 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:13.784085 11739 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:13.784411 11757 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.785493 11739 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.785794 11757 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:13.787586 11757 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.785738 11345 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.790990 11788 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.792294 11788 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.793207 11788 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.797264 11915 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.798386 11868 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.799769 11915 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.810911 11885 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.814275 11924 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:13.818897 11963 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.608120 12009 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:13.821038 12009 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:13.823227 12009 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.825974 12009 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:13.826902 11944 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:13.827724 11944 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.828181 11955 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.829037 11955 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.830892 11886 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:13.831382 11886 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.831734 11886 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:13.832757 12022 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.833624 12022 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.569051 11941 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.835479 12004 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.836421 12004 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:13.837088 11941 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.837914 12004 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:13.830364 11944 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:13.838374 12004 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.921464 12025 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:13.922048 12025 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.922510 12025 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:13.586642 11986 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.932051 11788 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.944078 11996 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.945016 11996 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:13.946944 11797 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:13.947798 11797 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.949841 12007 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:13.946969 11757 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.959982 11955 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.970389 11886 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.970937  6450 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:13.971053 11998 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:13.973222 11998 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:13.974972 12027 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:13.975874 12027 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:13.978543 11814 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.594s	user 0.018s	sys 0.002s
I20260709 12:27:13.979254 11814 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:13.979454 12045 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:13.979986 12045 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:13.980501  6457 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:27:13.981098 11892 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.797s	user 0.016s	sys 0.002s
I20260709 12:27:13.981371 12045 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.981594 11557 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.981676 11892 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.983409  6454 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 13. Ops: []
I20260709 12:27:13.984153 12045 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:13.984489 11892 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.992277  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2714) took 1040 ms. Trace:
I20260709 12:27:13.977438 12027 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:13.979321 12034 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.990294 11797 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:13.945861 11996 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:13.984167 11557 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:13.994668  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.919124 12004 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:13.976815 11886 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.006943  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.010630 11994 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:14.033854 11994 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:14.034580  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 252dcff751be4a3d948b0f83cb081100: Aborted: Op aborted by new leader
I20260709 12:27:14.035447 11994 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.040089  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.040956  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.041723  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.018158 11739 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.043215 11739 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 8. Current term is 9.
I20260709 12:27:14.043555  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2724) took 1048 ms. Trace:
I20260709 12:27:14.043690  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.044215 11739 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.044665  6452 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 11: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:14.045111 11739 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.045863  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.017079 11940 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.021782  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:13.920295 11868 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:14.030983 11757 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.041831  6457 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:14.021366  6455 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:14.047575  6459 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:14.023576  6451 rpcz_store.cc:276] 0709 12:27:12.951869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.952032 (+   163us) service_pool.cc:224] Handling call
0709 12:27:12.952415 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.456503 (+504088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.991730 (+535227us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b5400266e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:13.992077 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952644864}
I20260709 12:27:14.044137  6454 rpcz_store.cc:276] 0709 12:27:12.995050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.995187 (+   137us) service_pool.cc:224] Handling call
0709 12:27:12.995467 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:13.983862 (+988395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.043242 (+ 59380us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b54002c8654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:14.043489 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2271085824}
I20260709 12:27:14.059038 11941 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:14.059304  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.059523  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.059805 11941 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.060709  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.060132  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.050683 11868 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:14.066761  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.067660  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.068320  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.068588  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.068780  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.085422  6453 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:14.064579 12034 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:14.087249 12074 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.535s	user 0.000s	sys 0.002s
I20260709 12:27:14.087677  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.088994 12034 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:14.090001 11989 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.854s	user 0.012s	sys 0.006s
I20260709 12:27:14.085461  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.090734  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.089934  6378 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:14.084967  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:14.066701 11868 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.092346  6457 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:14.068595  6455 raft_consensus.cc:2368] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15.
I20260709 12:27:14.070585  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.085170 11944 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.085750  6456 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.590s	user 0.018s	sys 0.000s
I20260709 12:27:14.090343  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.050365 11757 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.067112  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.091797 12045 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.105309  7316 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:27:14.109658  6456 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:14.118631  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.122117  6378 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:14.124178  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.146987  6459 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:27:14.137068  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 372252us
I20260709 12:27:14.109376 11944 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.158643  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.162115  6376 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:14.165161  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.166673  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:14.167197 12079 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.559s	user 0.000s	sys 0.002s
I20260709 12:27:14.167649  7332 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:14.169742  7335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 761) took 2443 ms (client timeout 1682 ms). Trace:
W20260709 12:27:14.170333  7335 rpcz_store.cc:269] 0709 12:27:11.725933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.726051 (+   118us) service_pool.cc:224] Handling call
0709 12:27:14.169154 (+2443103us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400479ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:14.169422 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4060319616}
W20260709 12:27:14.171802  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.172950 11989 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:14.174052 11989 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.175787  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:14.176797  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 122447us
I20260709 12:27:14.177992 11989 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.137005 12045 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.199972  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.206679 11866 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:27:14.211315  7285 leader_election.cc:400] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:14.211927  7285 leader_election.cc:403] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:27:14.213150  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 96821.7us
I20260709 12:27:14.218760 11739 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.221048 12087 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:14.221735 12087 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.222213 12087 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:14.211195  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.211095 11757 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.231614 11866 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.250066  6385 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.236954  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2412) took 1402 ms. Trace:
W20260709 12:27:14.256935  6272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.002s	sys 0.032s
W20260709 12:27:14.263106  6272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.002s	sys 0.032s
I20260709 12:27:14.261283  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:14.263526  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 568486us
I20260709 12:27:14.229554 11941 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:14.274658 11978 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.005s
I20260709 12:27:14.275086  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:14.274691  6383 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:27:14.279191 12084 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:14.281925  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:14.282590  6458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.000s	sys 0.031s
W20260709 12:27:14.282974  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.282686 11996 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.193771 11790 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.292748  6457 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:14.300709 12084 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:14.301396 12084 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.301899 12084 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.304526 11927 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:14.306440 11927 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:14.229952 12025 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.307859 11927 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:14.308415 11998 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:14.308817 11998 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:14.309932 12025 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:14.310040 11998 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.311486 11998 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:14.311699 11940 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:14.307859  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:14.314783  6458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.000s	sys 0.032s
W20260709 12:27:14.314815  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:14.282858 11869 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.739s	user 0.004s	sys 0.010s
I20260709 12:27:14.316388  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.326020  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:14.326678 11955 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.327493 11955 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:14.332981  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58025.5us
I20260709 12:27:14.317467  6378 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:14.318531  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.319883  6377 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:14.330698 11955 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:14.339775  6380 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.012s	sys 0.030s
I20260709 12:27:14.338013  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:14.347170  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.348943  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.349920  6377 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:14.346246  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.018s	sys 0.024s
W20260709 12:27:14.354005  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.018s	sys 0.024s
I20260709 12:27:14.254554  6303 rpcz_store.cc:276] 0709 12:27:12.826331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.826505 (+   174us) service_pool.cc:224] Handling call
0709 12:27:12.826806 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.827007 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:12.827271 (+   264us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.154037 (+1326766us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:14.165868 (+ 11831us) spinlock_profiling.cc:243] Waited 907 us on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:14.180511 (+ 14643us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:14.187671 (+  7160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.196694 (+  9023us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.228947 (+ 32253us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1131,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":382084,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":281,"spinlock_wait_cycles":2086144,"thread_start_us":1319046,"threads_started":2,"wal-append.queue_time_us":768366}
I20260709 12:27:14.355798  6379 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:14.332180 12034 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:14.357440  6380 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.972s	user 0.012s	sys 0.030s
I20260709 12:27:14.294133 11978 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.337628 11869 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.339694 11927 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.364198  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.371939 12097 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:14.373246 12097 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:14.373634 12097 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:14.358829 12025 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:14.374617 11873 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.798s	user 0.000s	sys 0.014s
I20260709 12:27:14.374840 12025 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.375272 11873 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.375317 12097 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.377254 12097 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.378692  6301 raft_consensus.cc:2368] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15.
I20260709 12:27:14.355085 12004 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:14.381685 11915 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.004s
I20260709 12:27:14.384258 11869 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.375373 12025 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.386626 11873 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.318430 11941 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:27:14.388692 11941 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.389266 11941 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.390077  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.391640  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.392493  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 837) took 1334 ms. Trace:
I20260709 12:27:14.375476 11757 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:14.394277  6379 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:14.394472  6301 rpcz_store.cc:276] 0709 12:27:13.057738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.059349 (+  1611us) service_pool.cc:224] Handling call
0709 12:27:14.392261 (+1332912us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002df5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:14.392403 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2864681856}
W20260709 12:27:14.393440 12089 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.012s	sys 0.044s
I20260709 12:27:14.356884 11996 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:14.359503 11924 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.007s	sys 0.020s
I20260709 12:27:14.398715  6385 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:14.399403 11927 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.399534  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.400910 11790 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.421154  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.401559 12045 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.402082 11944 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:14.401847  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:14.402559  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.694s	user 0.009s	sys 0.033s
W20260709 12:27:14.424618  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.009s	sys 0.033s
I20260709 12:27:14.402657 12034 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.403105  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:14.398422 12089 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.012s	sys 0.045s
I20260709 12:27:14.363621 11955 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
W20260709 12:27:14.403937  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 688553us
I20260709 12:27:14.404089  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.406514  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.407009  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.408390 12084 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.429684  7318 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.429723  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:14.430462 12084 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.430871  6456 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:14.431208 11993 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.019s	sys 0.049s
W20260709 12:27:14.431581 11993 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.019s	sys 0.049s
I20260709 12:27:14.431813 11927 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.408543 11915 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:14.413256  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.435340  6457 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:14.415203  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.416316  6454 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:14.415273 11996 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.401008  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.422060  6385 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:27:14.424275  6452 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:14.418896 11924 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.007s	sys 0.020s
I20260709 12:27:14.453087 11924 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:14.453969  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:14.454901  6303 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:27:14.455842  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2873) took 2919 ms. Trace:
I20260709 12:27:14.456111  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.456952 11869 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.457705 11869 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.449952  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:14.417984  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59659.7us
I20260709 12:27:14.450068  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.460134  6379 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12. Re-sending same reply.
I20260709 12:27:14.461314  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.450800 11955 consensus_queue.cc:1243] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:14.461961  6379 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:14.463076  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.456199 11873 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.466986 11873 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.469444  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:14.458058 11915 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:14.470831 11915 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:14.454288 11924 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.472178 11915 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.473219 12094 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:14.473716 11915 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:14.449194 12003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.016s	sys 0.041s
I20260709 12:27:14.463341 12096 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:14.474992 12096 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:14.476374 12096 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.477061  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.477990  7331 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:27:14.451634  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.478901  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.479138  6385 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:14.479732  7314 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:14.480382  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.481668  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.464112 11989 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:27:14.465487  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.487396  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.488230  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.489578 12025 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:14.474964 12003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.017s	sys 0.042s
I20260709 12:27:14.490329  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.490923  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.491345  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:14.492019  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.477926 12096 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:14.492779  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.493662  6379 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:14.451236 11875 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.492772  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.496444 11955 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:14.430277 12004 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:14.470067 11873 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.506317  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:14.494983 11996 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:14.517058  7318 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:14.510679 11345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.009s	sys 0.066s
I20260709 12:27:14.529932  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.499507 11866 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:14.504423  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.532114  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.536327  7332 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:14.545805  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 46337.5us
I20260709 12:27:14.503965 11955 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:14.532120 11345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.010s	sys 0.066s
I20260709 12:27:14.487396 11868 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:14.582187  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47055.2us
I20260709 12:27:14.500123  6380 rpcz_store.cc:276] 0709 12:27:11.536678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.771744 (+235066us) service_pool.cc:224] Handling call
0709 12:27:11.772121 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.961404 (+1189283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:12.961439 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:12.961878 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.363793 (+401915us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:13.604873 (+241080us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:14.369379 (+764506us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.454634 (+ 85255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.455395 (+   761us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54003d3bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:14.455449 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.455518 (+    69us) raft_consensus.cc:1685] finished
0709 12:27:14.455528 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.455763 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":885605,"prepare.queue_time_us":837678,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":874481,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":57379,"spinlock_wait_cycles":776133760,"thread_start_us":1279621,"threads_started":2,"wal-append.queue_time_us":522719}
W20260709 12:27:14.587728  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.504993 12003 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:27:14.594946  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.596122  6305 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:14.597061  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:14.554222 11885 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.005s	sys 0.032s
I20260709 12:27:14.607553  6378 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:14.616146 11955 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.616343  6301 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:14.617579 11868 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:14.619695  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:14.620551  6379 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:14.621497  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.622666  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.626480  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.627157  6379 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:14.627988 12034 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.627990  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.629848  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.630218 11978 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.579457 11875 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:27:14.644539 12025 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:14.645746 11986 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.016s	sys 0.015s
I20260709 12:27:14.647392  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.648551  6385 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:14.650695 11993 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:14.652596  6304 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:14.653039 12045 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:14.653615 12045 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:14.646296 11986 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.016s	sys 0.016s
I20260709 12:27:14.659179 11557 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:14.659053  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:14.608076  6303 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:14.609710  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:14.687006  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 367218us
W20260709 12:27:14.690263  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:14.665310  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.666913 11941 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.666679 12098 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:27:14.671043  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:14.671964  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 352384us
I20260709 12:27:14.679327 11869 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.681012  6305 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:14.681751 11814 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:14.683030  6301 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:14.661706  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:14.656934  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:14.611177 11873 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:27:14.685652 11885 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.006s	sys 0.033s
I20260709 12:27:14.695372 11557 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:14.709177  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52333.3us
W20260709 12:27:14.712895  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51289.8us
I20260709 12:27:14.704280 11873 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:14.718518 12027 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:14.718940 11873 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:14.719200 12027 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:14.720008  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 245213us
I20260709 12:27:14.714056  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:14.722378  6451 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:14.724207  6377 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:14.725622  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:14.721485 12004 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:14.728987  7316 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
W20260709 12:27:14.713949 12022 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.880s	user 0.014s	sys 0.003s
I20260709 12:27:14.728650  6376 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:27:14.719687 12027 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:14.738391  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:14.713763 11557 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:14.698374 12087 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.742280  6459 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:14.713811 12045 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.748358  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.748920 12045 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:14.749269  7332 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:14.750144  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.750818  7319 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:14.750876  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49231us
I20260709 12:27:14.752579 12091 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.753232 12091 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.754274 12102 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:14.778658  7319 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:14.783866  6377 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:27:14.784396  6377 pending_rounds.cc:85] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:27:14.788463  6377 pending_rounds.cc:107] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:27:14.788859  6377 raft_consensus.cc:2889] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.789075  7320 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:14.791051  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.755584 11927 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:14.791890  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:14.784716 12106 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 12:27:14.812444  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71579.9us
W20260709 12:27:14.813830  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70193.2us
I20260709 12:27:14.739686 11869 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:14.812077  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 169621us
I20260709 12:27:14.755709 12022 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:14.822471  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.740718 12004 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:14.787496 11866 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.834476  6305 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 9: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:14.729604 11814 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.835582  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:14.814533 11927 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:14.843041  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.842885 11869 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:14.871271  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79618.7us
I20260709 12:27:14.872582  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:14.845175 12114 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.873308 11869 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.873931 12114 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.877344 12114 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.878965 12092 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:14.878291 11989 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:14.885180  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 811) took 1108 ms. Trace:
I20260709 12:27:14.885618  6381 rpcz_store.cc:276] 0709 12:27:13.776577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.776753 (+   176us) service_pool.cc:224] Handling call
0709 12:27:13.777303 (+   550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.164081 (+386778us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:14.180220 (+ 16139us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.228137 (+ 47917us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.683734 (+455597us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:14.683772 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.683848 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.884941 (+201093us) raft_consensus.cc:1685] finished
0709 12:27:14.884964 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.885117 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200736,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":271,"spinlock_wait_cycles":89856,"thread_start_us":808288,"threads_started":2,"wal-append.queue_time_us":526893}
I20260709 12:27:14.886648 12004 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.891172  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:14.854324  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.894641 11924 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:14.786558 12025 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:27:14.896162 12025 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:14.896525 12025 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:14.897482 12025 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.860877  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.899358  6310 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:14.861164 12093 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:14.861876  6304 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:14.862146 12091 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.866437  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:14.867234  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 810) took 1093 ms. Trace:
I20260709 12:27:14.869758 12087 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.848485 11814 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:14.909365 12025 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:14.841284 12098 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:14.913532  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.914356  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.915138  6310 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:27:14.915621  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.916715  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.921125 11869 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:14.925227  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:14.913177  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:14.943728  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97597us
W20260709 12:27:14.944360  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.942919  6310 pending_rounds.cc:85] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 9
I20260709 12:27:14.947448  6310 pending_rounds.cc:107] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:27:14.947741  6310 raft_consensus.cc:2889] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.948696  6301 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:14.956348 11989 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:14.956836 11989 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:14.957327 11989 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:14.958722 12003 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:14.929862 11557 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.962136 11557 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.963017  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.964088  6302 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:27:14.967016  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 100673us
I20260709 12:27:14.969079 11885 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:14.969722 11885 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:14.970016 11978 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:14.971400 12093 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.967101 11993 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:14.972692 11993 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:14.973701 11993 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.946902  6305 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:14.936659  6384 rpcz_store.cc:276] 0709 12:27:13.773920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.774050 (+   130us) service_pool.cc:224] Handling call
0709 12:27:13.774668 (+   618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.141338 (+366670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.141386 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.159056 (+ 17670us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.722262 (+563206us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:14.779601 (+ 57339us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.780666 (+  1065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.780754 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:14.866931 (+ 86177us) raft_consensus.cc:1685] finished
0709 12:27:14.866959 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:14.867126 (+   167us) inbound_call.cc:177] Queueing success response
0709 12:27:14.887138 (+ 20012us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b180025c960. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:27:14.891754 (+  4616us) spinlock_profiling.cc:243] Waited 1.9 ms on lock 0x7b180025c960. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
Metrics: {"mutex_wait_us":7089,"prepare.queue_time_us":60488,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":203505,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":3428,"spinlock_wait_cycles":9112192,"thread_start_us":842850,"threads_started":2,"wal-append.queue_time_us":662322}
I20260709 12:27:14.974985  6303 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:27:14.980685  6383 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.691s	user 0.014s	sys 0.001s
I20260709 12:27:14.967037  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:14.981714  6383 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:27:14.982542  7320 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:14.974977  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:14.983989  6300 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:14.980132  6308 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:27:14.976492  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:14.974413 12093 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.989869  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:14.859404 11866 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:14.991199 11915 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.007s
I20260709 12:27:14.992177  6378 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:14.992347 11866 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:14.981666 12098 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:14.994612  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:14.995584 12093 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:14.995445 12098 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.997008  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:14.997123  7320 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:14.998167  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:14.994622 11996 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.001797  6309 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:14.986843  7285 leader_election.cc:400] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15.
I20260709 12:27:15.002910  7285 leader_election.cc:403] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.003186 11996 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:14.999294  6301 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:15.003477  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47909.4us
I20260709 12:27:15.004441 11866 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15.
I20260709 12:27:15.004805  7285 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.006274  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:15.007088 12117 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:15.010011  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:14.996016 12093 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.027848  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.029037  7318 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:15.017390 12093 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.027278 11814 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:15.032999 11814 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.034590 11866 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:15.034906 11868 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.035291 11866 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:15.035694 11868 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.036404 11866 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:14.978226 11885 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:15.066013  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69111.3us
I20260709 12:27:15.068166  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.069528  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.070928  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.072909  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:15.073490  7319 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:15.076303  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.077162  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.038647 11869 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:15.078785  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.080457  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:15.086057  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 250691us
I20260709 12:27:15.082408  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.087767  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.058380 12092 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:15.093024  6454 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:15.057565 11868 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:15.096253  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.987989 11927 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.055130 11557 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:15.095976  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:15.109261  6303 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:15.105907  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.116570  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.074306 11996 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:15.131016  6309 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:27:15.120574  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.131282  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:15.111809  6452 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:15.132205  7320 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:15.115073  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 323545us
W20260709 12:27:15.126166  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.089953 11869 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:15.133002  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.134867  6381 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:15.140964  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.141850  6454 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
W20260709 12:27:15.142036 12126 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:27:15.143388 12091 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:15.143864 12091 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.144217 12091 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:15.146652 11996 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:15.147106 11814 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:15.147692 11996 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.149233  6456 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:15.137634 11869 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.151454  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.154937  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.155040  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.156422  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.157035  6381 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.159368  7320 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
W20260709 12:27:15.159816  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 170028us
I20260709 12:27:15.160429  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.160566  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.161434  6380 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:15.163910  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.166352  6378 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:15.169836  6379 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:15.162420  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.038429  6458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53146508e564f8aae279a61e6696e80"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
W20260709 12:27:15.177359  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97015.3us
I20260709 12:27:15.178539  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:15.181229  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:15.191411  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.192008  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.216177  6383 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:15.220465  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:15.223501  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.224475  7320 raft_consensus.cc:2410] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:27:15.219808  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.190913  6458 raft_consensus.cc:606] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:15.234246  6379 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:15.235939  6381 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:15.241460  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.243422  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:15.244182  7319 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
W20260709 12:27:15.225131 11873 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.006s
I20260709 12:27:15.249432  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:15.253652  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:15.260646  6383 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:15.261152  6383 pending_rounds.cc:85] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 8
W20260709 12:27:15.261037  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45037.7us
I20260709 12:27:15.271661  6383 pending_rounds.cc:107] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:27:15.272437  6383 raft_consensus.cc:2889] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:15.267298  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.256989 11557 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:15.250849 11875 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.255400 12102 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:15.260478 12025 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:15.284577  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124087us
I20260709 12:27:15.288193  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.269183 11814 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:15.289719  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57586.6us
I20260709 12:27:15.286895  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:15.252336 11873 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.277382 12114 raft_consensus.cc:2764] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:27:15.254439 11941 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:27:15.282732 11557 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.322777 11557 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.323086 11989 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.323925 11940 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:15.324339 11940 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:15.322868 11941 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:15.325254 11941 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:15.325498 11989 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.326118 11941 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.312067 11875 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.327559 11941 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:15.286788 12025 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.328173 12025 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.329273 11875 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:15.330054 11940 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.331117 11940 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.331400 11873 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.309180 11814 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.271242 12093 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:15.314133 12127 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:27:15.318919 12128 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:27:15.285926 12115 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.319593 12087 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:15.320497  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.285652 12102 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:15.321167  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:27:15.295118 11739 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:15.322304 12027 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 12:27:15.312978  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.340308  6380 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:15.321086  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.341388  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:15.370613  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95632.7us
I20260709 12:27:15.335275 11814 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:15.374018  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53610.4us
I20260709 12:27:15.372210  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:15.374329  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126447us
W20260709 12:27:15.375268  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62412.9us
I20260709 12:27:15.380412  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.366431 12045 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.367422 12087 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.367364 12102 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.368505 11345 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:15.362809 12092 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:15.384691  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.380779 12117 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.394291  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.400647  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.401824  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.384361 11994 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.403517  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.399725  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.402801 12034 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:27:15.409157  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.373790 11869 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.384222 12141 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.390331 12087 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.407689  6451 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:15.383999 11739 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:15.413765 12034 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:15.374446 12115 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.415084 12092 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.416625  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.417155 12003 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.418582 12093 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.498749  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.500005  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.500495  6451 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.412878  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.503088 12115 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:15.426478 12147 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.484346 12034 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:15.503844 12147 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.504803 12034 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.505650  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.506098 12147 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.427520 11996 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:15.507074 11739 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.507380 11996 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.508591 12003 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:15.509095 12003 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:15.509460 12003 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:15.509510 11996 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.429056  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.429430  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.510703  6309 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:15.511742  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.512015 11869 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.512331  6309 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:15.513312  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.430859  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.431254  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.433123  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.434892  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.514546  6305 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:15.437531  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.438824  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:15.439844 12134 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:27:15.440815 12137 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:15.375394 12027 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.449787  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.456244 11345 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:15.461318  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.519805  6456 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:15.477496  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.491820  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.492708  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.523535  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:15.493355  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 182797us
I20260709 12:27:15.524816  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.525564  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.497428  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:15.503198  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 90607.3us
I20260709 12:27:15.515537  6307 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:15.505889 12034 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.506865  6376 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
W20260709 12:27:15.480495 12127 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:27:15.534055  6456 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:15.516906  6302 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:15.526003  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:15.536417  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86724.7us
I20260709 12:27:15.528174  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:15.547838  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.548144  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.547142  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.548866  6300 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:15.550419  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.555269  7317 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:15.556747  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet b776445e68c14533bcde1f784caa703a: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:15.558725  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.559388  7320 raft_consensus.cc:2376] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17. Re-sending same reply.
W20260709 12:27:15.561228  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.561801  7320 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:27:15.563786  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.565145  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.565925  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.567180  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.570556  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:15.516355 12045 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.571579  6309 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.572556  6381 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:15.564585 12159 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.573387  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.573387  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.577541  6305 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:15.581045  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.583127  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2891) took 1258 ms. Trace:
I20260709 12:27:15.498181 11557 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.576992  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.585759  6457 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:15.583693  6385 rpcz_store.cc:276] 0709 12:27:14.324332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.649244 (+324912us) service_pool.cc:224] Handling call
0709 12:27:14.649716 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.650270 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.650303 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.695913 (+ 45610us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.930357 (+234444us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:14.953316 (+ 22959us) log.cc:844] Serialized 14 byte log entry
0709 12:27:14.953640 (+   324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.953745 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.582828 (+629083us) raft_consensus.cc:1685] finished
0709 12:27:15.582860 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.583050 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40481,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":236,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":199,"thread_start_us":201617,"threads_started":1,"wal-append.queue_time_us":421518}
I20260709 12:27:15.547628 11996 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.578259  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.579418  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.573926  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.589418  6308 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:15.537357 11345 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.568464 11869 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:15.592515  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.593761  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68289.5us
I20260709 12:27:15.594743  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.590670 11993 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:15.592559 11869 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:15.597944  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:15.598393  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.575709 11978 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:15.597625 11993 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:15.598104 11869 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.596920  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.589330  6376 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 9.
I20260709 12:27:15.591822  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.601615  6307 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:15.602248  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:15.603488  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.604192  6304 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:15.599129  6302 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:15.583374 11873 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.606846  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.611562  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.614907  6385 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:15.615211  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:15.618655  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.603132 12141 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:15.619935 12141 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:15.620316 12141 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:15.620780  6379 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:15.621724 12025 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.616730 11941 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:15.622172  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.622701 11941 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:15.623186 11941 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.623975 12025 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.625419 11557 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.582978 12045 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.628301  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.629078  6308 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:15.629439 12045 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.629841  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.631004  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:15.631361 12027 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.604947 11978 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.645655 11978 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.605599 11993 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.577559 12091 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.631541  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:15.652663  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45908.1us
I20260709 12:27:15.618534 12093 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:15.640116 12136 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.319s	user 0.005s	sys 0.000s
I20260709 12:27:15.640746  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.663452  6302 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:15.664477  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.665107  6302 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:15.641814  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44954.6us
I20260709 12:27:15.642737 12115 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:15.643231  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:15.671452  6309 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:15.644624  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.642864  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.654764  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:15.658413 12091 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.659379 12093 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:15.659922  6303 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:15.659951 12022 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:15.660975  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:15.664603 12153 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.189s	user 0.001s	sys 0.001s
I20260709 12:27:15.610728 12159 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.667341  6454 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:27:15.667711  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52617.6us
I20260709 12:27:15.640965 12087 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.671116 12115 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:15.719846 12115 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.720466 12087 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:15.722262  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:15.722838  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.723053  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:15.724436  6425 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:15.724694  6425 leader_election.cc:403] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.725349  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:15.726265  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.727002 11986 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.727659  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.727595 11986 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.728996  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.729646  6381 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:15.730055 11986 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.714828 12025 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:15.730726 12098 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:15.737946 12098 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:15.730928  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:15.730880 12004 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.739573 12004 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:15.739746  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.739997 12004 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.678825  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.740422 12004 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:15.741276  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74211.8us
I20260709 12:27:15.741544 12102 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.683019  6309 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:15.743024  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.744067 12091 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:15.744652  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.683364  6453 pending_rounds.cc:85] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 9
I20260709 12:27:15.684201  6310 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:15.678960 12147 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:15.715183  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54297.3us
W20260709 12:27:15.719383 12155 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
W20260709 12:27:15.720546 12152 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:27:15.738327 12098 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.746201 12144 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.757000  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:15.761070  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.730454  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:15.740423  6376 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.735265  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.676405  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:15.746713  6453 pending_rounds.cc:107] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 14.10
W20260709 12:27:15.766889  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.767021  6453 raft_consensus.cc:2889] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:15.731729  6454 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:27:15.750754  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.755695  6383 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.777931  6310 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:15.790577 12093 raft_consensus.cc:2764] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 12:27:15.791786  6309 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:15.792501 12141 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.747479  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.737485  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:15.794004  6309 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:15.605273 12092 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:15.681304  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:15.799117  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48443.5us
I20260709 12:27:15.799417  6302 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:15.799612  6458 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:15.807436  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 126264us
I20260709 12:27:15.809242  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:15.731015 11885 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
W20260709 12:27:15.818987  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71589us
I20260709 12:27:15.746469 12003 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.822562 12003 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.822824 11940 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:15.795050 11927 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.811968 11875 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:15.806421 11993 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.825887  6452 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:15.826912 11993 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:15.806476 12141 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:15.811777  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74341.8us
I20260709 12:27:15.829447  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:15.829517 12159 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:15.812655  6308 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:15.814200 11915 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:15.794065 11869 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.837066  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.829377 11345 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.837829  6458 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:15.732731 12025 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.815263 12102 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.815343  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:15.816169 12151 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:27:15.816293 12144 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:15.851217 11345 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.836930 12141 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:27:15.850095 11993 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:15.747020 12147 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:15.854866 12147 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.816188 12160 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.855535  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.855580 12160 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.858191 12160 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.738063 11994 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:15.832095  6301 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:27:15.738065 11978 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.874982  6301 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:15.875264  6451 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:15.876669  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.877792  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.880043 11978 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.832515  6383 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:15.833467  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.814949 12117 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.812958  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.835994  6376 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:15.846776  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.811717 11866 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:15.854756  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:15.858673  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.859166  6302 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:15.822901  6378 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:15.860502  6381 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:15.846849 12025 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:15.817078 12034 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:15.862983  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:15.907939  6455 raft_consensus.cc:2368] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
W20260709 12:27:15.863804  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48557.5us
I20260709 12:27:15.909970  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:15.866711 11940 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:15.866397 12147 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.867341 11993 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:15.867956 12004 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.871668 11869 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.893074  6308 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:15.865804 11941 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.897182  6454 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:15.754030 12115 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:15.901150  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88298us
I20260709 12:27:15.902245  6382 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:15.903864 12117 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:15.911465  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78091.6us
I20260709 12:27:15.929353  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:15.939131  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:15.940544  6301 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:15.916311 12114 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:15.923045  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 91546.6us
W20260709 12:27:15.941949  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 376861us
I20260709 12:27:15.914398 12141 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:15.946902  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.950901  6450 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:15.925480 11814 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:15.965363  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.941185 12163 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.970072  6302 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:15.945655 12114 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:15.923223 11885 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:27:15.978368  7331 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:15.978229  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.978389  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.952467 12084 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:15.994855 12084 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.995306 12084 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:15.995133  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.997364  7329 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:15.998622 12087 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:15.974063  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:16.000172 12098 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.944087 11866 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:27:16.004938 11866 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:16.005419 11866 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:15.956637 11978 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:27:15.981624 11993 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.007488  7314 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:15.927786 11940 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:15.983634 12004 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:16.010589 12004 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:16.010967 12004 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:16.011088 11941 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:16.011900 12004 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.983732  7316 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:15.978217 12163 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.971834 11814 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:15.985890  7318 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:15.986114 11885 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:15.987109  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.987640 12115 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.017678 11814 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:15.988972  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:15.940928 12034 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.018793  7319 raft_consensus.cc:2368] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:15.989982  7335 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:16.019986 11345 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:16.020581 12034 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:15.990248 12165 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:16.021799 12165 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:16.022290 12165 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:16.023456 12165 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.025380 12165 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.002338 12098 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:16.029335  6300 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:27:16.032238 12154 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.393s	user 0.003s	sys 0.003s
I20260709 12:27:16.032061 12025 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.032886 12144 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:16.034711 12025 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:15.978428 12114 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.041332  6275 leader_election.cc:400] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:15.986378  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:16.041584  6451 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:16.042120  7320 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:16.043421  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.043860  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:16.044524  6303 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:16.045816  6303 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:16.046023  6275 leader_election.cc:403] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:16.016722 12163 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:15.988749  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:16.047223  6451 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:16.048219 12141 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:16.054826  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:16.056439  6302 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:27:16.057655  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 145156us
I20260709 12:27:16.063256  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.036952  6300 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:16.034770 12147 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.040130  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.035298 11866 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:15.991757  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.067440  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:16.082960  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:16.084115  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:16.082683  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 153465us
I20260709 12:27:16.085261  6456 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:16.080709  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:16.087224  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:16.089792  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124502us
I20260709 12:27:16.118701 11869 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:16.121806 11869 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.122279 11869 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.088783 12098 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:16.076292 11941 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:16.091946  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:16.127733 12147 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:16.129283  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:16.129679 12162 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.226s	user 0.000s	sys 0.002s
I20260709 12:27:16.130089 12147 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:16.130677 12147 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:16.131613 12147 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.116845 12114 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.134214  7331 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:16.136319 12141 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:16.136139 11739 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:16.138016 12114 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.138446  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:16.139405  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52546.9us
I20260709 12:27:16.139818  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:16.091624 12022 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:16.088996 11875 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:27:16.138643 12004 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:16.079180 12163 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:27:16.142987  6348 leader_election.cc:400] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:27:16.109575  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:16.185166  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.181154  6454 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:16.187162  6348 leader_election.cc:403] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:27:16.123842  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132185us
I20260709 12:27:16.178046  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:16.158699  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71049.8us
I20260709 12:27:16.180752  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:16.188985  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47530.9us
I20260709 12:27:16.140535 11739 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:16.189473  6305 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:16.188769  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:16.191330  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.192397  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:16.192518  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.193464  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.194841 12166 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:16.195334 12166 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.195592 12145 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:16.195684 12166 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:16.089041 11885 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:27:16.197170  6459 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:16.197242  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87793.9us
I20260709 12:27:16.199105 11994 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:16.200140 12160 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:16.200567 12160 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:16.201615 12160 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:16.207297  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.141345 12098 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:16.215618  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.183080 11941 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:16.197762 12034 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:16.142819 11978 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:27:16.230957  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.182731 12141 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:27:16.120823 12084 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.231638  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.235850  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.216503 12003 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:16.142742 12092 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 12:27:16.240082 11345 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:16.242120  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.241211  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:16.247035 11927 raft_consensus.cc:2764] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:27:16.247315  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.242687 11941 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:16.248705  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.250061 11927 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:16.249818 11941 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.250619 12173 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:27:16.248701 12165 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:16.224251 11993 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:27:16.246358 12170 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:27:16.253715 12170 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:27:16.215744 12102 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:27:16.251919  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:16.255920  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67261.9us
I20260709 12:27:16.258612 12165 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:16.261092  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.243775 11868 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:16.262086 12098 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.230458 12117 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:27:16.262547 11866 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.264955  6452 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:16.241777 12087 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:16.266994  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.239305 12022 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:27:16.272039  7331 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:27:16.274331  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63355.7us
I20260709 12:27:16.289276 11885 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:16.249735 12147 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:16.242707 12003 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:16.290707 12093 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.290941 12003 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:16.291345 12093 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.296774  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.298005  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.254827 12102 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:16.300909  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.304061  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:16.309715  6307 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:16.237665 12114 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:27:16.313969  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:16.310397 11994 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:16.338111  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.316161  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:16.341182  7331 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:16.278398 11868 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:16.294808 12144 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:27:16.310681 12093 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.300969 12145 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:16.274699 12025 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:27:16.311609 12102 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:16.276043 11927 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.064s
W20260709 12:27:16.352385  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95296.4us
I20260709 12:27:16.353489  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:16.310192 11739 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:27:16.341465 12165 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:16.334684 12158 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:27:16.311738 11814 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:27:16.344611 12087 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:27:16.357941 12163 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:16.371292  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:16.373211  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:15.791162  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:16.368695 11927 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:16.388167  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.401203  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.402038  6301 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.374778 11994 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:16.416340  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.390182 12158 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:27:16.435333  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.426689 11885 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:16.441013 11885 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:16.441597 11885 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:16.441447 12166 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.382802 11915 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:27:16.450392  6301 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:16.475615  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.443811 12166 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.488536 12145 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:16.489006 12145 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:16.448464 12091 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:16.489904 12180 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:16.490468 12091 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:16.491545 12091 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.492779 12091 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:16.493708 11885 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.413316 11994 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:16.494815 12084 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:16.498095  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.499967  6383 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.501614  6454 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:16.505209  6424 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
W20260709 12:27:16.442409 12176 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
W20260709 12:27:16.506845  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.515020  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:16.381812 12102 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.527047  6383 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:16.494786 11994 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:16.460911 12185 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:16.399538 11868 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.509848 12022 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:16.509892 11875 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:16.530405  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:16.549613 11345 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.550215 11345 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.512970 12084 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:16.573629  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 200481us
I20260709 12:27:16.582024 11345 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.543831 12185 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:16.587826  6305 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:16.534219 12158 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:27:16.568781 12173 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:16.565268 11875 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:16.590894  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:16.592890  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78612.8us
I20260709 12:27:16.609448 12147 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:16.590085 12185 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:16.577142 12084 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.620182 12022 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:16.621377 12189 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:16.622426 12160 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.627211  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.607836 12173 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:16.655035  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69709.7us
I20260709 12:27:16.602366 11915 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:16.667222  7320 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:16.670128 12170 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:16.671826 12201 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:16.674075 12034 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
W20260709 12:27:16.676313 12192 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:27:16.659067 11869 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.531s	user 0.012s	sys 0.000s
I20260709 12:27:16.678732  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.679250  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:16.685460  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:16.687547  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.701236  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:16.654188 11885 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:16.697716  6308 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:16.698665  6307 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [11.11-13.12]
I20260709 12:27:16.700315  6309 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:16.704576  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:16.662793 12200 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.722275  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:16.723783  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.705538  6303 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:16.719432  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:16.749307  7319 raft_consensus.cc:2368] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:27:16.720110  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:16.720395 12084 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.724020  7318 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:16.716553 11866 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:16.632818 12098 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:16.724647  6383 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
W20260709 12:27:16.750473  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.699432 12165 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.766077 12165 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.766605 11941 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.767215 11941 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.768484 12165 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.769590 11941 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.770295  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:16.772169 11875 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.773283 11875 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:16.775206 11994 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.764928  6459 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:16.765602  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.751113  7335 raft_consensus.cc:2368] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:16.776888 12145 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.753708  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:16.706671 12160 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:27:16.705628 11869 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.778343 12145 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.804481 12159 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:16.754604 11866 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:16.786067 12003 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.820444 11866 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.821535 11994 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.821651 11866 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.819593 12159 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:16.778201 12165 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:27:16.824090  6302 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:16.824358 12160 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:16.824154  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:16.824856 12160 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:16.825194 12207 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.003s
I20260709 12:27:16.825752 11869 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.825960 12160 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.822561  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:16.804407 11986 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.792059  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:16.829603  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 173816us
I20260709 12:27:16.778259  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:16.829313  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:16.832515  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 205383us
W20260709 12:27:16.778862 12154 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:16.753556 12022 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:16.826781 12003 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.779846 12102 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.850777  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.750828 11996 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.853657  7319 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:16.811929 12172 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:16.841259 12114 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:16.873934  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.875312  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.747917 12200 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.764066 12173 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.859035 12022 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:16.879195 12022 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:16.883018  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.865960 12172 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:16.886170 12172 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.887779 12172 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:16.834527 11986 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.894466  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.895123  6385 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:16.895942  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:16.891717  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.870280 12114 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.849269 11993 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:27:16.897372  6379 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:16.900885  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.900951  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.861860 11814 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:27:16.902232  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:16.902944  7335 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:16.904053 11986 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.904589  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.905131 12114 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.905476  6385 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:16.906324  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.907027  6385 raft_consensus.cc:2368] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:16.870702 11996 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.907529  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.907738  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.904899  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.910810  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.909287  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:16.909672  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.911046  6385 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:16.877182 12208 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.911201  6376 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:16.912691  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.911976  6380 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:16.911914  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.914669  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.915283  6379 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:16.915998  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.916451  6379 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:16.918448  6384 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:16.919951  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:16.897145 12098 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:16.921866  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.922621  6385 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:16.923978  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:16.924607  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.930367 11996 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:16.930879 12170 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:16.931285 12170 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.931619 12170 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.932056  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:16.932714 12212 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.002s
W20260709 12:27:16.934801  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.938805 12159 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.939419  6453 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:16.940109 12159 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:16.917261 12208 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:16.940999  6455 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:16.925205 11940 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:16.942839 11940 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:16.943177  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:16.943763 11940 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.945145 11940 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:16.945343  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:16.946171  6453 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:16.949381  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:16.951445  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:16.953898  6376 raft_consensus.cc:2410] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:27:16.942166 12208 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:16.962383  6380 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:16.963611  6308 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:16.965505  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:16.967185  6310 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:16.971431  6301 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:16.977983  6301 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:16.981707  6307 pending_rounds.cc:85] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 5
I20260709 12:27:16.982030  6307 pending_rounds.cc:107] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:27:16.982302  6307 raft_consensus.cc:2889] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:16.982089  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:16.979310  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:16.985668 12092 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:16.910077 12210 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:27:16.923323 12098 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:16.987974  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:16.988772  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.990375  6309 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:27:16.996886  6385 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:16.986266 12092 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:16.959205 11941 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:17.002774  6310 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:16.999266  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.000875  6376 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:17.003317  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:17.004570  6385 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:17.006206  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.007113  6385 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:16.974629 12003 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:17.014058  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.015585  6385 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:17.016965  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:16.989420 12173 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.016460  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:17.021629  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.999742 12098 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.016785  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:17.032863  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44971.9us
I20260709 12:27:17.035956  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:17.014076 11868 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:17.040107  6351 leader_election.cc:400] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13.
I20260709 12:27:17.041774  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.002058 11814 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:27:17.045303  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.017755 11941 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:17.050714  6351 leader_election.cc:400] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:17.051038  6351 leader_election.cc:403] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:17.047572  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:17.070883  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.067655  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.059806  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.076776  6302 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:17.078032  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.079392  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:17.067245  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.001557 12022 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:17.143831  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68014.5us
I20260709 12:27:17.081346 11345 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:17.145421  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74631.5us
I20260709 12:27:17.147073  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.041314 11994 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:17.151616  6308 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
W20260709 12:27:17.093477  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.155205  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:17.086901  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.043733 11868 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:17.030550 12173 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:17.144526 12181 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:17.049585 11885 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:17.075145 11927 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.076519 11915 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:17.028820 12003 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.075361 12163 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:17.144779 12117 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:17.047065 12166 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.116s
I20260709 12:27:17.042110 12091 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:27:17.049278 12141 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.118s
I20260709 12:27:17.185786 12171 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:17.186627  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.186635 12211 raft_consensus.cc:2764] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:27:17.191310 12092 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:17.199241 12215 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:17.199891 12203 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:17.205363 12022 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:17.207432 12213 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:17.209628 12160 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.212736 12173 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.223013  6383 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:17.227350  6382 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:17.074802 12115 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:27:17.165843 12180 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:17.179649 12165 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:17.178148 11915 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:17.253922 12185 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.000s
I20260709 12:27:17.269539  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.245617 11978 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:17.173013 12208 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:27:17.248854 12225 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:17.279179  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.297436  6379 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:17.280030  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:17.298272  6385 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:17.268734 12158 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:17.277148 12166 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.271129 12147 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.214952 11927 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.298717  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.300521  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:17.301090  6450 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:27:17.303134  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:17.327087  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 167434us
I20260709 12:27:17.285182 12165 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.290349 11915 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.300414 12225 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:17.300618 12158 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:17.371958  6382 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:17.332440 12165 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:17.384433  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 118405us
I20260709 12:27:17.290894 12180 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:17.381481  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78456.6us
I20260709 12:27:17.301262 11978 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.303738 12185 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.291241 11868 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.282666 11941 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.326645 12147 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.385978 11866 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.328126 12166 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:17.389740 12145 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
I20260709 12:27:17.394039 12003 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.394367 11978 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.394852 12022 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:17.395118 12222 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:17.413527  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.414150  6455 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:17.368338 12225 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.415548 12147 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.415642  6380 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:17.415786  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:17.405941 12158 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.417109 12203 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:17.417359  6380 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:17.417676 12203 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.418174 12203 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.418845 12234 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:17.408751 11868 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:17.419538 12234 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:17.420506 12022 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:17.420748 12218 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:27:17.421660 12022 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:17.433632  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.395840  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63812.5us
I20260709 12:27:17.362828 12171 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:17.396679  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:17.396696  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.400244  6379 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:27:17.389046  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:17.405020  6452 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:17.448580  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:17.405843 11866 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.410642 12166 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:17.399447 12172 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.002s
I20260709 12:27:17.417116 12158 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.450225  6382 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:17.419548 12235 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.456408 12228 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:27:17.459144  6382 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:27:17.451072  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 304069us
I20260709 12:27:17.463429 12211 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:17.464813 12211 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:17.465166 12211 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:17.465912 12214 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:27:17.443813 12003 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:17.472456  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:17.483484  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94548.5us
I20260709 12:27:17.486450  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:17.489084  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 402272us
I20260709 12:27:17.489929  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:17.488020  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:17.467043 12223 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.080s	user 0.001s	sys 0.001s
I20260709 12:27:17.498245  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.502671  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.467584 12180 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.543744  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.468096 11927 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.445128 12098 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.510641 11875 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:27:17.493358 12228 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:17.420557 12234 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:17.514704 12220 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:27:17.516969 11885 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:17.494084 12003 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.466019 12211 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.509171 12172 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:27:17.499508 12022 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:17.466144 11915 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.524433  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.506072 12166 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:17.511477 12227 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:27:17.512159 12171 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.483444 12235 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.529243 12200 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:17.531071 12229 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 12:27:17.527256 12114 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.622s	user 0.013s	sys 0.000s
W20260709 12:27:17.533607 11940 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.004s
I20260709 12:27:17.505981 12117 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:17.535368 12170 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.603s	user 0.009s	sys 0.002s
I20260709 12:27:17.535468  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.536217 12203 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.468474 12185 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.547035  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.548477 12226 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:17.553371 12163 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.553853  6454 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:17.554965 12098 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.575740  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:17.582304  6379 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:17.547953 12208 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:17.568887 11866 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.570429 11885 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:17.606205  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70821.3us
I20260709 12:27:17.629323 12226 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.608464 12242 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:17.629921 12226 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:17.630169 12242 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.630586 12242 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:17.631036  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:17.608465  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61545.2us
I20260709 12:27:17.585990 12166 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:17.637167  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:17.642700  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.644444  6379 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:17.600916 12211 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:17.652482  6381 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16.
I20260709 12:27:17.653899 12173 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:17.657302 11996 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:17.657712 11996 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:17.632920 12117 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:17.671548  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65124.3us
I20260709 12:27:17.637756 12025 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.640091 12166 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:17.681109 12102 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.688875 12022 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:17.699286  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:17.603142 12114 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.711606  6379 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:17.587815 12225 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.648504 12163 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:17.708237  7320 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:17.701077  7319 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:17.701429 11941 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.715214 12145 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:17.715555 11941 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.715946 12235 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.717746 11941 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.719416  6378 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:17.663417 12243 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:17.721201  6379 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
W20260709 12:27:17.721674 12245 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:27:17.721228 12224 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:27:17.724236  6377 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:17.633102 12170 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.672636 12208 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.736682  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:17.740378  6452 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:17.752704  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.733355 12225 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.753687  6453 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:17.744639 11940 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.740063 12117 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.743880 12243 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:17.740098 12208 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.726944 12102 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.754945  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:17.674566 11996 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:17.765157 12250 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:27:17.759377 11940 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:17.766376 12247 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.165s	user 0.001s	sys 0.000s
I20260709 12:27:17.766804  6458 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:17.768679 11940 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.778210  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:17.778954  7319 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:17.781319  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:17.781324  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:17.782406  7319 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:17.782675 12234 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:17.745066 12160 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:27:17.783103 12234 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:17.762912 12246 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:27:17.796202 12234 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:17.757776 12243 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.798938 12243 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:17.801126 12003 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.802538 12160 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:17.803045 12160 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:17.803352  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.804023 12160 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.806942 12165 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.807456 12114 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.809417 12165 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.810238 12098 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.810941 11996 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:17.810860 12098 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.813488 12098 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.815380  6376 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:17.815824  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:17.816805  7319 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:17.818243  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:17.829443  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:17.787264 11915 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.851178  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:17.855141  6457 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:17.829239 12239 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.322s	user 0.000s	sys 0.005s
I20260709 12:27:17.825706 11993 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:17.814431 12243 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.757114 12022 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.864881  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:17.867326  6456 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.852806 12003 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:17.874562 12234 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.857851 12226 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.862988 11993 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.909395  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:17.910265  7318 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:17.912226  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:17.879266 12102 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.879464 12186 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.882375 12091 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.882431  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.883765  6272 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:17.887220 12141 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:17.887360 11885 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.927577  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:17.888319  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2909) took 1013 ms. Trace:
I20260709 12:27:17.886322 12147 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:17.963192 12147 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.965730 12147 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.865607 11915 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.890836  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.966614  6305 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:17.966457 11915 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.967867  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:17.968860 11915 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.891269  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:17.970211  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.891736  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.894883  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:17.895576 12258 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:27:17.859525 12203 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.900424  6457 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:27:17.903558  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 413715us
I20260709 12:27:17.915253  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:17.929039  6456 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:17.917843 11993 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:17.943621 12210 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:17.944090  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:17.944543  6303 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:17.945825 12243 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.980139  6274 leader_election.cc:400] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:17.945889 11885 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:17.946722  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.947333 12187 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:17.935659 12240 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.908463 12242 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.948992 12141 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:17.951777  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2596) took 1057 ms. Trace:
I20260709 12:27:17.952034 12226 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:17.939205 12257 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:27:17.953583 12234 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.953759 11940 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:17.906451 12093 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:17.908857 12003 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.970471  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:17.970703  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.002909  6376 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:17.941032  6378 rpcz_store.cc:276] 0709 12:27:16.875211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.875366 (+   155us) service_pool.cc:224] Handling call
0709 12:27:17.888172 (+1012806us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b540022f454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:17.888262 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921907968}
I20260709 12:27:18.003485  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:17.942260 12091 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.970880  6459 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:17.976581  6457 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:17.919144 12225 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:17.937808 12220 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:17.987037 12203 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.981051  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:17.938546 12186 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:17.982265  6301 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:17.982283  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:17.987808 12170 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:17.971081 12168 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.987393  6384 rpcz_store.cc:276] 0709 12:27:16.895489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.933129 (+ 37640us) service_pool.cc:224] Handling call
0709 12:27:16.933506 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.933886 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.933918 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.934466 (+   548us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.404465 (+469999us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:17.404509 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.404589 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:17.798225 (+393636us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b2400297a58. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:27:17.950296 (+152071us) raft_consensus.cc:1685] finished
0709 12:27:17.950323 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.951705 (+  1382us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":808,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":35279104,"thread_start_us":413435,"threads_started":1,"wal-append.queue_time_us":533739}
I20260709 12:27:17.995780  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2927) took 1032 ms. Trace:
I20260709 12:27:17.969182  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:17.972189  6307 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:17.973964  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.064988  6382 rpcz_store.cc:276] 0709 12:27:16.963348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.459964 (+496616us) service_pool.cc:224] Handling call
0709 12:27:17.460281 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.586608 (+126327us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:17.602141 (+ 15533us) log.cc:844] Serialized 14 byte log entry
0709 12:27:17.748667 (+146526us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:17.748739 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:17.748784 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.995395 (+246611us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003d0c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:17.995463 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:17.995669 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15338,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":3115,"raft.queue_time_us":1619,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4336,"spinlock_wait_cycles":289609088,"thread_start_us":146094,"threads_started":1,"wal-append.queue_time_us":228914}
I20260709 12:27:18.009829  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:17.930195  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.009829  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.097105  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.096937  6378 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:18.097002  6379 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:17.981082  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:18.098657  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.048596 12242 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:18.052990  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 72015.1us
I20260709 12:27:18.100978  6310 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:18.101516  6300 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:18.101414  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.058408  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.104688  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:18.105348  6382 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:18.110697 12147 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.111454  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:18.113111  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.114257 11866 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.114871 11866 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:18.116022 12262 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:27:18.117069 12261 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:27:18.117775 11866 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:18.007292 12225 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.118717 12225 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:18.120040  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2933) took 1133 ms. Trace:
I20260709 12:27:18.120464  6377 rpcz_store.cc:276] 0709 12:27:16.986149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.632419 (+646270us) service_pool.cc:224] Handling call
0709 12:27:17.646813 (+ 14394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.724753 (+ 77940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.119780 (+395027us) spinlock_profiling.cc:243] Waited 76.9 ms on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:18.119975 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176972672}
I20260709 12:27:18.099862  6379 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:18.097528  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:18.102770 12168 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.136826  6304 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:18.138111 12003 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:17.993899 12166 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.205623  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:18.122769 12141 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:18.123476  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 2920) took 1225 ms. Trace:
I20260709 12:27:18.209213  6385 rpcz_store.cc:276] 0709 12:27:16.897784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.051791 (+154007us) service_pool.cc:224] Handling call
0709 12:27:17.298094 (+246303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.298673 (+   579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.123301 (+824628us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400037d50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:18.123422 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355440384}
I20260709 12:27:18.143869  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:18.210177  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.210788  6452 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:18.117928 12236 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:18.214027  7318 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:18.214206  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:17.984421 12187 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:18.098702  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 120588us
I20260709 12:27:18.153270  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.062619  6376 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:18.224653  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:18.164772  6378 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
W20260709 12:27:18.227155  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.213449  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.182574  6382 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:18.229526  6377 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:18.206218  7331 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:18.167136  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.207588  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:18.212409  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:18.147082  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.231127  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:18.231665  6310 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:18.103317 12226 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:18.103890 12240 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.233098  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.223640  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.234925  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.158757 11940 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.243676  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.066833 12203 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.245535  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:18.208039 12173 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:18.238024  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 70980.9us
I20260709 12:27:18.250572  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.238679  6384 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:18.193161  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2590) took 1210 ms. Trace:
I20260709 12:27:18.251015  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.240901  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.250329  6376 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:27:18.241559  6310 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:18.205040 12147 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:27:18.215494  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.102164 12093 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:27:18.248251  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95092.2us
I20260709 12:27:18.234385 12243 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:18.251386  6383 rpcz_store.cc:276] 0709 12:27:16.892498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.900626 (+  8128us) service_pool.cc:224] Handling call
0709 12:27:18.064704 (+1164078us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54003d40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:18.102370 (+ 37666us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":672970240}
I20260709 12:27:18.226954 12242 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:18.213307 12236 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:18.306687  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.333859  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:18.307286  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44588.3us
I20260709 12:27:18.307822  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.250820  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:18.336140  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85386.5us
I20260709 12:27:18.336374 11814 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:18.321373  6381 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:27:18.258415  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.337244 11814 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:18.337570  6308 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:18.339504  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 89017.1us
I20260709 12:27:18.340565  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.341821  6383 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:27:18.239377 12158 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:18.337376  6381 pending_rounds.cc:85] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 11
I20260709 12:27:18.357852  6381 pending_rounds.cc:107] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:27:18.358217  6381 raft_consensus.cc:2889] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:18.248464 12114 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:18.397567  7319 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:18.257802 12226 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:18.319252 12092 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:18.344359 12087 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:18.410871  6385 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:18.323661 11866 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:18.435441 12091 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:18.415359 12214 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:27:18.435570 11866 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.428936 12166 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.429821 12144 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.248106 12240 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.360737 11994 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:18.259100 11940 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:18.424714  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84266.2us
I20260709 12:27:18.321203 12173 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.405664 12114 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:18.425984 11996 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.002s
I20260709 12:27:18.417863 12232 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.433444 12160 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:18.450057  6308 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:18.345647 12243 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.449522 11866 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.435388 12179 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:18.350565 12242 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.476534 12243 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:18.476907 12242 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:18.427047  6310 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:18.477618 12235 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:18.478087 12235 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:18.478628  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.478994 12235 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.480103  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.480918  6310 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:18.481971  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.436884 12211 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.462808  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.463783  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:18.444972 12117 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.465377 12253 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:27:18.467028 11940 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:18.455569  7335 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:18.490945 12117 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.461619 12114 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:18.498922  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.500250 12117 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.500576  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.500830 12117 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:18.501266 12117 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.501375  6304 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:18.453447 11994 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.484018  6308 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:18.502166 11994 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:18.486761 12232 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.504595 12232 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:18.480296 12235 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:18.447115 12087 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:18.501672 12117 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:18.510046  6308 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:18.488579  6310 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:18.515239  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 291652us
I20260709 12:27:18.436067 11814 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.546118  6310 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:18.564013  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.565027  6384 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:18.566325  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.572057  6304 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:18.508962 12232 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:18.459954 12236 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:18.590632  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 255570us
W20260709 12:27:18.595259  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 354382us
I20260709 12:27:18.556169 12173 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:18.598431  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 264683us
I20260709 12:27:18.610699  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.492758 12226 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.612054  6450 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:27:18.579155 12211 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.552703 11866 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.615957  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:18.619242  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.145947  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:18.593797 12091 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.637215  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.597200 12187 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.612578 12203 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.615162 12173 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.645565  6377 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:18.595283 12232 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.611725 11996 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:27:18.606604 12144 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:27:18.642915  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.628563 12242 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:18.623934 12211 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.642066 12091 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.612049 12087 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.673995  7331 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:18.695087  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 135221us
I20260709 12:27:18.696259 12285 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.732805  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.737057 12144 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:18.733517 12187 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:18.672518 12242 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:18.673906 12279 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:18.736277  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 120791us
I20260709 12:27:18.699607 12285 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.653776 12173 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.742805 12293 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:18.752476  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:18.747247 12144 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:18.692800 11996 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 12:27:18.756088  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 142320us
I20260709 12:27:18.757155  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:18.743891 12240 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.762112  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.762296 12091 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:18.763587 11996 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.784173 11996 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:18.784698 11996 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:18.764050 12187 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:18.785184 12187 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:18.784858  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.786028 12187 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.786739  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:18.765609 11993 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.782s	user 0.008s	sys 0.000s
I20260709 12:27:18.754727 12293 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:18.753575 12279 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.746927 12141 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.775044 12144 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:18.776372 12271 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:27:18.792358  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:18.793026  7331 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:18.781535 12092 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.788066  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.759479 12283 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:18.795260  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.796563  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:18.796702  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.798435  7335 raft_consensus.cc:2368] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:27:18.798641  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.800182  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:18.797339 12283 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.801172 12283 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:18.790225 12272 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:27:18.749339 12276 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
W20260709 12:27:18.790524 12274 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:27:18.795784 12240 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:18.790140 12278 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:18.807338  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:18.791033 12187 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:18.812570  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.806432 12160 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.815639  6305 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:18.790082 12289 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:18.807870 11915 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:18.848408 11915 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:18.849551 11915 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:18.807274 12273 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:27:18.815160 12291 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:18.822953 12243 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.823662 11996 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.828641  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 848) took 1108 ms. Trace:
I20260709 12:27:18.831822 12279 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.832490  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:18.835096  6301 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:18.925351  6384 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:18.815076 12232 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:18.830029 12114 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.810268 12203 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:18.841074 12283 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.847641  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2618) took 1270 ms. Trace:
I20260709 12:27:18.930764  6380 rpcz_store.cc:276] 0709 12:27:17.577397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.123149 (+545752us) service_pool.cc:224] Handling call
0709 12:27:18.123599 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:18.124087 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:18.124130 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.124769 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.456752 (+331983us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:18.456799 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.456872 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.847375 (+390503us) raft_consensus.cc:1685] finished
0709 12:27:18.847403 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.847570 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2427,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":128,"thread_start_us":331521,"threads_started":1,"wal-append.queue_time_us":488096}
W20260709 12:27:18.818586  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.817154 12091 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:18.791200 11993 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.784181 12242 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.833861 11866 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:18.920514  6378 rpcz_store.cc:276] 0709 12:27:17.720501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.227522 (+507021us) service_pool.cc:224] Handling call
0709 12:27:18.227833 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:18.235519 (+  7686us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:18.625416 (+389897us) log.cc:844] Serialized 14 byte log entry
0709 12:27:18.750673 (+125257us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.755694 (+  5021us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.755832 (+   138us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:18.755861 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.755931 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.828400 (+ 72469us) raft_consensus.cc:1685] finished
0709 12:27:18.828428 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.828579 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10492,"prepare.queue_time_us":209387,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":6870,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":8186,"thread_start_us":399767,"threads_started":2,"wal-append.queue_time_us":175788}
I20260709 12:27:18.924515 12034 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:18.940004  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 139884us
I20260709 12:27:18.821308 12092 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
W20260709 12:27:18.941641  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 137635us
I20260709 12:27:18.917004 12291 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:18.918061 12158 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.928643 12203 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:18.946470  7285 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:18.938650 12091 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:18.945633  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 847) took 1280 ms. Trace:
I20260709 12:27:18.927471 12117 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.835096 12160 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.938876 12166 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:18.923666 11994 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.946756 12279 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:18.926479 12243 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.956795  6379 rpcz_store.cc:276] 0709 12:27:17.665234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.128971 (+463737us) service_pool.cc:224] Handling call
0709 12:27:18.129330 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:18.131491 (+  2161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:18.131532 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.132157 (+   625us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.409992 (+277835us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:18.410035 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.410103 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.945453 (+535350us) raft_consensus.cc:1685] finished
0709 12:27:18.945471 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.945583 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1587,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":277395,"threads_started":1,"wal-append.queue_time_us":424345}
I20260709 12:27:18.964169  6351 leader_election.cc:400] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:27:18.969868  6351 leader_election.cc:403] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:18.971357  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.856302 12293 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.955713 12203 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:18.971980  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.972154  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.950021 12034 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:18.972589  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.022465  6381 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:19.022802  6383 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:18.966049 12279 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.982312 12147 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:18.985675 12236 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:18.987146  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:18.987775  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:18.961575 12092 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:27:18.989223 12159 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.990003 12144 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:18.975888 12166 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:18.995584 12186 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.006768 12211 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.007740  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.008438 11993 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:18.954893 12289 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:19.016566  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.030881 12279 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:18.958043 12091 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.019780  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.024818  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.026213  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.027179  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:18.965268 12276 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:19.033689  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.086265  6458 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:19.034461  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:18.955662 12158 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.035804  6377 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:19.045812  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:19.070465  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.074327  6379 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:18.952520 12291 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.091231  6454 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
W20260709 12:27:19.091578  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.093083  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.069761  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.076615  6453 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:19.074172 12289 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:19.100376 12289 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.101936 12289 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:19.095731  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.089259  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:19.118359  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.037240 12147 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.127475  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.055531 12159 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.091158 12232 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.131527 12186 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:19.135497 12299 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.133127 12240 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:19.145180 12240 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.145674 12240 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.146989 12285 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:19.102986 12114 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:19.172566 12294 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:27:19.173591 12295 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:27:19.127748 12211 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:19.144176 12232 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.178905 12034 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.180408 12166 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.193743 12293 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:19.199617  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 958545us
I20260709 12:27:19.218899  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.219767  6310 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:19.201725  6379 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:19.203471  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.204206  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.225955  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:19.228245  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.229149  6377 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:19.204924  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.205422  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.231051  6302 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:19.231199  6308 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
W20260709 12:27:19.133652 12225 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.014s	user 0.012s	sys 0.003s
I20260709 12:27:19.173486  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 2) took 2186 ms. Trace:
W20260709 12:27:19.154650 12298 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:27:19.236125  6379 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:19.237442  6305 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:19.237915  6309 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:19.232625  6376 rpcz_store.cc:276] 0709 12:27:16.986989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.341367 (+1354378us) service_pool.cc:224] Handling call
0709 12:27:19.173018 (+831651us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b5400065254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ad735 00007f6fa826dd29 00007f6fa17ec1d4 00007f6fa17ec162 00007f6fa17ec091 00007f6fa17ec00c 00007f6fa17eb2c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:19.173351 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425845632}
I20260709 12:27:19.239934  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:19.209017  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.240828  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:19.241577  7331 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:19.243455  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:19.240828  6382 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:19.209604 12093 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.236907  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:19.247224  6304 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.253130  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.254290  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.255131  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:19.255986  6310 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.262230  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:19.262252  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:19.264292  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:19.264911  6454 raft_consensus.cc:606] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:19.265252  6454 raft_consensus.cc:614] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:19.207142 12297 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:19.270051  6377 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:19.271148  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:19.267309  6378 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:19.266522  6424 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16.
I20260709 12:27:19.272377  6424 leader_election.cc:403] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:19.273723  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.217051 12283 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.224478 12034 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.250592 12225 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.281198 12303 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.252105 12114 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:19.289350  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 301646us
I20260709 12:27:19.290001  6379 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:19.277034 12172 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:19.295379  6384 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:27:19.278851  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:19.217584 12211 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.270574 12297 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:19.291235  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.311645  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.312994  6310 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:27:19.323647  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.286397 12205 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16.
W20260709 12:27:19.313591  6348 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 1.564s (SENT) [suppressed 9 similar messages]
I20260709 12:27:19.325713  6310 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:19.245901 12093 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.327059  6302 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:19.321291  6307 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15.
I20260709 12:27:19.329708 12093 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.315259  6382 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:19.298298 12117 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.301745 12283 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:19.341549  6302 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:19.342348  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:19.348732  6379 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.281699 12210 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:27:19.317392 12226 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:27:19.320699 11993 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:19.389355  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2477) took 1462 ms (client timeout 1839 ms). Trace:
W20260709 12:27:19.347373 12187 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.514s	user 0.014s	sys 0.000s
I20260709 12:27:19.331677 12297 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:19.305900 12034 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.333263 12172 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:19.389966  6300 rpcz_store.cc:269] 0709 12:27:17.926712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.927227 (+   515us) service_pool.cc:224] Handling call
0709 12:27:19.389083 (+1461856us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54000938d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:19.389274 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358370688}
I20260709 12:27:19.311152 12114 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.309410 12303 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:19.443075 12292 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:27:19.444912 11996 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.370097 12283 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.443743 12158 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.457820 12172 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.451603 11993 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:19.494282  6304 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:19.509491  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.512912  6379 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:19.514449  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.468178 12290 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:19.521495  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 432916us
I20260709 12:27:19.495461 12280 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:19.530609  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 481d3efac77d41e6bba0961a140626dc: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:19.466885 12240 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.499392 11915 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.551714  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.565085  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:19.521207 12187 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:19.570089  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 256507us
I20260709 12:27:19.524865 12092 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:19.494867 11996 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.577265  6305 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:19.592067  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 330808us
I20260709 12:27:19.601214  6376 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:27:19.602903  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 290670us
I20260709 12:27:19.567998 11915 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.614949 11915 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.605688  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.616858 11915 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.617426 11915 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.571861 12232 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.619902 11915 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.620877 12309 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:19.607113 12158 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:19.625211 12158 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.627434 12158 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:19.629092  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.631662 12283 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:19.572101 12187 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:19.632354  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 884) took 1626 ms (client timeout 1935 ms). Trace:
W20260709 12:27:19.632798  6307 rpcz_store.cc:269] 0709 12:27:18.008826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.112821 (+103995us) service_pool.cc:224] Handling call
0709 12:27:19.632164 (+1519343us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002d23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:19.632294 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2770631552}
I20260709 12:27:19.633234  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:19.633705 12091 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
I20260709 12:27:19.565613 12280 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.527959 12290 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.637656 12234 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:19.607954 11866 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.598037 12225 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:19.609102 12301 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 12:27:19.634615  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.659425 11866 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.660421  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.661621 11866 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.636211 12319 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.614524 11993 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.641163 12168 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.642318 12141 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:19.634341 12091 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.639331 12159 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.636989  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.643182 12303 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.605556 12211 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.636063  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.607169 12240 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.650233 12287 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.634193  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.664412  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:19.671952  6308 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:19.687481  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.672462  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.674583  6302 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:19.679066  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:19.692545  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.693567  6305 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
W20260709 12:27:19.682049  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65440.5us
W20260709 12:27:19.679584  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 84530.4us
I20260709 12:27:19.689888  6458 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:19.699148  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:19.698804 12326 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.075s	user 0.000s	sys 0.008s
I20260709 12:27:19.702158  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.663793 12319 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.703166  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.703426  6458 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:19.670706 12302 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:19.704403  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.705154  6454 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:19.705319  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.706017  6457 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
W20260709 12:27:19.614339 12289 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.512s	user 0.013s	sys 0.000s
I20260709 12:27:19.670878 12092 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:19.706670 12310 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:27:19.634265 11996 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:19.700567 12315 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.078s	user 0.008s	sys 0.000s
W20260709 12:27:19.681627 12296 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.538s	user 0.000s	sys 0.004s
I20260709 12:27:19.685544  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.632326 12187 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:19.642832 12093 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.635095 12114 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:19.699880 12273 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:27:19.705753 12200 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:19.710391  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:19.632812 12034 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:19.730513 12299 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.590s	user 0.003s	sys 0.009s
I20260709 12:27:19.665512 12168 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.744633  6457 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:19.767649  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.733356  6309 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:19.733139  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.739696  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:19.793884  6376 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:19.734397  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.704825 12319 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.663791 12243 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.670593 11994 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.672811 12234 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:19.743412  6378 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:27:19.707578 11866 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:19.740688  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:19.742378 12022 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:19.796446  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62179.9us
I20260709 12:27:19.746124 12227 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:27:19.717634 12276 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.717844 12205 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:19.751068 12320 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.681703 12287 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:19.807016  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66457.9us
I20260709 12:27:19.796748 12323 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:19.762512 12144 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.000s
I20260709 12:27:19.817643 12320 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.814162  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:19.804461  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94194.7us
I20260709 12:27:19.819303 12287 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.819761  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:19.819998 12320 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.815297  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.743577  6204 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 16, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 16 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:19.857975  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:19.871304  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57227.7us
I20260709 12:27:19.872670  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.662400 12297 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.742612 12210 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:27:19.872049  6385 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 12.
I20260709 12:27:19.874342  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.878468  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:19.880355  6453 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:19.880836  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:19.881609  6454 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
W20260709 12:27:19.872248 12291 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.765s	user 0.009s	sys 0.003s
I20260709 12:27:19.806697 12091 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:19.812254 12243 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:19.812250 12034 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.820688 12316 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:19.756119 12324 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 12:27:19.812852 11866 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.812721 12299 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:19.870527 12323 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:19.878384 12320 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:19.977027  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:19.922946  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48731.8us
I20260709 12:27:19.813396 12114 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:19.806113 12168 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.815366 12022 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:19.807117 12093 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.972831 12034 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.882776 12285 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:27:19.920058 12234 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.999735 12234 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:19.919811 12203 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.887s	user 0.016s	sys 0.000s
I20260709 12:27:19.879403 12280 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.002508 12203 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.004171  6384 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:19.987897  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:19.992911 12320 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:19.968477 12319 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:19.994714  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.005681 12319 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:20.006155 12319 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.006567 12319 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.007474  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.007750  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:20.008229 12313 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.008479  6377 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:20.008692 12313 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.009068 12313 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.011196 12322 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:19.999945 12168 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.011693 12322 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:20.000480 12093 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.012115 12322 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:20.013149  6301 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:19.799008 12200 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.015267 12200 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.015465  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.015782 12200 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:20.016249 11994 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:20.016681 11994 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:20.017048 11994 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.017287  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.019665  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.020134 12313 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.021041  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.021799  6457 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:20.022698 11927 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:20.022835  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.023547 11927 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:20.022739  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.025039 11927 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.968642 12211 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.879405 12303 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:20.004884 12320 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.026342 12211 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.016237 12200 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:20.028469 12211 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.988082 12316 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:20.029484 12323 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.029650 12316 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.023576 12203 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.001704 12034 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.926363 12210 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:20.035118  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:20.044618  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 298935us
I20260709 12:27:20.045643  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.046901  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:20.001403 12328 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:27:20.024184  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.049540  6458 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:20.018342  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.028143 12200 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.058342 12210 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:20.058975 12210 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:20.059751 12210 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:20.060252  6383 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.019s	sys 0.040s
W20260709 12:27:20.060658  6383 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.019s	sys 0.040s
I20260709 12:27:20.060961 12210 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:19.918414 12290 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:19.926782 12291 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:19.871842 12276 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.063405 12276 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.040848 12114 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:19.972597 12166 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:20.039726 12279 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.950s	user 0.008s	sys 0.006s
I20260709 12:27:19.977347 12242 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:19.974432 11866 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:19.969954 12287 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:19.874399 12205 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:19.886744 12236 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.147s
I20260709 12:27:20.058225 12234 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:20.064880 12034 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:20.041271 12319 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.066025  6304 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:20.072183 12091 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:20.022248 12313 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:20.079471  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61185.4us
I20260709 12:27:19.974592 12243 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.043960 12280 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.083863  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.083220  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.085645  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.086112 12211 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:20.089040  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.063445 12291 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.090230  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.048483 12093 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.072769 12299 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:20.071975 11996 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.078039 12242 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.071943 12144 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:20.132522  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.008s	sys 0.019s
W20260709 12:27:20.180747  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.009s	sys 0.019s
I20260709 12:27:20.132606  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.137580  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.137748  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.085685 12303 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:20.150663  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.152727  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:20.087704  6300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.024s	sys 0.038s
W20260709 12:27:20.187809  6300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.024s	sys 0.038s
I20260709 12:27:20.189152  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.189760  6300 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:20.190135  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.159919  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.160681  6302 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:20.191030  6454 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:20.094674 12034 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.167704  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.133155 12236 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:20.169401  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.076220 12279 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.096606  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.079372 12166 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.093297 12158 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:27:20.086499 12287 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:20.181241  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 585714us
I20260709 12:27:20.162770 12280 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.185604  6308 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:20.154729 12211 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:20.153695  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70569.6us
I20260709 12:27:20.196372 12034 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.194895  6303 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:20.141700 12324 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:27:20.167202 12147 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.141321 12091 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.193681  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.197252  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.261684  6457 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.212821 12297 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:20.195359  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.266814  6300 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:20.267741  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.202155  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.268035  6457 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:20.153419 11993 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:27:20.184703  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:20.207125  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 326373us
I20260709 12:27:20.186964 12303 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.270226 12303 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:20.270967 12320 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.271876  6454 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:20.273842 12320 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.277735 12303 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.167748 12203 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:20.278705 12203 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.279095 12203 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:20.170939 12093 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.214653 12291 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:20.283028 12291 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.283370 12291 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:20.284107 12093 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:20.201113 12293 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.942s	user 0.011s	sys 0.005s
I20260709 12:27:20.214030  7331 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:20.286230  6300 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:20.218143  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.217612 12211 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.287050  6308 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:20.288024  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.288532 12203 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.289290  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.289959  6309 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:20.290675 12203 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.196172 12144 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.225394 11994 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.229400  7288 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15.
W20260709 12:27:20.334697  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 105430us
I20260709 12:27:20.335464  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.335979 11994 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.336934  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.231657 12335 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:20.171667 12114 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.210116 12173 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.358194 12211 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.360484 12211 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.360751 12117 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:20.361316 12117 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:20.362236 12117 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.364821 12318 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.365281 12318 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.365615 12318 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.370150  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.371181  7335 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:20.181319 12340 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:27:20.239684  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.376452  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.377205  7320 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:20.216104  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.240871  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.179807 12299 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.232393 12243 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:20.243479  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.250823 11866 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:20.251253 12092 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.001s
I20260709 12:27:20.252231 12280 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.254117 12341 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:20.190237 11996 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.254711  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:20.255867  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.393802  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.258294 12287 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.260591 12279 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.408557  6308 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.262987 12091 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.269968  6383 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:20.270102 12242 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.209824  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:20.207068  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 110586us
I20260709 12:27:20.291204 12205 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.290261  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.418686  6305 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:20.292390 12303 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.417717  6452 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:20.420658  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.421604  7316 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:20.201404  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.195792 12236 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.294261  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.426285 12236 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:20.426471  6309 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:20.299454  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2502) took 1004 ms. Trace:
I20260709 12:27:20.427709  6304 rpcz_store.cc:276] 0709 12:27:19.295459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.499992 (+204533us) service_pool.cc:224] Handling call
0709 12:27:19.534268 (+ 34276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.101174 (+566906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:20.299148 (+197974us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54002e7f54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:20.299376 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1222394624}
I20260709 12:27:20.429158  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.429785  6304 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.431164  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260709 12:27:20.306432 12187 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.004s
I20260709 12:27:20.314975 12324 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:20.434743 12299 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.434943 12324 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:20.435920 11866 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.435860 12324 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:20.315547  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45534.5us
W20260709 12:27:20.437183  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:20.437698  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.438108  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:20.439880  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:20.331375 12319 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.440299 12205 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:20.440871 12234 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.441336 12034 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.344193  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:20.444597  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 100524us
I20260709 12:27:20.354573 12144 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.403219 12243 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:20.378428  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 240840us
W20260709 12:27:20.386373  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 170310us
I20260709 12:27:20.389190  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.392992  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.458909  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.400154  6303 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:20.393123  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.404178  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.410118  7320 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:20.415650  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:20.292419 12289 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.419636  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.465682  7314 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.419751  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.465912  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.466509  6459 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:20.466619  7320 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:20.390365 12092 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:20.333422 12322 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.423537 12341 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:20.496829 12234 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:20.497493 12341 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.498980  6304 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
W20260709 12:27:20.501929  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 207352us
W20260709 12:27:20.504472  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44768.2us
W20260709 12:27:20.505824  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 264857us
I20260709 12:27:20.453603  6308 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:20.385252  7335 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:20.461748  6451 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.462786  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.507100  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:20.468701  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.508131  7320 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
W20260709 12:27:20.469159  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65061.2us
I20260709 12:27:20.434779 12187 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.472353 12034 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.455179 12293 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:20.475790  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.437211 12324 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.483408  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.467644 12091 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:20.519814  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:20.491554 12352 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:20.493567  6309 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:20.494376 11996 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.495796  7314 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:20.526054  6385 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:27:20.502908  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.511324  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:20.515913  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.516357  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.516669  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.530786  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.531584  6305 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:20.528677  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.517339  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:20.457324 12243 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.521770  7316 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:20.521517  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45827.6us
I20260709 12:27:20.447549  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:20.522558  6383 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:20.534988  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.523948  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.490366 12322 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:20.539222  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 91889.2us
I20260709 12:27:20.540550  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.541220  7320 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:20.502869 11866 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.534018  6459 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:20.535585  6309 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:20.536309  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.484278 12289 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.532325  6307 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:20.547082  6451 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:20.547343  6378 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:20.549805  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.550387  6309 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:20.551175  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.551780  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:20.552590  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.553009  6305 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:20.530049  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.555800  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.557680  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.558720  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.560412  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:20.525315 12280 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:20.532672  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.566777  6383 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:20.536252 11996 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:20.536029 12168 raft_consensus.cc:2764] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 12:27:20.573331  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:20.520396 12291 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.540813 12293 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:20.539366 12243 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.537814 12034 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:20.573972  6380 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.559319  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.575361  7320 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:20.561362 11866 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.561949 12211 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 12:27:20.577868  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.543965 12340 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:20.562812 12203 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:20.563050 12242 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:20.590786  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:20.567718 11994 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:20.592975  6383 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:20.587121  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:20.591914  6303 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 12: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:20.580454 12339 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:27:20.541417 12091 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.599864 12091 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.600531 12091 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:20.601253 12091 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:20.601743 12091 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:20.592447 11927 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:20.607537 11927 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.607983 11927 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:20.610762 12236 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.611574 12187 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.610939 12091 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.570562 11996 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:20.612704 11996 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:20.613453 12236 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:20.567358 12144 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.614377 12144 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.614866 12144 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.614948  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:20.615442  6380 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:20.615895  6376 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:20.594105  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.616052  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.567934 12354 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.592717 12205 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:27:20.618827 12354 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.620759 12318 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.571151 12290 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:20.625072  6455 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:20.628223 12291 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.629973  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:20.630184  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.630780  6377 raft_consensus.cc:2368] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13.
I20260709 12:27:20.631186  7320 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:20.568734 12280 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.632306 12280 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:20.632596 12022 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.634722  6302 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:20.635484  6303 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:20.636647 12319 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:20.637058 12319 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.637388 12319 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.640204  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.642524  7320 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:20.650295  7319 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:20.651497  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.655890  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.656178  6305 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:20.664777  6459 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:20.666281  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.587929 12034 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.678907  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:20.679121  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 85130.6us
I20260709 12:27:20.679885  6451 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:20.680047  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.682116  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.615137 12242 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.639401 12203 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:20.685935  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44128us
I20260709 12:27:20.634367 12022 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:20.725281  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:20.709826 12319 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.727262  6383 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:20.700168 12321 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:20.669153 12236 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:20.681702 12117 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:20.705226  6301 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2686) took 1503 ms (client timeout 1520 ms). Trace:
I20260709 12:27:20.705760 12354 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.706477  6310 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:20.710362 12160 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.682816 12323 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:20.712440  6303 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:20.713908  6377 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:20.716676 12320 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:20.718887 11866 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.719885 12318 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.720224  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.681125 12034 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:20.723946  6376 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:20.724730  6380 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:20.657084 12309 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:20.718226 11994 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:20.684163 12242 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:20.728533  6301 rpcz_store.cc:269] 0709 12:27:19.202208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.202338 (+   130us) service_pool.cc:224] Handling call
0709 12:27:20.705037 (+1502699us) spinlock_profiling.cc:243] Waited 766 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:20.705154 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1762748800}
I20260709 12:27:20.703377 11927 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.694433 12203 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:20.748910  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 188605us
W20260709 12:27:20.731331 12358 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.073s	user 0.004s	sys 0.000s
I20260709 12:27:20.694186 12168 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:27:20.803929  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:20.804406  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 122378us
I20260709 12:27:20.730695 12321 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:20.788172  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.749845 12320 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:20.807904 12320 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:20.810827  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.748826 12226 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:20.808048  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.796149 11927 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.809299  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.816637  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.816732  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.817410  6308 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:20.812690  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 125405us
I20260709 12:27:20.746757 12236 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:20.820109  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 287490us
I20260709 12:27:20.829722  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:20.804093 12309 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:20.832528  6300 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:20.834419  6308 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:20.834533  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.835134  6300 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:20.840713  7319 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:20.813372 12147 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:20.842242 12319 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.842860 12147 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.843493 12147 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.847091 12299 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:20.847760  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:20.847966 12299 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:20.848397  6301 raft_consensus.cc:2376] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15. Re-sending same reply.
I20260709 12:27:20.849694  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:20.849321 12299 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.850643 12291 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.851152  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.852082 12092 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.852530 12092 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.852916 12092 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:20.746927 12117 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.855113 12187 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:20.856619 12117 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.851167 12291 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.859695 12291 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.862775  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:20.863757 12287 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:20.864164 12287 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.864523 12287 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:20.868561  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.808917 12160 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.869715  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:20.873687  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.831908 12323 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.875147 12366 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.875396 12323 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.875734 12366 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.877956 12366 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.877964 12144 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.878829  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.879287  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:20.879632 12323 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:20.879793  7335 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:20.880430 12309 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.880928  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:20.882100 12291 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.882825  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.883270 12144 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:20.878896 12234 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:20.882153 12114 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:20.884986  6351 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:20.887254  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:20.888023  7317 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:20.884131 12117 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:20.971129  7318 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
W20260709 12:27:20.971928  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 87087us
I20260709 12:27:20.962108  7335 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:20.974846  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:20.832075 11994 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.976718  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:20.979460  6380 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:20.817823 12280 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:20.885946 12318 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:20.973924 11866 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:20.994416  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:20.887363 12323 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.968887 12114 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:20.867337 12340 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.121s
W20260709 12:27:21.011168 12357 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:27:21.046754 12243 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.049847 12243 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.016156 12365 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:21.050345  6376 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:21.050420  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.050995 12365 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:21.051465  6377 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:21.051736  6384 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:21.052099 12365 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.026382 12166 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:21.061336  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2694) took 1544 ms (client timeout 1563 ms). Trace:
I20260709 12:27:21.064451  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:21.066442  6301 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:20.989778 11994 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.983340 11996 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.981182 11927 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:27:20.882843 12291 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.033210 11866 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:20.985137 11814 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:27:21.050714  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.024606 12236 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:20.889253 12160 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.087810  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:21.150326  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.151186  6451 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:21.152026  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.088984  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:21.153151  6457 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:21.154177  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.090423  6453 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:21.099581  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.155670  6459 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:21.091008  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.107452  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:21.157900  6458 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:27:21.158290  6458 pending_rounds.cc:85] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 7
I20260709 12:27:21.158545  6458 pending_rounds.cc:107] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:27:21.159036  6458 raft_consensus.cc:2889] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:21.110845  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.115759  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:21.161585  6450 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:21.116966  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:21.163920  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:21.165593  6304 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:20.981401 12211 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 12:27:21.130887  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 154238us
I20260709 12:27:21.045032 12114 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:20.884285 12321 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:20.990893 12318 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:21.266018  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.266794  6452 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:21.061950  6307 rpcz_store.cc:269] 0709 12:27:19.517248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.633719 (+116471us) service_pool.cc:224] Handling call
0709 12:27:21.061023 (+1427304us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000aa854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:21.061254 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1952519424}
I20260709 12:27:21.132884 12340 consensus_queue.cc:1243] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:27:21.168296  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 77438.9us
I20260709 12:27:21.175639  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:21.270905  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:21.183310  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.184690  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:21.272924  6451 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.186400  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:21.188077  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:21.246881  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:21.247854  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:21.248467  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.275883  6380 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:21.277236  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:21.248826  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.249029  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.252815  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:21.279106  6383 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:21.045449 12323 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.281418  6382 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:27:21.280539  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.115815 12034 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.288451  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.289192  6380 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:21.290922  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.291636  6384 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:21.291726  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:21.292052  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.293030  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.293792  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.294685  6377 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:27:21.133318 12203 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:21.302541  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 178276us
I20260709 12:27:21.309660 12323 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:21.311694 12022 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.004s
I20260709 12:27:21.317544  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:21.318336  6307 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:21.299314 12144 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.319399  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:21.319269 12144 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.320180  6307 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.324501 12289 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.337486  7331 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:21.339437 12368 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.308s	user 0.001s	sys 0.001s
I20260709 12:27:21.342103  6305 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:21.343281  6305 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.344467  6305 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:21.347761 11994 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.351169  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:21.351938  6383 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:21.354002  6456 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:21.364032  6381 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:21.277422 12318 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:21.372741  6385 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:21.365612  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.375375  6382 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:21.375468  6451 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:21.363488 12320 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.544s	user 0.016s	sys 0.001s
I20260709 12:27:21.302981 12321 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.399581  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:21.406383 11814 consensus_queue.cc:1243] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:21.411142 12022 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:21.379372 12144 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.414050 12323 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:21.420995 11994 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:21.383085 12302 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:27:21.449244  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:21.434887 12211 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:21.409561 12320 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.378270 11866 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:21.470758  6300 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.000s
I20260709 12:27:21.471175  6300 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:21.472985  6305 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.470783  6452 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:21.478345  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:21.480104  6307 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:21.481324  6376 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:21.455565 12280 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:21.464993  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:21.466926  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67460us
I20260709 12:27:21.488934  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:21.468448 12147 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.624s	user 0.004s	sys 0.008s
I20260709 12:27:21.490618 12147 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.453732 12211 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.493561 12211 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:21.500610 12320 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.501648 12320 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.503038 12361 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.503798 12361 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.507707 12324 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:21.508774 12291 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:21.508950 12340 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:21.510742 12377 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:21.511332 12203 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.514424 11814 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:21.515962 12376 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:21.516418 12376 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:21.517421 12376 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.508780 12324 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:27:21.522118 12022 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:21.525696  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:21.527236  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:21.528278  7318 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
W20260709 12:27:21.528355 12362 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.675s	user 0.007s	sys 0.005s
I20260709 12:27:21.529402  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:21.453794 12323 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:21.458698 11994 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:21.438503 12370 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:27:21.506246 12361 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:21.552196  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87339.6us
I20260709 12:27:21.552977  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:21.544378  6383 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:21.538769  6459 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:21.464879  6450 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
W20260709 12:27:21.548007  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 262725us
W20260709 12:27:21.549093  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 256113us
I20260709 12:27:21.565627  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:21.549664  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 279813us
I20260709 12:27:21.567857  6348 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
W20260709 12:27:21.550118 12309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.022s
I20260709 12:27:21.576872 12022 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:21.577358 12200 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:21.577994 12376 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:21.578689 12022 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:21.579319 12344 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:21.579175 12022 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:21.586305 12022 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.587936 12022 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:21.538251 12160 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:21.552433 12287 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.004s
W20260709 12:27:21.575095  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.596177 12200 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:21.596884 12200 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:21.576870 12309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.007s	sys 0.022s
I20260709 12:27:21.569052  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:21.569204  6348 leader_election.cc:403] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:21.502321 12320 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.574858 12340 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:21.627046  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:21.567981 12382 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:21.666806  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 99139.4us
I20260709 12:27:21.683233  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:21.678186  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 73979us
W20260709 12:27:21.679152  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 73408.5us
I20260709 12:27:21.607633 12344 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:21.679425  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52515.9us
I20260709 12:27:21.686228  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.687633  6308 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:21.687994 12226 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:21.688560 12226 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:21.688629 12320 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:21.688974  6308 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:21.593600 12034 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.690963  6308 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.603014 12236 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:21.659272 12280 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.695806  7287 leader_election.cc:400] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13.
W20260709 12:27:21.695989  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81194.6us
I20260709 12:27:21.601481 12303 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:21.696344  7287 leader_election.cc:403] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:27:21.610163 12242 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.803s	user 0.012s	sys 0.000s
W20260709 12:27:21.629151 12092 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.776s	user 0.010s	sys 0.003s
I20260709 12:27:21.701010 12302 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.705045  6308 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:21.676926 12340 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:21.708227  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.603241 12287 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.712769  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.711937  6308 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.669646 12160 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:27:21.729269  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:21.736232  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.744810  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:21.746213  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:21.701690 12365 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:21.706771 12384 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:21.611302 11994 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.703233 12378 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:21.691998 12344 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.712082 12303 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:21.754951  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68470.9us
I20260709 12:27:21.697433 12383 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14.
I20260709 12:27:21.726827 12340 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:21.721280 12372 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:21.675601 12323 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.765861 12352 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:21.706871 12280 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.759009 12173 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:21.767553 12365 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:21.760162  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:21.711110 12092 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.784874  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:21.785068 12092 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.737703 12117 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 12. Current term is 13.
I20260709 12:27:21.785744  7332 raft_consensus.cc:2368] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19.
I20260709 12:27:21.787635 12144 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:21.788071 12144 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:21.788436 12144 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:21.789165 12147 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:21.778082 12372 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:21.791189 12372 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.793555  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 899) took 1275 ms. Trace:
I20260709 12:27:21.793992  6303 rpcz_store.cc:276] 0709 12:27:20.518077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.636163 (+118086us) service_pool.cc:224] Handling call
0709 12:27:20.712227 (+ 76064us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.747696 (+ 35469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.793364 (+1045668us) spinlock_profiling.cc:243] Waited 75.7 ms on lock 0x7b54000b3950. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:21.793493 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174219520}
I20260709 12:27:21.769358 12299 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:21.795198 12299 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:21.795615 12299 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:21.796548  6456 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3219) took 1557 ms (client timeout 1549 ms). Trace:
I20260709 12:27:21.796346 12299 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:21.796998  6456 rpcz_store.cc:269] 0709 12:27:20.239185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.239367 (+   182us) service_pool.cc:224] Handling call
0709 12:27:21.796297 (+1556930us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400266e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:21.796411 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2151512832}
I20260709 12:27:21.773192 12382 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.798280 12382 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.798624  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2499) took 2504 ms. Trace:
I20260709 12:27:21.799041  6310 rpcz_store.cc:276] 0709 12:27:19.294666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.333848 (+ 39182us) service_pool.cc:224] Handling call
0709 12:27:20.706271 (+1372423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.731356 (+ 25085us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.798365 (+1067009us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b5400020dd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:21.798539 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3068543872}
W20260709 12:27:21.799433 12114 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.001s
I20260709 12:27:21.799923 12114 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.710184 12242 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.770687  6378 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:21.768392  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:21.823433  6383 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:27:21.734020 12203 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:27:21.815250 12022 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:27:21.713006 12302 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.757408  6304 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:21.765318 12378 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.778331  6379 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [12.7-13.8]
I20260709 12:27:21.781307  6274 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
W20260709 12:27:21.831444  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50268.5us
I20260709 12:27:21.832324  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:21.782637 12386 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:27:21.782812 12380 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:27:21.782280 12303 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.737454 12289 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:21.797359 12299 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:21.769177 12200 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:21.746702 12291 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:21.766788 12384 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:21.752235 12320 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:27:21.804420  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:21.786270  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1085) took 1331 ms. Trace:
I20260709 12:27:21.811877 12114 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.812153 12211 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:21.813081  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:21.815296 12117 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:21.818290 12290 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:21.833060 12378 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:21.819111 12168 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:21.847493 12289 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:21.819723 12287 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.819816 12396 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:21.821478 12344 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:21.752499 12371 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:21.873270  7317 raft_consensus.cc:2368] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19.
W20260709 12:27:21.891144  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 117341us
I20260709 12:27:21.870183  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2651) took 1338 ms. Trace:
I20260709 12:27:21.895270  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:21.895092  6385 rpcz_store.cc:276] 0709 12:27:20.531854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.532033 (+   179us) service_pool.cc:224] Handling call
0709 12:27:20.548987 (+ 16954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.373242 (+824255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.869778 (+496536us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b5400037d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:21.870089 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893910912}
I20260709 12:27:21.897482  6275 leader_election.cc:400] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19.
I20260709 12:27:21.837081  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:21.799443 12392 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:21.863533  7332 rpcz_store.cc:276] 0709 12:27:20.455040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.455414 (+   374us) service_pool.cc:224] Handling call
0709 12:27:21.784616 (+1329202us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:27:21.786175 (+  1559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30676224}
W20260709 12:27:21.900190  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:21.901815  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64821.8us
I20260709 12:27:21.905301  6304 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:21.908763  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:21.867960 12390 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:21.944315 12395 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:21.872330 12289 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:21.948418  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45638.8us
W20260709 12:27:21.948963  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46050us
I20260709 12:27:21.892114 12383 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.884646 12354 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:21.871266 12290 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:21.863368 12401 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:21.875191 12302 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:21.980382  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:21.982640  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:21.997372 12401 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:21.997541  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:21.997831 12401 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.998204 12401 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:21.911078 12352 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:22.000133 12352 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:22.000576 12352 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:21.985517 12293 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.006s	sys 0.012s
I20260709 12:27:22.006186  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.012161  6303 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:21.989041 12354 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:21.986796 12290 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:22.007459 12318 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.003s
I20260709 12:27:22.021507 12391 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:22.026897 12208 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:22.011586 12293 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.007s	sys 0.013s
I20260709 12:27:22.056802 12092 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.057334 12401 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.058523 12208 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.057454 12092 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.059425 12208 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.060662 12092 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.060902 12208 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:21.874058 12384 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.062752 12172 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:22.063256 12172 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.064139 12172 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:21.990252 12383 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.015914 12147 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:22.077492 12147 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:21.908183 12392 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.081089 12382 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:22.082064 12401 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.071319 12293 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:22.095054  7335 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:27:22.098909  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 212707us
I20260709 12:27:22.101069  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.101267  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:22.103904  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:22.105785  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 334898us
W20260709 12:27:22.110599  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 166962us
I20260709 12:27:22.085692 12144 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.119011 12400 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:22.119693 12378 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.120826 12144 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.121928 12378 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:22.053660 12318 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.136713  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:22.139004  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:22.140421  6377 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:22.142930  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.146318  6380 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:22.145776  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:22.122766 12280 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.078474 12147 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:22.149679 12376 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.571s	user 0.006s	sys 0.007s
I20260709 12:27:22.080343 12354 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.163904  7335 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:27:22.164012  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.168529  6377 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:22.154111 12236 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:22.175604  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.176847  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:22.177368  6377 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:22.097317 12290 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.179163 12383 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.119705 12400 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:22.181257 12404 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:22.149758 12205 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:27:22.134331 12340 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.153293 12173 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:27:22.190954  6377 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:22.192857  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.194294  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:22.195917  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.171955 12376 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:22.164186 12200 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:27:22.170745 12144 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:27:22.213267 12376 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.196847 12293 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.232681 12376 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:22.189178 12396 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:22.233372 12396 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.224445  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:22.229612  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:22.221169 12364 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.367s	user 0.003s	sys 0.000s
W20260709 12:27:22.214409 12226 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.004s
I20260709 12:27:22.241623 12318 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.244168 12392 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.209378 12383 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.216771 12372 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:22.250217 12376 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.241662 12226 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.252727 12208 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:22.253268 12208 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.254545 12226 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.254693  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:22.252835 12376 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.256595  7335 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:22.245064 12401 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.254096 12208 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.233243 12352 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.260142  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.261351 12401 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.267199 12423 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.267802 12423 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.269292 12401 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.272296 12352 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.274329 12396 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.277281 12376 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.251924 12289 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.278888  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2953) took 1031 ms. Trace:
I20260709 12:27:22.279541 12340 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:22.280512 12414 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:22.265969 12208 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:22.280972 12414 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.284092  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:22.279318  6381 rpcz_store.cc:276] 0709 12:27:21.247212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.247551 (+   339us) service_pool.cc:224] Handling call
0709 12:27:22.278663 (+1031112us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b5400038254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:22.278824 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179819392}
I20260709 12:27:22.302991  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:22.303714  7314 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:22.305274  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.305918  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:22.306244  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.306833  6381 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:22.264212 12227 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.280206 12289 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.256295 12383 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.293494 12423 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.281873 12414 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.335137  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:22.337065  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:22.337706  6384 raft_consensus.cc:2368] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:22.259094 12372 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.307916 11814 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.344874  6384 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:22.344610 11814 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.350705 12414 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.351249 11814 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.355971  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:22.356810  7314 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:22.339701 12423 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:22.359248 12423 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.359774 12423 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.330278 12227 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:22.378369  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.387521 12383 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:22.362788 12339 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.675s	user 0.014s	sys 0.029s
I20260709 12:27:22.396328  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.388538 12227 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:22.393100 12339 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.703s	user 0.015s	sys 0.029s
I20260709 12:27:22.390529 12293 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:22.403239  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.403996  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.405421  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.405548  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:22.404218  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.404310  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 2967) took 1151 ms. Trace:
I20260709 12:27:22.406587  6454 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:27:22.404848  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.406725  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.406706  6376 rpcz_store.cc:276] 0709 12:27:21.252482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.280194 (+ 27712us) service_pool.cc:224] Handling call
0709 12:27:22.404154 (+1123960us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:22.404251 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457537408}
I20260709 12:27:22.407598  6450 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:27:22.408722  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.409051  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.409478  6450 raft_consensus.cc:2410] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:27:22.410465  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.411059  6459 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:22.413508  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.414136  6452 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:27:22.414547  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.416013  6458 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:22.416139  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.416719  6454 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:22.417182  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.412755 12293 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.434582 12293 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:22.435037  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.435458 12293 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.436072 12293 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.437125  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:22.437707  7314 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:27:22.434374  6453 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:22.441440  6459 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.442936  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:22.443280 12293 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.443787  6301 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:27:22.441748  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:22.445446 12400 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.446872  6456 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:27:22.447824 12242 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.448087  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.448959  6301 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:22.450718  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.451418  6453 raft_consensus.cc:2376] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14. Re-sending same reply.
I20260709 12:27:22.451504  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.451644  6308 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:22.452925  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.429085 12372 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.454085  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:22.454695 12427 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 12:27:22.457944  6459 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:22.457844  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.462702  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.464599  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.465214  6303 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:22.466898  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.469604  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:22.471200  6307 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:22.472456 12375 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.472626  6303 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:22.473048 12375 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.474699  6379 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:22.476748  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.477388  6379 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:22.478143  6301 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:22.478440  6300 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:22.481276  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.481823  6458 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.482669  6454 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:27:22.482793  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.483417  6454 pending_rounds.cc:85] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 10
I20260709 12:27:22.483820  6454 pending_rounds.cc:107] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:27:22.484361  6454 raft_consensus.cc:2889] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.483795  6453 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:22.486544  6452 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:22.475718 12375 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:22.486937  6452 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.495308  6452 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.484404 12423 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.496558  6452 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.489145  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.501798  6458 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:22.482399 12339 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.505709  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.505754  6458 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:22.508795  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3214) took 2313 ms. Trace:
I20260709 12:27:22.509138  6455 rpcz_store.cc:276] 0709 12:27:20.194305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.194497 (+   192us) service_pool.cc:224] Handling call
0709 12:27:20.194783 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.625517 (+430734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.507056 (+1881539us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54002c7c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:22.507195 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989088512}
I20260709 12:27:22.511492  6452 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:27:22.511991  6452 pending_rounds.cc:85] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 12
I20260709 12:27:22.512292  6452 pending_rounds.cc:107] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:27:22.512617  6452 raft_consensus.cc:2889] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.514186  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.505456 12339 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.518743 12339 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.515905 12423 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:22.523782 12398 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.530s	user 0.004s	sys 0.003s
I20260709 12:27:22.526473 12329 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:22.531333 12329 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.532303 12329 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.533926 12329 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:22.537392  6309 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 901) took 2018 ms (client timeout 2364 ms). Trace:
W20260709 12:27:22.537854  6309 rpcz_store.cc:269] 0709 12:27:20.518596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.550998 (+ 32402us) service_pool.cc:224] Handling call
0709 12:27:22.537214 (+1986216us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54000a9e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:22.537337 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4115344256}
I20260709 12:27:22.539366 12211 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.542590 12375 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:22.543157  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:22.546729  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:22.547499 12117 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:22.547888 12117 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:22.548265 12117 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.547334  6382 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:22.551474 12375 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:22.551849 12375 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.552702 12375 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:22.554697  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.557691  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:22.560081 12290 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:22.571732  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.574450  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:22.575402  7316 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.591213  6424 leader_election.cc:400] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:22.592365  6424 leader_election.cc:403] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:27:22.599797  7317 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:22.607429  7314 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:22.608503  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:22.609586 12392 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:22.612684 12236 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:22.613502 12236 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.617825 12236 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:22.656163  7316 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:22.657428  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:22.660311  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.660490  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.593374 12291 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.620263  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:22.619556 12289 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:22.668859  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:22.673116  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:22.678099  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.678721  6454 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:22.679275  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.679591  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.680146  6458 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:22.680249  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:22.680857  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.682864  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.560129 12375 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:22.593070 12377 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:22.691423 12377 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.685176 12289 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.692222 12377 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.663866 12339 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:22.687404  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.689606  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.690440  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.691475  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:22.688927  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.657552 12236 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:27:22.702634 12428 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:22.731794  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:22.683334 12423 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:22.732906 12203 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:22.661334 11993 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:27:22.714545 12339 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:22.731851 12234 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:27:22.717377  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:22.735016 12339 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.683761 12208 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:27:22.736646  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2510) took 1848 ms. Trace:
I20260709 12:27:22.737038  6305 rpcz_store.cc:276] 0709 12:27:20.888219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.345124 (+456905us) service_pool.cc:224] Handling call
0709 12:27:21.472776 (+127652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.473437 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.736425 (+1262988us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b54001b21d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:22.736590 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":292895104}
I20260709 12:27:22.718334 12354 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.718721 12414 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:22.721239 12331 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.722744  6204 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:22.732097 12428 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.732656 11996 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.657404 12361 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:27:22.663480 12429 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:27:22.715390 12289 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.740455  6458 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
W20260709 12:27:22.750910  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44168.2us
W20260709 12:27:22.755990  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2726) took 1505 ms (client timeout 1711 ms). Trace:
I20260709 12:27:22.756506  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 902) took 2237 ms. Trace:
I20260709 12:27:22.756637 12280 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:22.758903 12341 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:22.759078  6204 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 13, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 13 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.775060  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:22.776717  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:22.666169 12291 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.795810  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.802021  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:22.802554  6452 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 12:27:22.802843  6304 rpcz_store.cc:269] 0709 12:27:21.250200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.270630 (+ 20430us) service_pool.cc:224] Handling call
0709 12:27:22.755693 (+1485063us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b540010fc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:22.755886 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1117300096}
I20260709 12:27:22.803745  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.805107  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.796732  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.775501  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:22.807731  6308 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:22.797856  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.753012  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:22.808768  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 174959us
I20260709 12:27:22.811316  6459 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:22.812444  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:22.762615 12428 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:22.821383  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.817792  6204 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:22.823963  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:22.800287  6302 rpcz_store.cc:276] 0709 12:27:20.518826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.638289 (+119463us) service_pool.cc:224] Handling call
0709 12:27:20.647827 (+  9538us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:20.663847 (+ 16020us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:20.663907 (+    60us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:21.056074 (+392167us) spinlock_profiling.cc:243] Waited 13.6 ms on lock 0x7b18003ca780. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a
0709 12:27:21.057901 (+  1827us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:27:21.142243 (+ 84342us) log.cc:844] Serialized 52 byte log entry
0709 12:27:21.446822 (+304579us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:21.736889 (+290067us) log.cc:844] Serialized 14 byte log entry
0709 12:27:21.748670 (+ 11781us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.023622 (+274952us) spinlock_profiling.cc:243] Waited 2.12 ms on lock 0x7b18003ca780. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:27:22.755966 (+732344us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b18003ca780. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287
0709 12:27:22.756129 (+   163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.756230 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:22.756285 (+    55us) raft_consensus.cc:1685] finished
0709 12:27:22.756302 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.756444 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32890,"prepare.queue_time_us":146012,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":103,"raft.queue_time_us":48891,"raft.run_cpu_time_us":436,"raft.run_wall_time_us":16956,"spinlock_wait_cycles":53524480,"thread_start_us":384768,"threads_started":3,"wal-append.queue_time_us":360010}
I20260709 12:27:22.815007  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:22.738276 12117 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.821750  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.822089  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.794379 12344 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:22.825433  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:22.819934  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.832819  6452 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:27:22.794127 12341 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.831794  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:22.850922  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.843982  6274 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:22.852545  6274 leader_election.cc:403] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:27:22.853128  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:22.850126  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:22.849020  6204 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:22.818931 11996 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:22.856843  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:22.762707 12331 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.822793 12160 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:22.826712 12289 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:22.830332 12383 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:22.819015 12211 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:22.762307 12414 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.859347  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:22.865788  6452 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:22.857936 12172 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:22.857538 11927 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:22.870821  6204 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:22.829694 12344 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:22.838439 12400 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:22.868239 12331 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:22.882371  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:22.879096 12168 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.898989  6204 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 13, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 13 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:27:22.884382  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 203214us
I20260709 12:27:22.884593 12383 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:22.861231 11996 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:22.823984 12280 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.865091 12160 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:22.860394  7335 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:22.901502  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:22.904615  6204 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:22.906396  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:22.886826 11994 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:22.916473  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.882123 12372 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:27:22.893311 12400 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.894147 12144 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:22.935187  6204 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 13, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 13 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:22.904670 11996 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:27:22.936362  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.892222 12361 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:22.883697 12211 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.886276 12378 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:27:22.880306 12242 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:27:22.907660 12160 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:22.949777 12383 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.957713 12211 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.960178  6204 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 15. New cstate: current_term: 15 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:22.939198 12173 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:22.943682 12400 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:22.963920  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.949167 12318 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:22.905459 12168 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:22.954097 12392 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14.
I20260709 12:27:22.958972 12117 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:22.895823 12341 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.997766  6204 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:27:23.001771  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 157993us
I20260709 12:27:23.005757  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:23.000679  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 143952us
I20260709 12:27:23.009802  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:22.979290 12324 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.981657 12173 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.003734  6204 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 12, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:22.981657 12361 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:22.946105 12344 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:22.998070 12291 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.023005  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:23.020494  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 114197us
I20260709 12:27:23.012964 12293 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:23.025151  6204 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 13, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 13 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:27:23.021943  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118281us
I20260709 12:27:23.001559 12318 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:23.003571 12093 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.029953  6450 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:23.019485 12412 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:23.037132  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.038395 12382 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:23.022603 12414 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:23.040066 12443 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:27:23.039927 12318 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.044098  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.048223  7316 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:23.031921  6204 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 12, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:23.021329  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:23.050769  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:23.020546 12324 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.035180  7318 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:23.034849 12093 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.032681 12444 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:27:23.026289 12117 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:23.030095 12291 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:23.049381  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:23.032337 12344 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:23.064155  6204 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 14 to 15, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 15 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:27:23.044353 12438 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:27:23.022842 12361 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.069727 12361 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.057736  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.062981  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:23.030807  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 180763us
I20260709 12:27:23.072155  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:23.072660  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:23.069959  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51892.9us
I20260709 12:27:23.073786  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.062371 12173 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.075332 12173 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.075678  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:23.077203 12093 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:23.077263  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:23.078684  6200 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 298) took 35107 ms (client timeout 15000 ms). Trace:
I20260709 12:27:23.081305  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:23.081403  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.054162 12378 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:23.055781 12412 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:23.091744 12370 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:23.091907 12412 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.092033  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:23.071895 12446 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:23.100176 12446 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:23.101198 12446 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.082670 12211 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.052453 12441 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:27:23.086021 12293 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:23.086463 12449 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.085263 12450 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:27:23.089411  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:23.090294 12361 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:23.090574 12428 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.074193 12117 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.094213 12400 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.091763 12377 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.113135 12361 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:23.115653 12361 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.116353 12428 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:23.109786 12211 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:23.113618 12093 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.004s
I20260709 12:27:23.118332 12400 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.118983 12377 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:23.119824  6454 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:23.079924 12291 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:23.097515  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:23.092324 12412 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:23.049501 12242 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:23.078797 12160 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.111938  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:23.079241  6200 rpcz_store.cc:269] 0709 12:26:47.937319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.985660 (+ 48341us) service_pool.cc:224] Handling call
0709 12:27:23.024655 (+35038995us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:12.165644 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:12.208973 (+ 43329us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:12.209608 (+   635us) write_op.cc:435] Acquired schema lock
0709 12:27:12.222608 (+ 13000us) tablet.cc:662] Decoding operations
0709 12:27:12.242840 (+ 20232us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:12.242920 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:12.242946 (+    26us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:27:12.246815 (+  3869us) tablet.cc:701] Row locks acquired
0709 12:27:12.246835 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:27:12.246990 (+   155us) write_op.cc:270] Start()
0709 12:27:12.247121 (+   131us) write_op.cc:276] Timestamp: P: 1783600032246939 usec, L: 0
0709 12:27:12.247144 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:27:12.247839 (+   695us) log.cc:844] Serialized 3909 byte log entry
0709 12:27:12.769615 (+521776us) op_driver.cc:464] REPLICATION: finished
0709 12:27:13.200468 (+430853us) write_op.cc:301] APPLY: starting
0709 12:27:13.363886 (+163418us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:13.618945 (+255059us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:13.619070 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:14.019135 (+400065us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:14.019551 (+   416us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:27:14.019607 (+    56us) write_op.cc:312] APPLY: finished
0709 12:27:14.020222 (+   615us) log.cc:844] Serialized 97 byte log entry
0709 12:27:14.593247 (+573025us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:14.662814 (+ 69567us) write_op.cc:454] Released schema lock
0709 12:27:14.662917 (+   103us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:14.663674 (+   757us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:14.663721 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7343,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":401089,"apply.run_cpu_time_us":64633,"apply.run_wall_time_us":1479831,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":307018,"prepare.run_cpu_time_us":22135,"prepare.run_wall_time_us":177141,"raft.queue_time_us":244237,"raft.run_cpu_time_us":13539,"raft.run_wall_time_us":169369,"replication_time_us":522307,"thread_start_us":1098283,"threads_started":5,"wal-append.queue_time_us":808429,"wal-append.run_cpu_time_us":33072,"wal-append.run_wall_time_us":2411610}]]}
I20260709 12:27:23.129833  6204 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 10, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 10 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:23.139967 12291 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.142781 12168 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.107890 12445 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:27:23.148902  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51480us
I20260709 12:27:23.148921  6272 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:23.154052  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.154340  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:23.116257 12117 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.126219 12211 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.162603  7335 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:23.170979  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:23.164426  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.164078  6204 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:23.172842  6204 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 12, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 12 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:23.161322  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:23.161382 12331 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:23.159101 11927 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.159951 12203 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:23.182804  6204 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:23.166630  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:23.177404  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:23.166235 12451 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:27:23.149385 12242 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.196800  6204 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:27:23.192739  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 93619us
I20260709 12:27:23.181095 12400 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:23.179741 12383 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.177122 12445 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:23.211421  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44918.7us
I20260709 12:27:23.189796 12147 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:23.215399  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:23.219715  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.192719 12331 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.201210 12400 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.196655 12446 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:23.222622  6204 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 11, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 11 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:23.227988  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:23.228065  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50770.9us
I20260709 12:27:23.225509 12360 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:23.231685 12360 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.232077 12360 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:23.217509 12457 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:23.225586 12324 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:23.239145 12447 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:27:23.243326 12445 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.246685 12289 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.217760 11994 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:23.248088 11994 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.249125 11994 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.240998 12446 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.251566  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.251902 12242 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.252199  6309 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:23.252542 11996 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.253228 12446 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.241815 12457 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.255231  6302 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:23.219766 12093 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:23.247048 12331 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.256857  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.241950  6204 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 2 to 15, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:23.228706 12400 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:23.260622  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.261314  6308 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:23.229416 12375 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.255088 12160 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.262813  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.268604  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.269145 12375 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.262871  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:23.264228  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.266407  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:23.240895 12211 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.267648  6305 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:23.267771  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:23.270416  6310 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:23.241401 12361 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.271828  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.258857 12093 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.277333  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.278267  6305 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:23.278203  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.280721  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:23.278402  6204 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 17, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 17 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:23.282256  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.282647  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.283592  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.284250  6381 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:23.285394  6305 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:23.287698  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:23.289529  6376 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:23.290311  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.290866  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.290691  6204 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:23.263386 12462 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:23.297667 12462 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.290311  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:23.293613  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.294651  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.296182 12376 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:23.255402 12457 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.256407 11927 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.293598  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:23.291958  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.318161  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.315228  6451 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:23.316136  6204 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 13, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 13 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:23.308231  6302 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:23.306766  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:23.318027  6385 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:23.322045  6381 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:23.323704 12372 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:23.324966  6310 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:23.326272  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:23.331243  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.331468  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.338897  6307 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:27:23.343283  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.286010 12093 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:23.347419  6300 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:23.347746  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:23.354924  6310 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:23.343888  7316 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:27:23.286917 11996 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:23.365360  6204 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 19, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:23.318538 12234 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.368850  6204 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 10, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 10 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:23.343093 12172 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.324163 12372 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.347608 12361 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:27:23.317899 12376 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:23.381040  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.384327  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:23.389987  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.363623 12160 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.402475  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.403040  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:23.368588 12451 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:23.369170 12370 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:23.405817  6300 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.410982  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.346501 12366 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:27:23.346222 12324 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:27:23.418146  6456 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:23.418866  6456 pending_rounds.cc:85] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 8
I20260709 12:27:23.419251  6456 pending_rounds.cc:107] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:27:23.419628  6456 raft_consensus.cc:2889] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.411301 12160 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.428738 12160 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.414537 12451 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.433822 12451 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.384172 12234 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.441521 12234 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:23.443555 12291 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:23.443679  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.445271 12451 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:23.445772 12376 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.446903 12376 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:23.447566 12290 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.887s	user 0.015s	sys 0.002s
I20260709 12:27:23.391353 12172 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.449437  6456 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:23.450552 12172 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:23.452500  6456 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:27:23.452683 12280 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:23.452929  6456 pending_rounds.cc:85] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 7
I20260709 12:27:23.453263  6456 pending_rounds.cc:107] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:27:23.453523  6456 raft_consensus.cc:2889] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.445879 12451 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:23.455965 12354 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.695s	user 0.011s	sys 0.000s
I20260709 12:27:23.456422 12451 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:23.456732  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:23.455812 12173 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.450592 12372 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.458868  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.459556 12372 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:23.409046 12446 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.391711 12469 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:27:23.415802 11996 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:23.474651  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.475229  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:23.471263  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.476406  6456 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:23.476603  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:23.476830  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:23.480645  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.481884  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.482731  6450 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:23.484620  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:23.485031  6300 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:23.483047  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.487112  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:23.491971  6456 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:23.492108  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:23.495432  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.498080  6308 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 9. Current term is 10. Ops: []
I20260709 12:27:23.492937  6300 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:23.500425  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.422348 12370 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.491333 12210 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:23.517093  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 344112us
I20260709 12:27:23.493402 12366 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.520216  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.497795 12205 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.527351  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.525261  6308 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:27:23.530274  6456 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:23.504027 12343 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.533619  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.507336 12451 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:23.532232 12205 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:23.536293 12471 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:27:23.536006 12160 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.534547 12451 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:23.537997 12205 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:23.538714 12451 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.540082 12160 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:23.540539 12360 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.541509  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:23.541687  6303 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:23.480537 12400 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.511034 12428 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:23.493283 12324 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:27:23.544742  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68925.6us
I20260709 12:27:23.550588  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:23.518350 12173 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.519093 12210 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.516258 12375 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:23.560948  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 199953us
I20260709 12:27:23.540082 12329 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:23.559844  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:23.549085 12472 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:27:23.565250  6304 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:23.577837  6456 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:23.579453  6454 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:23.581445  6455 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:23.592314  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.516083 12412 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.549346 12457 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:23.584223  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:23.674510  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90375.4us
I20260709 12:27:23.675639  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.583598 12329 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:23.630261  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.630909  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.680648  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:23.582054  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:23.580755  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:23.633821  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:23.531594 12366 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.565721 12210 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.665359  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.564816 12280 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.564625 11996 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.556463 12242 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:23.596619 12354 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:23.554437 12428 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:23.627208 12429 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:23.713567  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.743142  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.580186 12293 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.117s
I20260709 12:27:23.563412 12291 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
W20260709 12:27:23.604111  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.750855  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.789683  6383 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 17 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:27:23.768253  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 176009us
I20260709 12:27:23.627774 12383 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
W20260709 12:27:23.745074 12474 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:27:23.679265 12445 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:23.673630 12360 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.738637 12446 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:27:23.794914 12344 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.001s
I20260709 12:27:23.577664 12173 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.782326 12354 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.846355  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:23.780980 12428 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:23.850639  6383 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:23.864346  6454 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
W20260709 12:27:23.802752  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 222123us
I20260709 12:27:23.872452  6308 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:23.874330  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:23.918917 12478 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:27:23.922792 12462 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.806671 12445 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:23.923812 12445 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:23.927492  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.928478 12400 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:23.848762 12242 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.935070 12242 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.937759 12242 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:23.840740 12457 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:23.950929 12370 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:23.848097 12472 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:23.963032 12236 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:23.750082 12093 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.752157 12331 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.111s
I20260709 12:27:23.978442  6381 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 18 index: 16. (index mismatch)
I20260709 12:27:23.778594 12280 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:23.778466 12481 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.209s
I20260709 12:27:23.788688 11996 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:23.988337  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:24.003891  7332 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:24.004001  6383 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:27:23.993636  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:23.997675  7329 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:27:23.966148 12372 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.002s
I20260709 12:27:23.982055  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:23.960191 12446 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:27:23.964407 12474 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:27:24.006017  6381 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:23.966826 12354 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.011439  6380 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:24.017128  6301 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:23.987171 12324 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.021086 12480 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:24.025528  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:23.980319 12423 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:24.040833  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:24.091045  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50307.4us
I20260709 12:27:23.989969 12429 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.054499  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:24.057607  6308 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:27:24.058543  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 387312us
W20260709 12:27:24.068598  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.011570 12303 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:24.076426  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 395850us
W20260709 12:27:24.078388  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3253) took 1390 ms (client timeout 1707 ms). Trace:
W20260709 12:27:24.054044 12472 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:24.093034  6381 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:24.005681 12344 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 12:27:24.113511  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.115279  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.116748  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:24.117486  6456 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:24.099578  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:24.110846 12303 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:24.165272  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.013728 12343 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:27:24.057971 12446 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:24.098853  6458 rpcz_store.cc:269] 0709 12:27:22.688094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.690144 (+  2050us) service_pool.cc:224] Handling call
0709 12:27:24.078145 (+1388001us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b540028cdd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:24.078305 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78910208}
I20260709 12:27:24.119452  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:24.172797  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53422.5us
I20260709 12:27:24.170218  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:24.057035 12372 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:27:24.174744  6302 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:24.181311  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:24.093918 12480 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.097462 12242 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:24.179173  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:24.197582 12480 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:24.169605 12383 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:24.179831 12370 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:24.095824 12324 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.218765 12377 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:24.230914  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1107) took 1165 ms. Trace:
W20260709 12:27:24.232930  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3230) took 1829 ms (client timeout 1751 ms). Trace:
W20260709 12:27:24.251940  6451 rpcz_store.cc:269] 0709 12:27:22.403893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.406396 (+  2503us) service_pool.cc:224] Handling call
0709 12:27:24.232684 (+1826288us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:24.232835 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049357056}
I20260709 12:27:24.239359  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 955) took 1758 ms. Trace:
W20260709 12:27:24.240322 12482 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.388s	user 0.001s	sys 0.000s
I20260709 12:27:24.098265 12423 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.187340 12412 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.189114  6456 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:24.167367 12203 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:27:24.113202 12429 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.253317  6457 rpcz_store.cc:276] 0709 12:27:22.481211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.482592 (+  1381us) service_pool.cc:224] Handling call
0709 12:27:22.482900 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:22.519638 (+ 36738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.519674 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:22.520287 (+   613us) log.cc:844] Serialized 52 byte log entry
0709 12:27:22.758553 (+238266us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:22.758608 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.238835 (+1480227us) spinlock_profiling.cc:243] Waited 36.3 ms on lock 0x7b54000fc654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:24.238909 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.238984 (+    75us) raft_consensus.cc:1685] finished
0709 12:27:24.239004 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.239273 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1302,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":83528064,"thread_start_us":233325,"threads_started":1,"wal-append.queue_time_us":346193,"wal-append.run_cpu_time_us":7527,"wal-append.run_wall_time_us":1099643}
I20260709 12:27:24.249552  7318 rpcz_store.cc:276] 0709 12:27:23.065894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.066090 (+   196us) service_pool.cc:224] Handling call
0709 12:27:23.066462 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.077924 (+ 11462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.077960 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:23.080306 (+  2346us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.150409 (+ 70103us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:23.150457 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.230232 (+1079775us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400438ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:24.230306 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.230382 (+    76us) raft_consensus.cc:1685] finished
0709 12:27:24.230404 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.230789 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36093,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3162,"spinlock_wait_cycles":25462016,"thread_start_us":38755,"threads_started":1,"wal-append.queue_time_us":74373,"wal-append.run_cpu_time_us":1956,"wal-append.run_wall_time_us":1053491}
I20260709 12:27:24.207547 12366 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:24.283776  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 292915us
I20260709 12:27:24.289204  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:24.273756 12291 consensus_queue.cc:1243] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:27:24.268404 12468 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.018s	sys 0.047s
I20260709 12:27:24.267889 12331 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:24.310338 12280 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.267784 12377 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.324070 12481 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 12:27:24.365415 12491 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:27:24.305393 12446 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:24.368233 12489 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:27:24.379554 12480 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.381661 12480 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.390588 12514 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:24.391760 12514 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:24.392668 12514 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.390461 12516 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:27:24.433784  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.005s	sys 0.013s
W20260709 12:27:24.435235  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.005s	sys 0.014s
W20260709 12:27:24.439087  6307 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.005s	sys 0.003s
W20260709 12:27:24.439742  6307 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.005s	sys 0.003s
I20260709 12:27:24.442374  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2754) took 1102 ms. Trace:
I20260709 12:27:24.443099  6307 rpcz_store.cc:276] 0709 12:27:23.340357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.341388 (+  1031us) service_pool.cc:224] Handling call
0709 12:27:23.341772 (+   384us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:23.681778 (+340006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:23.681828 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:23.682574 (+   746us) log.cc:844] Serialized 52 byte log entry
0709 12:27:24.440660 (+758086us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:24.441323 (+   663us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.441728 (+   405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.441895 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.442002 (+   107us) raft_consensus.cc:1685] finished
0709 12:27:24.442018 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.442289 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2041,"raft.queue_time_us":33114,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":9466,"thread_start_us":939976,"threads_started":2,"wal-append.queue_time_us":585748}
W20260709 12:27:24.447710  6310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.010s	sys 0.028s
W20260709 12:27:24.448199  6310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.010s	sys 0.028s
I20260709 12:27:24.449671  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2540) took 1096 ms. Trace:
I20260709 12:27:24.450073  6310 rpcz_store.cc:276] 0709 12:27:23.353274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.355700 (+  2426us) service_pool.cc:224] Handling call
0709 12:27:23.356040 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.356276 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:23.785903 (+429627us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.448981 (+663078us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.449094 (+   113us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:24.449226 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.449333 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.449591 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45613,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2750,"spinlock_wait_cycles":1041152,"thread_start_us":1080925,"threads_started":2,"wal-append.queue_time_us":568178}
W20260709 12:27:24.463047  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 370588us
W20260709 12:27:24.465327  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 286845us
I20260709 12:27:24.475262 12303 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.480216 12093 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.480744 12324 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.483522 11996 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.486897 12361 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:24.488260 12343 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.488562 12361 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.499150  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2748) took 1168 ms. Trace:
I20260709 12:27:24.499574 12293 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.341796 12331 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.501093 12377 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.489491 12343 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.520682 12412 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.505887  6304 rpcz_store.cc:276] 0709 12:27:23.330286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.331803 (+  1517us) service_pool.cc:224] Handling call
0709 12:27:23.332231 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.630748 (+298517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.498535 (+867787us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b5400093dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:24.498944 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535451904}
I20260709 12:27:24.521631 12412 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.527240 12412 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.588454 12331 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:24.590452 12343 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.591300 12293 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.592281  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:24.598126  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71900.6us
I20260709 12:27:24.599607  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.600965 11927 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:24.607421 12527 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.611953 12527 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.612790 12343 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:24.613448 12343 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:24.614128 12343 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:24.616775 12376 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.169s	user 0.014s	sys 0.002s
I20260709 12:27:24.618166  6453 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:24.608518  6452 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 12:27:24.620191  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.628400 12173 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.631167  6309 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:24.589031  7317 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.023s
W20260709 12:27:24.635730  7317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.010s	sys 0.023s
I20260709 12:27:24.625977  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:24.636986  6455 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:24.610140 12423 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.624280  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:24.604187  6379 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.016s
I20260709 12:27:24.639149 12293 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.649669  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:24.654083  6272 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.017s
I20260709 12:27:24.659554  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:24.666093  6272 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.012s	sys 0.017s
W20260709 12:27:24.667155  6379 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.010s	sys 0.018s
I20260709 12:27:24.676376 12429 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.683271  6456 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:24.686339 12451 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:24.689107 12428 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.760s	user 0.009s	sys 0.006s
I20260709 12:27:24.689823 12428 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.694638 12450 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.698979  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:24.696836 12450 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.701402 12527 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.701903 12450 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.703568  7332 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:24.705224 12428 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.706017 12468 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:24.711251  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:24.711690 12468 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:24.714010 12480 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:24.713793 12370 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:24.714632 12480 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:24.715289 12413 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:24.715669 12493 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.433s	user 0.000s	sys 0.002s
I20260709 12:27:24.716234 12480 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.716543 12372 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:24.717679 12372 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:24.717971 12480 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:24.723311  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.727097 12423 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:24.727804  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:24.728035 12423 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.729228  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 922) took 1291 ms. Trace:
I20260709 12:27:24.729754  6305 rpcz_store.cc:276] 0709 12:27:23.438061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.727261 (+1289200us) service_pool.cc:224] Handling call
0709 12:27:24.729133 (+  1872us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:24.718534  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 190862us
I20260709 12:27:24.719017 12383 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.731027 12383 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.734051 12383 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.721005 12370 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:24.735946 12370 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:24.733953  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:24.736704 12383 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.721679 12450 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:24.738085 12450 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.737630 12383 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.741542 12423 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.720760  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:24.737612 12370 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.742825  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 881) took 1418 ms. Trace:
I20260709 12:27:24.743252  7335 rpcz_store.cc:276] 0709 12:27:23.324579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.348007 (+ 23428us) service_pool.cc:224] Handling call
0709 12:27:24.720258 (+1372251us) spinlock_profiling.cc:243] Waited 5.97 ms on lock 0x7b5800004898. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:27:24.742717 (+ 22459us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13721472}
I20260709 12:27:24.743599 12370 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:24.744793 12475 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.113s	user 0.007s	sys 0.007s
I20260709 12:27:24.745323 12383 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:24.735208  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.737200  6451 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:27:24.709751 12210 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.930s	user 0.004s	sys 0.023s
I20260709 12:27:24.731637  6456 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:24.714701  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.719538 12372 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:24.753619  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 634781us
I20260709 12:27:24.736014 12477 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:24.747390 12450 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.759251 12477 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:24.770217 12446 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:24.757454 12210 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.791204  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.779663 12428 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.807886 12329 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.809859  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 889) took 1239 ms. Trace:
I20260709 12:27:24.829830  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3260) took 1369 ms. Trace:
I20260709 12:27:24.831635 12173 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:24.836827 12210 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.837890  6451 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:24.838799 12372 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:24.830538  6453 rpcz_store.cc:276] 0709 12:27:23.458112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.458356 (+   244us) service_pool.cc:224] Handling call
0709 12:27:24.808956 (+1350600us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:24.827476 (+ 18520us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2663491328}
I20260709 12:27:24.823169 12376 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.823987  6379 rpcz_store.cc:276] 0709 12:27:23.570754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.570936 (+   182us) service_pool.cc:224] Handling call
0709 12:27:23.571291 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.571535 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:24.065196 (+493661us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.775723 (+710527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.793602 (+ 17879us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:24.809343 (+ 15741us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.809454 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.809725 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61258,"prepare.queue_time_us":171105,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":8819,"raft.queue_time_us":5777,"spinlock_wait_cycles":138036864,"thread_start_us":1027512,"threads_started":2,"wal-append.queue_time_us":630491}
I20260709 12:27:24.830224 12428 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:24.863226 12354 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.017s
I20260709 12:27:24.863845 12354 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.874914 12343 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.875543 12354 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.865447 12428 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:24.875820  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53988.6us
I20260709 12:27:24.877518  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.878726 12413 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.879410  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:24.881511 12318 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:24.883006 12276 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:27:24.884513  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 123464us
I20260709 12:27:24.885536 12318 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:24.886966 12444 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:24.889467 12331 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.884626 12276 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:24.877696 12343 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:24.891186  6452 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: [17.10-19.11]
I20260709 12:27:24.908849  6379 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:24.910771  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 175744us
I20260709 12:27:24.912400 12280 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:24.916092  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:24.916808 12366 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:24.917385 12366 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:24.898612 12276 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:24.917933  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:24.918831  6453 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:24.919159 12376 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.904456 12173 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:24.921113 12173 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:24.922077 12173 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.923801 12173 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:24.923422 12477 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:24.924690 12477 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.925266 12506 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:24.925937 12506 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:24.926142  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:24.926865  6455 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:24.927856  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2760) took 1057 ms. Trace:
I20260709 12:27:24.928380  6308 rpcz_store.cc:276] 0709 12:27:23.870691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.927222 (+ 56531us) service_pool.cc:224] Handling call
0709 12:27:24.927225 (+1000003us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54002d14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:24.927563 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297081216}
I20260709 12:27:24.917451 12331 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:24.917448 12382 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:24.937685 12382 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:24.938370  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:24.939225  6377 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:24.939460  6378 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:24.942305  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:24.943223  6453 raft_consensus.cc:2376] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19. Re-sending same reply.
I20260709 12:27:24.925293 12276 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:24.944307 12445 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.020s	user 0.008s	sys 0.014s
I20260709 12:27:24.944859 12445 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.946190  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:24.944875 12276 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:24.954663 12523 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.247s	user 0.000s	sys 0.002s
I20260709 12:27:24.937472  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:24.955404 12376 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:24.958191  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56564.5us
I20260709 12:27:24.944350  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:24.965538  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:24.966005  6455 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:27:24.966638  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:24.972070  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71417.6us
I20260709 12:27:24.917323 12280 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:24.969767  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:24.962656 12147 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:24.987746  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:24.993152  6455 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:24.997517 12341 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:24.998019  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:24.998900  6453 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.003085 12382 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.004750 12382 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.005475 12117 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:27:25.005717  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:25.006018 12117 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:25.007432 12117 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.009227 12117 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:25.011023  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.010840 12147 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.012323  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.012609  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2849) took 1410 ms. Trace:
I20260709 12:27:25.013119  6454 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:25.013442 12147 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.014062  6376 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:25.015043  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:25.015651 12234 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:25.023403  6377 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:25.015828  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100258us
I20260709 12:27:25.016916  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2525) took 1763 ms. Trace:
I20260709 12:27:24.987366 12446 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.025017  6309 rpcz_store.cc:276] 0709 12:27:23.253174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.253612 (+   438us) service_pool.cc:224] Handling call
0709 12:27:23.253905 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:24.637400 (+1383495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.016608 (+379208us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54002067d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:25.016851 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3165904640}
I20260709 12:27:24.961359 12366 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:25.032248  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54455.6us
W20260709 12:27:25.032752  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.034875  6378 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:25.036186 12446 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.022548 12445 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.037945  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.013077  6450 rpcz_store.cc:276] 0709 12:27:23.575288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.578585 (+  3297us) service_pool.cc:224] Handling call
0709 12:27:23.771842 (+193257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.772120 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:24.298570 (+526450us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.857167 (+558597us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.904548 (+ 47381us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:24.933327 (+ 28779us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:24.954070 (+ 20743us) spinlock_profiling.cc:243] Waited 67.2 ms on lock 0x7b24001ea928. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:27:24.962407 (+  8337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.965376 (+  2969us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:24.985159 (+ 19783us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20603,"prepare.queue_time_us":601373,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":47556,"raft.run_cpu_time_us":528,"raft.run_wall_time_us":3148,"spinlock_wait_cycles":154645888,"thread_start_us":957009,"threads_started":3,"wal-append.queue_time_us":407958}
I20260709 12:27:25.041706  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:25.046521  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.048321  6379 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:25.024420 12234 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:25.049446 12476 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.050253 12476 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.050765 12366 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:25.051741  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.052896 12476 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.050076 12234 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:24.984081 12506 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.055382  6455 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:25.061164  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.062589  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.067124  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.069692 12160 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.070582 12160 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.070955 12309 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:27:25.073488 12160 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.045616 12361 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.091003  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.091415  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:25.091575  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.098001 12506 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:25.099977  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.100927  6303 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:27:25.075281  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.093015  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.105089  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:25.074695 12537 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:27:25.076542  6450 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
W20260709 12:27:25.108541  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.093750  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.081238 12234 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:25.115064  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.117250  6454 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:25.104974  6380 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:25.074441 12309 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:25.103406 12160 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.129504 12160 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.130699 12412 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.131559 12412 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.133190 12361 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.094062 12446 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.115653  6378 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:25.094610 12280 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:25.136168  6379 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:25.127360  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:25.127544  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52410.1us
I20260709 12:27:25.128038 12423 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:25.105461 12451 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.147567  6310 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:25.150323 12211 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.150770 12451 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.151162 12211 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.151196  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:25.152338  6304 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.153231 12451 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.154280  6307 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:25.154700 12291 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.155185 12518 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.155676 12291 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.155850 12518 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.160578  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:25.161374 12331 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.161814 12481 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.153340 12211 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.141382  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.169227  6303 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:25.164170  6384 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:25.169883  6308 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:25.175824  6309 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:25.135622 12173 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:25.139757 12423 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.157497 12413 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.140187 12160 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.182615  6453 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:25.138852 12293 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.175392  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 216489us
I20260709 12:27:25.162338 12331 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.163730 12375 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:25.166023 12542 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:27:25.190795  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.601s (SENT) [suppressed 7 similar messages]
I20260709 12:27:25.200213  6450 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:25.200832  6455 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
W20260709 12:27:25.202567  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.202566  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.183200 12518 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.189356 12423 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.200861  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:25.206274 12293 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.207844 12331 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.202616 12173 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:25.211066  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50646.9us
I20260709 12:27:25.212015 12172 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.188159 12445 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:27:25.163910 12361 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:25.221354  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 215913us
I20260709 12:27:25.227007  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.228421  6385 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:25.228739  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:25.229360  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.231487  6455 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:25.167853 12309 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.233714  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.236080  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.239373  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:25.199380 12211 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.241396  6453 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:25.243417  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.246426  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.233472 12172 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.214629 12375 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:25.249420  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.252805  7318 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:25.247360  6450 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:27:25.256299  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.238587 12542 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.203389 12481 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.261943  6303 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:25.262132  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:25.262750  6457 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:25.269984 12173 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.270458  6308 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:25.272027 12542 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.272941  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.273332  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.275048 12293 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.277604  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:25.279122  6382 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:25.279765  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.280247 12291 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.234205 12445 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:25.293541  6382 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:25.299501  6384 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:25.299446  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.302089  6304 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:25.301165  6379 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:25.305245  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.304584  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.286111 12375 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.310037  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:25.310053  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71829.1us
I20260709 12:27:25.313273  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:25.315726  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.316022  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.317924 12547 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:25.318627 11993 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.319275 12173 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:25.321363  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:25.322286 12329 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:25.322958  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68312.7us
W20260709 12:27:25.324260  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.331050  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.324436  6310 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 12:27:25.331847  6310 pending_rounds.cc:85] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 12
I20260709 12:27:25.332105  6310 pending_rounds.cc:107] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 12:27:25.332381  6310 raft_consensus.cc:2889] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.325446  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.333600  6300 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:25.325834 12542 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.335062  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:25.336370 12515 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:25.336907  6385 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:25.336969 12515 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.337703 12515 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.339785 12361 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.326359 11993 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:25.341823 11993 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.342330 11993 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.345153  6300 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:25.346886  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:25.348831  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.349558  6308 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:25.350091  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.350620  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.350979  6457 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:27:25.351635  6310 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:27:25.327327  6382 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.352700  6382 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.359920  6455 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:25.364012  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.365655  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:25.369203  7317 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:25.369445  6452 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:25.371618  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:25.361828 12527 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:25.374571  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:25.373935  6304 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:25.373869  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 183113us
I20260709 12:27:25.360549 12481 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.378265  7318 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:25.379864  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:25.389925  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.391708  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:25.392640  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.393987  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:25.396507  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75287.1us
I20260709 12:27:25.397893  6310 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:25.393512 12361 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.404366  6308 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:25.405174  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:25.402746 12542 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:27:25.410597 12341 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:25.396251 12276 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:27:25.412115 12276 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:25.412504 12276 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:25.413475 12276 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.414479  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.415258  7314 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:25.387941 12527 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:25.417874 12527 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.421145  6458 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:25.402657 12361 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:25.430859  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.377408 12211 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:27:25.442553 12414 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:25.406936 12375 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.433764  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:25.398655 12506 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:25.439827  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:25.450472  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 62751.1us
I20260709 12:27:25.451488  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.453323  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.455245  6303 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:25.452984 11996 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:25.457650  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148733us
W20260709 12:27:25.458304  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53211.8us
I20260709 12:27:25.459619  6458 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:25.466588  7335 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:25.468544  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.470484  6304 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:25.473011  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.474819 12203 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:25.460718 12309 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.475199  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.458702 11996 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:25.478410  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.474291  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.479231  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.482731  6385 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:25.466153 12276 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.452489 12423 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.479938 12457 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:25.494055  6380 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:25.483534  6379 raft_consensus.cc:2410] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:27:25.466288 12481 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:25.487404  6381 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:27:25.489019  6456 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:27:25.489856 12293 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.497540  6385 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:25.454725 12375 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.496002  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.499703 12293 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.498301  6458 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:25.471166 12117 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 12:27:25.483276 12526 raft_consensus.cc:2764] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:25.483765 11996 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:25.466635 12506 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.496341  6456 pending_rounds.cc:85] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 8
I20260709 12:27:25.511442  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.498764 12481 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.514211  6382 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:27:25.514636  6456 pending_rounds.cc:107] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:27:25.514912  6382 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 7
I20260709 12:27:25.515076  6377 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:25.515316  6382 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:27:25.515722  6382 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.514972  6456 raft_consensus.cc:2889] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.495772 12457 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.517181  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.519150 12526 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:25.519320  7335 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:27:25.518452  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92650us
I20260709 12:27:25.519863  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:25.521407  7318 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:25.522213  6377 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:25.521636 12457 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.498893 11993 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.512478 12361 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.524787  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2837) took 2724 ms. Trace:
I20260709 12:27:25.521397  6450 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:25.517572 12481 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.534742  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:25.538017  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.521832 12526 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:25.541283 12552 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:27:25.542711  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.525260  6459 rpcz_store.cc:276] 0709 12:27:22.800020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.831359 (+ 31339us) service_pool.cc:224] Handling call
0709 12:27:24.078610 (+1247251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.483252 (+404642us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.483314 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.484532 (+  1218us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.643128 (+158596us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:24.643196 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.664127 (+ 20931us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:24.871848 (+207721us) raft_consensus.cc:1685] finished
0709 12:27:24.871873 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.524379 (+652506us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540028cdd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:25.524574 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":257421,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":8520,"spinlock_wait_cycles":2867555072,"thread_start_us":553859,"threads_started":2,"wal-append.queue_time_us":282071}
I20260709 12:27:25.545583  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.551004 12513 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:25.551489 12513 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:25.551854 12513 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:25.553217 12172 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.541923 12526 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.556360 12293 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.557785  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.558612  6307 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:25.561416 12412 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.549998  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.544991 12361 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:25.570115  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.573998 12423 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.587872 12234 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:25.579785 12375 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.592480  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:25.588903 12339 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:25.575941 12506 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.592722 12234 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.575027  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.594228 12234 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.593845 12339 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.597868 12339 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:25.586719 12551 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
W20260709 12:27:25.598969  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49064.7us
I20260709 12:27:25.574576 12361 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:25.599792  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.603338 12339 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:25.603761 12339 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.604149 12339 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:25.605906  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.607998  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.609153 12513 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.609772  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.610376 12517 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:27:25.611382 12518 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:25.611905 12518 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:25.611676 12515 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.612907 12518 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.613902 12515 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:25.614813  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 119292us
I20260709 12:27:25.616995  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.617280  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.617667  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.618029  6381 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:25.618345  6379 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:27:25.619132  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 88282us
I20260709 12:27:25.620215  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:25.621639 12445 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:25.621713  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.622368 12445 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.622843 12445 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:25.618975  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.623502  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.624147  7332 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:25.625456 12412 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.626013 12412 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.628261 12412 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.631418  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.633556  7332 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:25.632928  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.634142 12361 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.634980 12234 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.635690  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:25.635984 12555 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.101s	user 0.002s	sys 0.000s
I20260709 12:27:25.633351  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.637490 12457 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.640137 12513 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.621636  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.641084 12160 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:25.641594 12517 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:25.642252 12515 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:25.644070 11993 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.645128  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.646500 12445 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.647230  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.711716  6379 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:25.648173  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.648865  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.648772 12331 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:25.649515 12211 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:25.649932 12375 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.716269 12203 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:25.716751 12203 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:25.654058 12526 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:25.654982 12554 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.177s	user 0.000s	sys 0.002s
I20260709 12:27:25.631239 11996 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.718010 12203 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.720182 12203 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.720335 11996 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.656148 12481 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.658026  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.661295  6378 pending_rounds.cc:85] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 9
I20260709 12:27:25.680567  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.723698  6378 pending_rounds.cc:107] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:25.722813  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.663784  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.695175  7318 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
W20260709 12:27:25.705924  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 84357.7us
I20260709 12:27:25.726631  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:25.727675 12457 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.727905  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.728200 12144 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:25.728801 12293 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:25.728821 12144 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:25.729434 12293 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:25.730193 12144 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.724077  6378 raft_consensus.cc:2889] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.711053 12412 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:25.703890 12160 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.722321 12481 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.724385 12445 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:25.730435 12293 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.708812  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.709781  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.711663  6382 raft_consensus.cc:2368] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:25.716557 12375 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.722844  6377 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:25.690845  7317 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:25.725414  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.704684  7335 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:25.731859  6384 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:25.724025  6385 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
W20260709 12:27:25.732756  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83959.1us
W20260709 12:27:25.733089  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69447.8us
I20260709 12:27:25.738332  7316 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:25.741882  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.735816  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.744037  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.746891  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.706713 12517 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:25.731096  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.704248 12423 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:27:25.732851 12203 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:27:25.733345 12160 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:25.733291 12318 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:25.765540  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.731511 12412 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.730795 12234 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.770673  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.752007 12513 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:25.775543  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.776399  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.780365  6304 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:25.776778  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.754088 12444 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:25.755234 12506 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:25.789124 12444 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.789248 12506 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.789762 12444 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.789841 12506 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:25.771088 12318 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:25.791674 12318 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:25.793030  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.794054  7332 raft_consensus.cc:2410] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:27:25.771623 12412 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.783798 12517 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.787108  6305 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:27:25.769757 11996 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:25.761822 12203 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.798018  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.805682  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.814754  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.782634 12513 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:25.815186  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.815462  6303 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:25.815616 12513 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:25.814802  6302 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:25.815219  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.806567  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.826102  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:25.820055  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.830592  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:25.831390  6457 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:27:25.825506  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:25.842628  6452 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:27:25.837397  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:25.800276 11993 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:25.852948  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 219997us
I20260709 12:27:25.778440 12339 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.799038 12172 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.868530  6384 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:25.834749 12481 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:25.855875 12291 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:25.856323  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.873874  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.875137  6385 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:25.857004  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:25.857125  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:25.859093  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:25.808625 11996 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.822752 12203 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:27:25.810676 12429 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.811434 12234 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:25.867105  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:25.870401  6308 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:25.887648  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:25.888398  6457 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:25.835419 12361 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:25.853262 12382 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:25.908082 12382 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:25.908229 12339 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:25.884599 12172 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.909060 12382 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:25.910914 12172 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.874029  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:25.913645  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:25.853873 12526 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.868741 12515 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.884908  6458 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:25.862421 12414 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:25.879899 12457 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.855475  6304 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:25.886098 12366 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:27:25.894608 12429 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:25.943926  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:25.896071 12375 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:25.952261  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:25.896461  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.901505  6382 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:27:25.884356  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.869521 11993 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:25.920933  7316 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:27:25.956856  7316 pending_rounds.cc:85] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 11
I20260709 12:27:25.957124  7316 pending_rounds.cc:107] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:27:25.957446  7316 raft_consensus.cc:2889] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:25.922302  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.958009  6383 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.874814 12481 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:25.883908  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:25.961599  6384 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:25.880548 11996 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:25.880172 12517 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:25.928025 12414 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:25.887099 12234 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.912865 12361 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.916415 12518 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.944454 12457 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:25.954706 12557 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:25.964975 12375 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.946195  6385 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
W20260709 12:27:25.980803  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126833us
I20260709 12:27:25.982993  6450 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:25.997888  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:25.998566  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.966622 12481 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:25.951201 12339 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:27:25.995504  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:26.003142  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85910us
I20260709 12:27:25.933157 12515 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.009421  6308 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:25.982599 12414 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:25.988492 12234 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:25.968261 12445 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:27:25.987926 12361 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:26.002969 12526 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:26.046720  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 133569us
W20260709 12:27:26.050356  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47470.6us
I20260709 12:27:26.002897 12518 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.029408 12291 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:26.057510  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:26.018956 12375 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.050565 12144 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:26.038772 12331 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:27:26.029167 12527 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:27:26.076586  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65470.1us
W20260709 12:27:26.095389 12563 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:26.104574 12172 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:26.105151 12172 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:26.085454 12568 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:27:26.107033 12568 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:26.108075  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.108595 12568 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:26.108952  6303 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.111855 12518 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.113163 12515 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.077133 12144 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.115882 12144 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.116057 12375 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:26.071488 12567 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:27:26.120556 12506 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.052038 11993 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.125556 12513 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.127331 12457 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.044190 12526 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.087949  6458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:26.131367  6458 raft_consensus.cc:606] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:26.131137 12309 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:27:26.095615 12562 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:26.132352 12309 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:26.132091 12291 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.132756 12309 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:26.133699 12309 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.135691 12309 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:26.138155 12565 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:26.139410 12566 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.139861 12211 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:26.131780  6458 raft_consensus.cc:614] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:26.062659 12444 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.144143 12160 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.106149 12172 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.110230 12568 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:26.160676  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53869.5us
I20260709 12:27:26.161239  6383 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:26.155046  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:26.173774  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:26.181974  7318 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:26.183131  6384 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:26.177151  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 98909.6us
W20260709 12:27:26.186725  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 100242us
I20260709 12:27:26.190006  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.188792  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:26.191933  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:26.191301  6459 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
W20260709 12:27:26.199756  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.196702  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:26.205272  6377 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:26.215103  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.215971  6453 raft_consensus.cc:2368] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:26.175300 12211 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:26.222056  6450 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:27:26.169219 12309 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.230367  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.231292  6457 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:26.198843 12375 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:26.179607 12566 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.220593 12572 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.223243 12518 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:26.255460  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 168266us
I20260709 12:27:26.202728 12574 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:27:26.258396  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:26.271574  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.272902  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.247718 12172 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.273942  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:26.277518  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 220081us
I20260709 12:27:26.278532  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.274652  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:26.281147  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 407161us
I20260709 12:27:26.282424  6451 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:26.280452  6459 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:26.283313  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.283964  6450 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:26.233968 12309 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.255304 12518 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:26.291863  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.294340  6452 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:27:26.298791  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 74183.8us
I20260709 12:27:26.301144 12444 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.301707 12577 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.303011  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.247032 12414 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.309893  6459 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:26.310132  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51855.3us
I20260709 12:27:26.266288 12526 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:26.318141  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:26.323428  6384 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:26.323773  6377 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:26.319880  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.276461 12293 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:26.325135  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.254801 12375 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.333503  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:26.320039 11996 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.340030  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.297683 12518 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.355263  6454 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:26.355450  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:26.356652 12366 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:26.357905 12513 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.358575 12513 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:26.359154 12513 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.359508 12513 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:26.359714 12361 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.361564  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:26.363754 12445 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:26.364416 11996 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.293731 12527 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:26.365445  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:26.365583 12527 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.366086 12527 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:26.320955  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.370890  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.266906 12572 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.345304  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.375286  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:26.338927 12293 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.322098 12144 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.349035 12526 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:26.349385 12506 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.349697  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.349881  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260709 12:27:26.351310 12412 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.534s	user 0.006s	sys 0.005s
I20260709 12:27:26.351768  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.353050  6307 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:26.336532  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:26.333119  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:26.358619 12318 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.002s
I20260709 12:27:26.364055 11993 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:26.296445 12429 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.371696  6304 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:26.297214 12211 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.417449 12211 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.421324 12572 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.344286  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.414263 12445 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:26.422809  6309 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.344995  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.340965  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.372473  6305 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: [18.16-18.16]
I20260709 12:27:26.426539  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.411643 12581 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:26.427802  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.427951  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.429040  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.429171  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.371584  6379 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:26.430657 12160 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.362138 12361 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.345943  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:26.309358 12428 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:26.431919  6378 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:26.431802 12428 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.432890 12566 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.433840  6381 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:27:26.433992 12428 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.434229  6381 pending_rounds.cc:85] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:27:26.284988 12203 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.434469  6381 pending_rounds.cc:107] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:26.434755  6381 raft_consensus.cc:2889] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:26.434955  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:26.435596  6310 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.435936 12203 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:26.407765  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74747.2us
I20260709 12:27:26.437467  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:26.408656  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 72449.6us
I20260709 12:27:26.355319 12309 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:26.431402  6309 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:26.438808  6302 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:26.378904 12339 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:26.411162  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.412369  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.295370 12234 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.415153  6454 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:27:26.438409  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:26.427981 12581 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.460495 12581 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.460727 12580 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:26.461310 12580 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:26.430327  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.302268 12577 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.381075  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:26.441605  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:26.430217  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.464201  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:26.405704 12526 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.463243  6308 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:26.472069  6272 leader_election.cc:400] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
I20260709 12:27:26.472388  6272 leader_election.cc:403] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:26.466377  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:26.469254 11996 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:26.469365 12568 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:26.470428  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:26.470417  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.483260  6272 leader_election.cc:400] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:26.467319  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:26.446928 12566 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.494199  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.495713  6272 leader_election.cc:403] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:26.496687  6451 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:26.497539  6455 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:27:26.463264 12361 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.453845 12144 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.447014 12445 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:26.451881 12375 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:26.451419  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:26.501866  6308 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:26.505930  6424 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:26.524309  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:26.525487  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:26.515311  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.513149  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 256634us
I20260709 12:27:26.519490  6310 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:26.482975 12526 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.413971 12412 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.551098  6454 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:26.552030  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.552925  6455 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:26.495075 12568 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:26.536190  6459 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:26.539768  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.416047 12318 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.549764  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:26.494756 11996 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:26.494050 12291 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15.
I20260709 12:27:26.496690 12567 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:26.555629  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.535027  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:26.561627 12423 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:27:26.564759  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.568362  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:26.521063 12445 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:26.574852  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:26.499137 12414 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.576838  6451 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:26.575994  6452 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:26.579715  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:26.601420  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 416941us
I20260709 12:27:26.621050  6380 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:26.462276 12580 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.500762 12575 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:26.598292 12568 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.510639 12566 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:26.635571  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100648us
W20260709 12:27:26.615792  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 142536us
I20260709 12:27:26.550343 12506 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:26.641669  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:26.617234  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.550030 12339 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:26.555449 12366 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:27:26.649541  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69907.4us
W20260709 12:27:26.609869  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 244457us
I20260709 12:27:26.610970 12567 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:26.622148  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53909.1us
I20260709 12:27:26.623185  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:26.623696  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:26.607156  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.624665  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:26.605583 12481 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.000s
I20260709 12:27:26.520757 12309 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.093s
I20260709 12:27:26.575063  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:26.521461 12361 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.613584 11996 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.612340 12423 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:26.621955 12513 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.621560 12445 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:26.654579  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:26.632905 12428 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:26.681928  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:26.679962  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:26.687309  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:26.657656  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:26.667279  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.646967 12506 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:26.659022 12517 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:26.676638  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52687.2us
I20260709 12:27:26.647964  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:26.659806 12567 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.641227 12585 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:27:26.675607 12527 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.684075 12579 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:26.660849 12366 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:26.630669 12575 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.659634 12339 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:26.692230  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:26.688447 12588 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:26.675969 12423 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:26.694921 12513 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:26.707911  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:26.697623  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49769.4us
I20260709 12:27:26.713639  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.696466 12382 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:26.728171  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:26.685263 12428 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.666337 12481 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.674290 12603 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16.
I20260709 12:27:26.688838 12444 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:26.738746  6458 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:26.736759  6459 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.728127  6453 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 10.
I20260709 12:27:26.734118 12234 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.730657 12517 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:26.790884  6459 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:26.691530 12375 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:26.748823  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:26.713296 12203 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:27:26.765121  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 116884us
I20260709 12:27:26.763981  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:26.771243  6455 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:26.735105 12588 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.691298 12518 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:26.736920 12575 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:26.743764 12578 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:26.745645 12293 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.707429 12309 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:26.727800 12276 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
W20260709 12:27:26.829797 12593 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
W20260709 12:27:26.851464  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91747.2us
I20260709 12:27:26.731150 12586 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.092s
I20260709 12:27:26.835175 12577 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:26.817179  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68495.3us
I20260709 12:27:26.826138  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:26.822338  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58465us
I20260709 12:27:26.768381 12428 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:26.776046 12444 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.768085 12513 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:26.796808 12203 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.772737 12382 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:27:26.834685 12234 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:26.844132 12318 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.832075 12603 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:27:26.860982 12567 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.870002 12517 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:26.871505 12594 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:27:26.873329 12518 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.873924 12566 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.863070 12608 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:26.846927 12588 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:26.877842 12579 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:26.841629 12412 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:26.881596  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:26.961475  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79959us
I20260709 12:27:26.846680 12578 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.848206 12309 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:26.915078  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89014.5us
I20260709 12:27:26.967972  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:26.856422  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:26.904945 12361 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:26.900233 12444 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:26.920084 12172 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.637s	user 0.007s	sys 0.003s
I20260709 12:27:26.933503 12160 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:26.920113 12577 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:27:26.945866  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 249988us
I20260709 12:27:26.904176 12203 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:26.963310  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 308114us
I20260709 12:27:26.963330  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:26.970729  6457 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:26.933944 12586 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:26.851080 12339 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.980273  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:26.863756 12506 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.880946 12366 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:26.963515 12414 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:26.965803 12578 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:26.991039  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 300686us
I20260709 12:27:26.974138 12518 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:26.980686 12596 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 12:27:26.988426  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:26.995229  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:26.995998 12144 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:27:26.978009 12577 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.021004  6380 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:26.976223 12160 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:27.000185  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:27.034854  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.035481  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:26.981694 12607 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:26.986048 12361 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:26.991603 12602 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:27.054008  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:26.985208 12293 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:26.999470 12515 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:27.046077 12412 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:27.053467  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74629.1us
I20260709 12:27:26.997016 12518 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:27.019776 12481 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.049858 12577 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:27.047541 12603 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:27.065864  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:27.048081 12527 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.021682 12568 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.063627 12543 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:27.035810 12614 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:27:27.053517 12331 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:27:27.070318 12518 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:27.105548  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 105479us
I20260709 12:27:27.083037  6384 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [13.7-14.8]
I20260709 12:27:27.062094 12144 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:27.076243  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.078686  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.079589  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.046247 12606 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 12:27:27.103113  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.108723  6304 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:26.995543 12586 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.115339  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.031807 12309 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.16, Last appended by leader: 16, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.119524  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.062762 12457 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:27:27.121201  6303 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:27.122201  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.096503 12515 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:27.085568 12613 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.096184 12234 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.158805  6301 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:27:27.162401  6301 pending_rounds.cc:85] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 8
I20260709 12:27:27.162779  6301 pending_rounds.cc:107] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:27:27.163120  6301 raft_consensus.cc:2889] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:27.097381 12603 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:27.095867 12543 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:27.160372  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:27.161067  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 95830.5us
I20260709 12:27:27.070637 12412 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.166630  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:27.197618  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:27.199615  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.066133 12602 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.202162  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.203070  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:27.204862  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.209790 12481 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.200388 12515 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:27.195998 12234 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.153389 12617 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.215817  6378 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:27.249228 12144 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:27.250746 12144 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.252707 12211 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.253402 12211 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.256186 12211 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.169623 12276 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:27.283128  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.062851 12607 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.213176 12527 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.218928 12591 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:27.319860  6301 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:27.320739  6378 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:27:27.321201  6378 pending_rounds.cc:85] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 9
I20260709 12:27:27.321492  6378 pending_rounds.cc:107] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:27:27.321774  6378 raft_consensus.cc:2889] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:27.284235 12309 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.119897 12160 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.332059  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.347522  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:27.348014  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.348824  7314 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:27.349030  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.350034  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:27.350337  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:27.350914 12117 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:27.351176  7319 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:27.351263  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:27.351446 12117 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.352030  7314 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:27.217728 12481 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.365288  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:27.361451  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 284608us
I20260709 12:27:27.219463 12618 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.285s
I20260709 12:27:27.319500 12591 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:27.212487 12613 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.373708  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.373999  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.394230  7317 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 10.
I20260709 12:27:27.364409  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.219058 12621 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.265s
I20260709 12:27:27.398039  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.366662 12623 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:27.345417 12457 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:27.394428 12636 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:27.220085 12515 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.395553  7331 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 10.
I20260709 12:27:27.278155 12617 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.411438  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.437458  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.371351 12603 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.415835 12586 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.415035 12423 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:27.441144 12600 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.697s	user 0.000s	sys 0.006s
W20260709 12:27:27.412362 12625 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:27.443176  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2729) took 1117 ms. Trace:
W20260709 12:27:27.443754 12635 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:27:27.411674 12630 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:27.446864 12481 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.415230 12457 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:27.426656  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58383.3us
I20260709 12:27:27.443372 12203 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:27.449599 12526 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.851s	user 0.005s	sys 0.006s
I20260709 12:27:27.450696 12203 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.393630 12160 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.426450  7314 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:27.452593  6382 raft_consensus.cc:2368] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13.
I20260709 12:27:27.454653  7335 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:27.361673 12276 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:27.370908 12581 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.910s	user 0.006s	sys 0.008s
I20260709 12:27:27.443763  6385 rpcz_store.cc:276] 0709 12:27:26.326179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.326307 (+   128us) service_pool.cc:224] Handling call
0709 12:27:26.326662 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.330174 (+  3512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:26.330208 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:26.337267 (+  7059us) log.cc:844] Serialized 26 byte log entry
0709 12:27:26.608355 (+271088us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:26.608398 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.608462 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:27.345878 (+737416us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:27.442688 (+ 96810us) raft_consensus.cc:1685] finished
0709 12:27:27.442735 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.443069 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3099,"raft.queue_time_us":115502,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"spinlock_wait_cycles":25414912,"thread_start_us":167523,"threads_started":1,"wal-append.queue_time_us":313219}
W20260709 12:27:27.466747  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 112146us
W20260709 12:27:27.430770 12616 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 12:27:27.449836 12457 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:27.470455  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59117.5us
I20260709 12:27:27.371307 12527 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:27.402382 12591 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:27.451676 12331 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.473245  7316 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:27.462114 12636 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:27.450281 12526 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.468698 12581 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.493132  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.493723  6308 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:27.494921 12581 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.496001  6304 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15.
I20260709 12:27:27.409557 12623 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.507052  6454 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:27.470343 12276 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.511574  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:27.512756 12575 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.008s
I20260709 12:27:27.500119 12639 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:27.488010 12527 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.489485 12331 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.525552 12637 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:27.531859 12444 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.555s	user 0.012s	sys 0.000s
I20260709 12:27:27.543085 12444 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.544392 12591 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.544445  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:27.530735  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.533094  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.533226  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:27.549358 12621 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.549894 12621 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:27.536388  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.537263 12607 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.507073 12634 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:27:27.534739 12613 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.492295 12636 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.554096  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.564920  6451 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:27.565059  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:27.551597 11996 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.844s	user 0.008s	sys 0.004s
I20260709 12:27:27.577878 12611 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.579946 12611 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.582139 12611 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:27.558406 12445 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.868s	user 0.007s	sys 0.005s
I20260709 12:27:27.574573 12639 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.588981 12639 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.591689 12639 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.572770 12375 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:27.531463 12575 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.595283 12575 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.600530 12623 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.600690 12331 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.575783 12444 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.574150 11996 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.603801 12621 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.604799 12617 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.605448 11996 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.602267 12636 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:27.576872 12428 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.669s	user 0.008s	sys 0.004s
W20260709 12:27:27.612795  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 505646us
I20260709 12:27:27.615331  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.576500 12339 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:27.632138  6457 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
W20260709 12:27:27.642021  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 161247us
I20260709 12:27:27.575317 12276 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.663153  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:27.666838  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:27.656157  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:27.676364  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:27.669999  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:27.682880  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:27.683038  6453 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:27.640048 12634 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.689086  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.690456  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:27.690068  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:27.692070  6451 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 9.
I20260709 12:27:27.610098 12445 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.693182  6450 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:27:27.660079  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:27.694931  6457 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:27.630805 12428 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.686470 12506 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:27.653189 12581 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:27:27.706188  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:27.687723 12631 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:27.689785 12621 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.695411 12611 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:27.707671  6307 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:27.711594 12577 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.607s	user 0.012s	sys 0.000s
I20260709 12:27:27.692901 12339 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:27:27.721371  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.712610 12445 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:27.713339 12578 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.693s	user 0.010s	sys 0.000s
I20260709 12:27:27.726358 12578 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.728566 12578 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.694109 12651 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:27:27.713753 12428 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.714799  6452 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:27:27.733904 12293 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:27.734398 12293 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:27.735430 12293 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.736917 12293 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:27.709628 12518 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 12:27:27.703833 12506 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:27.717662 12605 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.746s	user 0.004s	sys 0.000s
I20260709 12:27:27.723839 12361 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.723868 12644 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:27.725033 12646 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.746516  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:27.724735 12527 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.747748  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:27.751992  6305 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 10: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:27.753705  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.756189  7316 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:27.716051 12588 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.742s	user 0.009s	sys 0.000s
I20260709 12:27:27.757236  6451 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:27.797757  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114965us
I20260709 12:27:27.801867  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.818805 12588 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.819738  6452 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:27.819943  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.820685  7320 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:27.820989 12588 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.821177 12646 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.822269  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.823419 12646 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.823508  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.752852 12506 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.824759  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.825911  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.755661 12518 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.819010  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.832409  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.829413  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:27.833307  6451 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:27.835536  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:27.834174  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:27.837098  6305 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:27.836702  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.837916  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.839062  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.723555 12577 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.845439  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:27.852319  6305 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:27.854930  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.845440 11996 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:27.857695  6303 raft_consensus.cc:2368] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11.
I20260709 12:27:27.835847 12603 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:27.758329 12361 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.888938  6300 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:27.886943 12644 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:27.887939  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:27.902186  7314 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:27.889778  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:27.851243 12414 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:27.892887  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:27.897734  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:27.887460  6301 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:27.900499  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:27.911111  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:27.912506  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:27.907563  6301 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:27.915092  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:27.916864  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:27.924799  6305 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:27:27.887567 12428 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.887719 11996 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:27.899175 12276 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.901789 12644 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:27.934981  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:27.922523 12658 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:27:27.935180  6376 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:27.893963 12659 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:27.935936  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:27.937024 12648 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:27:27.940059  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:27.935653  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.940210  6380 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.941465  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:27.936618  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:27.945179  6377 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.917837 12309 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:27.917843 12515 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:27.907532 12445 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:27:27.921970 12382 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:27.924338 12444 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:27.918538 12527 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:27.941851  6301 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.952380  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:27.954900  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.952486  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.939077 11996 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:27.918030 12414 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.943007 12659 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:27.963150  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:27.948733 12515 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.965463  6377 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:27.941380 12428 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:27.970302  6301 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:27.970629 12657 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:27:27.966954 12382 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:27.989279 12428 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.993494 12414 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:27.995843 12660 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:27.943646 12276 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.011629 12644 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:27.957294 12309 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:28.015313 12309 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.018962 12518 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.019289 12602 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:27.970042 12515 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:28.027071  7316 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:28.028805  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:28.031203  6380 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:27.965335 12659 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.045506  6301 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:27:28.033612 12585 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:28.050277  6301 pending_rounds.cc:85] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:27:28.051219  6301 pending_rounds.cc:107] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:27:28.052402  6301 raft_consensus.cc:2889] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:28.067631  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126277us
I20260709 12:27:28.068689  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:28.052938 12585 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:28.065822 12566 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 12:27:28.082346  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 165506us
I20260709 12:27:28.076236 12445 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:27:28.078471 12172 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:28.091459 12445 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.113970 12428 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:28.116210  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:28.113225  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.119779  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:28.083693 12506 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.119338 12445 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:28.128988  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.104127 12172 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.140611  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.141860  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:28.120664 12414 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
W20260709 12:27:28.144631 12598 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.307s	user 0.005s	sys 0.007s
I20260709 12:27:28.144960  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.128887 12585 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.145921  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 902) took 1963 ms. Trace:
I20260709 12:27:28.146173  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:28.147476  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.131173  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:28.146337  7318 rpcz_store.cc:276] 0709 12:27:26.182949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.183116 (+   167us) service_pool.cc:224] Handling call
0709 12:27:26.183480 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.183996 (+   516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:26.184039 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:26.185454 (+  1415us) log.cc:844] Serialized 26 byte log entry
0709 12:27:26.301349 (+115895us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b48000cd758. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:27:26.647762 (+346413us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:26.647808 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.647874 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:28.143886 (+1496012us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:28.145709 (+  1823us) raft_consensus.cc:1685] finished
0709 12:27:28.145731 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.145843 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":152,"raft.run_cpu_time_us":3967,"raft.run_wall_time_us":116230,"spinlock_wait_cycles":17421312,"thread_start_us":412054,"threads_started":1,"wal-append.queue_time_us":529201}
I20260709 12:27:28.149066 12670 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:28.149505 12670 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.149993 12670 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:28.124020 12621 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.159368 12610 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:28.161267 11996 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.162308  6452 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:28.128572 12669 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:28.163568 12172 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.133426 12276 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.166837  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.167654  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.167281 12670 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.192198  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.192880  6458 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:28.209933  6459 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:28.212365  6459 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:28.219089  6304 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:28.224962  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:28.218604 12668 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:27:28.227705 12574 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:28.227593 12636 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:27:28.231900 12574 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.236009 12574 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:28.237890 12599 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.403s	user 0.003s	sys 0.000s
I20260709 12:27:28.241091 12659 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.233682 12588 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 12:27:28.242769 12588 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:28.245507 12588 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:28.246654 12588 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.248379 12588 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:28.218120 12585 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:28.250857 12585 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.177093 12621 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.248685 12670 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.220038 12518 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:28.250146  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.268759  6305 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:28.232344  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:28.262188 12429 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:28.263592 12664 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
W20260709 12:27:28.269709  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.271773  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.191138 12610 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.266259  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:28.193156 12669 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.304854  6351 leader_election.cc:400] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15.
I20260709 12:27:28.305371  6351 leader_election.cc:403] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:27:28.252914 12585 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.312495  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.301110 12428 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:28.300450 12636 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:28.331919 12675 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.013s
I20260709 12:27:28.347036  6305 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:28.304203 12429 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.326696 12636 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.314503 12518 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.341991 12572 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15.
W20260709 12:27:28.363652  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58940us
I20260709 12:27:28.342439 12506 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.361778 12515 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.363384 12428 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.370069 12659 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.372020 12669 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.373337 12621 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.374612  6310 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:28.375895  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.376070  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.377074  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 910) took 1396 ms. Trace:
I20260709 12:27:28.386886  6379 rpcz_store.cc:276] 0709 12:27:26.980214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.980482 (+   268us) service_pool.cc:224] Handling call
0709 12:27:26.990561 (+ 10079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.164105 (+173544us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.164158 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:27.164226 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:27.164278 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.376840 (+1212562us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54001941d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:28.376885 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.377021 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398411520}
I20260709 12:27:28.357561 12568 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.389204 12591 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.382570  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:28.390058  7318 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.389266  7317 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 16. Preceding OpId from leader: term: 19 index: 16. (term mismatch)
I20260709 12:27:28.394052  7317 pending_rounds.cc:85] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 15
I20260709 12:27:28.394268  7317 pending_rounds.cc:107] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 18.16
I20260709 12:27:28.394472  7317 raft_consensus.cc:2889] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.394997  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.361563 12636 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.397650 12117 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.383605  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.388725 12428 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.399065 12361 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.400090 12610 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.400219 12428 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.395262 12515 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.387924  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:28.402947  6305 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:28.403803  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:28.404615  6453 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:28.405490  7318 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:28.406224  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:28.407066  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.407388  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.407678  6301 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:28.403898  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.390295  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.409976  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.410435  6300 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:28.419448  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:28.420320  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:28.421182  7316 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:28.421944  6459 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:28.424558  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:28.425871  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.426183  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:28.426473  6459 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
W20260709 12:27:28.514184  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.519134  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.520347  6458 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:28.417475 12429 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.536844  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:28.522076  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 95951.5us
I20260709 12:27:28.523927  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.538223  6458 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:28.539563  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.540759  6459 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:28.529549  7316 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:28.529549  6453 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:28.530849  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.541858  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.406153 12234 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.542690  6453 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:28.542124  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.544467  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.539639  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.543923  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:28.545337  6301 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:28.546778  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:28.547363  6459 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:28.549474  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.554319  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:28.555706  6302 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:28.557710  6453 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:28.526803 12621 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.560444  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:28.531064 12585 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:28.523641 12608 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:28.626529 12574 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.524585 12445 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.623852 12518 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.538846 12578 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.607052  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:28.603776  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:28.711741  7335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1128) took 1364 ms (client timeout 1665 ms). Trace:
W20260709 12:27:28.712414  7335 rpcz_store.cc:269] 0709 12:27:27.347509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.347770 (+   261us) service_pool.cc:224] Handling call
0709 12:27:28.711477 (+1363707us) spinlock_profiling.cc:243] Waited 87.6 ms on lock 0x7b5400508554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:28.711655 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201376512}
I20260709 12:27:28.521591 12591 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
W20260709 12:27:28.656792  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 102293us
I20260709 12:27:28.647040  6302 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:27:28.614814  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.656880  6383 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:28.716804  6459 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:28.607684  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:28.717748  6458 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:28.658324  6300 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:28.660336 12572 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:28.715572  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:28.662428  6451 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:27:28.720645  6384 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:28.666854  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2893) took 2113 ms. Trace:
I20260709 12:27:28.720813  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:28.668114 12637 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:28.668592 12592 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.833s	user 0.040s	sys 0.064s
I20260709 12:27:28.668865 12624 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:28.665773 12423 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:28.604846  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.729900  6305 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.678280  6380 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:28.679329  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2788) took 1593 ms. Trace:
I20260709 12:27:28.684851 12611 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.691679  6304 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:28.693395  6457 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:28.694630 12412 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.696746 12646 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:27:28.697448  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:28.698251 12659 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:28.698657 12665 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:28.699954  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:28.738348  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:28.640779 12682 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.703682 12684 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:28.712363  6381 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:28.714645  6382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3027) took 1645 ms (client timeout 1824 ms). Trace:
I20260709 12:27:28.716830  6301 raft_consensus.cc:2410] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:28.719883 12585 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:28.723165  6385 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:28.721129  6455 rpcz_store.cc:276] 0709 12:27:26.553843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.554884 (+  1041us) service_pool.cc:224] Handling call
0709 12:27:26.555250 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.843490 (+288240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.666448 (+1822958us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b5400344254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:28.666768 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495873280}
I20260709 12:27:28.724644  6300 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:27:28.725277  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:28.726992  6302 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:27:28.727735  6459 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:28.731110  6309 rpcz_store.cc:276] 0709 12:27:27.085759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.086010 (+   251us) service_pool.cc:224] Handling call
0709 12:27:28.679031 (+1593021us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54002062d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:28.679235 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329324800}
I20260709 12:27:28.657459 12621 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.754058  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:28.729293 12624 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:28.745576  6382 rpcz_store.cc:269] 0709 12:27:27.065597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.065765 (+   168us) service_pool.cc:224] Handling call
0709 12:27:28.706650 (+1640885us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54003d2cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:28.710810 (+  4160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792084608}
I20260709 12:27:28.726430 12608 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:27:28.720904  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3320) took 1033 ms. Trace:
I20260709 12:27:28.720314 12578 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.749634 12684 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.764643  6305 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:28.760131  6451 rpcz_store.cc:276] 0709 12:27:27.687561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.844046 (+156485us) service_pool.cc:224] Handling call
0709 12:27:27.844371 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.662062 (+817691us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b54000ca920. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa321edc0 00007f6fa32a91c0 00007f6fa32aa231 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c
0709 12:27:28.720619 (+ 58557us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.720838 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11428736}
I20260709 12:27:28.746954 12610 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:28.730268 12636 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.732201 12572 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.734269 12518 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:27:28.744856 12604 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:28.729537 12637 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.777585  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:28.779402  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 832) took 2611 ms. Trace:
I20260709 12:27:28.739886 12611 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.754948 12580 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:27:28.770215 12610 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:28.746863 12682 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.800122 12682 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.801589 12580 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:28.802029 12580 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:28.802940 12580 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.803577 12445 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.804267 12580 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:28.805086 12684 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.805965 12574 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.778510 12361 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:28.806947 12361 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.810990 12518 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:28.811852 12518 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.771730 12543 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.815930 12543 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.809532 12361 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.794764 12585 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.819963 12578 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.820822 12578 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:28.821192 12578 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:28.796236 12572 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.797226  6457 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:28.766870 12665 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:27:28.822077 12572 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:28.822674 12572 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:28.823462 12572 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.824157 12637 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.789880 12604 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:28.826342 12604 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.830806 12624 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.832231 12624 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:28.836186 12636 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.837105 12591 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:28.837725 12591 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:28.838742 12591 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.840245 11996 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:28.841377 12526 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:28.846833 12610 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.780009  7329 rpcz_store.cc:276] 0709 12:27:26.163327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.163496 (+   169us) service_pool.cc:224] Handling call
0709 12:27:26.173552 (+ 10056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.174001 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:26.174031 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:26.174612 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:27:26.540024 (+365412us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:26.540062 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:26.540123 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:28.667801 (+2127678us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:28.769749 (+101948us) raft_consensus.cc:1685] finished
0709 12:27:28.769781 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.774743 (+  4962us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":954,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15374,"spinlock_wait_cycles":13553024,"thread_start_us":346810,"threads_started":1,"wal-append.queue_time_us":588443}
W20260709 12:27:28.850867  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 45579ad5e67d4843a6f90bf59939a2d7: Illegal state: Replica 4268023ecea149cd83cba77db53c3100 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:28.852217  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2565) took 2537 ms. Trace:
I20260709 12:27:28.852639  6307 rpcz_store.cc:276] 0709 12:27:26.315036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.475272 (+160236us) service_pool.cc:224] Handling call
0709 12:27:26.475731 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.719522 (+1243791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.851944 (+1132422us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001414d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:28.852116 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2828997376}
I20260709 12:27:28.854480  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2896) took 2202 ms. Trace:
W20260709 12:27:28.855561  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2792) took 1691 ms (client timeout 1755 ms). Trace:
I20260709 12:27:28.798101 12611 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:28.854887  6454 rpcz_store.cc:276] 0709 12:27:26.652144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.652268 (+   124us) service_pool.cc:224] Handling call
0709 12:27:26.652631 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.508534 (+855903us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.850913 (+1342379us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54000ebd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:28.854419 (+  3506us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1964336384}
W20260709 12:27:28.856038  6303 rpcz_store.cc:269] 0709 12:27:27.156864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.157085 (+   221us) service_pool.cc:224] Handling call
0709 12:27:28.847971 (+1690886us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b5400063bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:28.848121 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476404096}
W20260709 12:27:28.855794  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 221731us
I20260709 12:27:28.855278 12619 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:28.855950 12527 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.866587 12318 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:28.868666 12615 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:28.887020 12518 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:28.881644 12318 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.881685 12619 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.905129 12429 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.910670  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:28.911453  6307 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:28.912002  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.912985  6302 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:28.900096 12318 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:28.918601  6302 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:28.924583  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:28.920361  7288 connection.cc:256] RPC call timeout handler was delayed by -1.0747509970001374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27540000000000003s for a response
I20260709 12:27:28.921865  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.915969  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:28.882874 12527 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.926605  6304 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:28.927211  6307 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:28.927825  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:28.928917  6303 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:27:28.880656 12578 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:28.896416 12615 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:28.939591  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 162066us
I20260709 12:27:28.941305  6303 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:28.941500  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:28.943405  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 189456us
I20260709 12:27:28.928589 11996 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:28.944130  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:28.952172 12636 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:28.954802  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:28.955895  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.921277 12429 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:28.962127  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:28.936808 12659 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:27:28.958123 12588 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:27:28.967183  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:28.963912 12636 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:28.959556 12578 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:28.967533  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 229249us
I20260709 12:27:28.955425 12690 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:28.964058 12518 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:28.968286 12627 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:28.973436  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:28.968657 12688 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:28.953330 12608 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:28.989523 12653 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.268s	user 0.004s	sys 0.000s
I20260709 12:27:28.968595 12574 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:29.005995  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:28.967089 12691 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:27:28.972563 12543 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:28.999593 12690 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.000069 12627 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:29.014568  6351 leader_election.cc:400] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 10 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11.
I20260709 12:27:29.018297  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.009582 12636 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.022604 12591 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:29.024717 12543 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.027130 12543 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.028568 12543 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:29.022653 11996 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.028680 12627 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.030134 11996 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.034950  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.030827 12693 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:29.037354 11996 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:29.034858 12606 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:29.039609  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.040179  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.010679 12692 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.029004 12543 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:29.043301 12700 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.014035 12690 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:29.046814  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:29.052560  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.048754  6302 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:29.048684  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.051481  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:29.057041  6307 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.060014  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.061436  7332 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:29.056432 12543 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.064141  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.066042  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.066813  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.067790  7335 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:27:29.060014  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:29.070525  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.060153  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.072137  7317 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:29.074098  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.075212  6305 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:29.075330  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.076007  7317 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.080260  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.041188 12627 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:29.087544  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.052736 12619 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.089296  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.091400  6380 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:29.092862  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.093504  6384 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:29.093694  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.079499 12574 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:29.096115  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.050915 12606 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.082913 12527 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:29.111914 12574 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:29.113703 12606 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.113157 12574 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:29.095927 12657 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.102s	user 0.014s	sys 0.016s
I20260709 12:27:29.056254 12692 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.110994  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.085579 12160 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:27:29.090391 12703 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:27:29.084052 12706 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.079545 12615 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.101661 12611 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.116115 12614 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:29.090955 12608 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.119073  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.119649  6379 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:27:29.079798 12695 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:27:29.085816 12518 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:27:29.079149 12610 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.123441  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.183487  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:29.124231  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.125728  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.184672  7314 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:29.184854  6303 raft_consensus.cc:2410] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 16 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:29.127322  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:29.129055  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.129315  7316 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:29.130786  7317 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:29.132444  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3328) took 1022 ms. Trace:
I20260709 12:27:29.186172  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.132551  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:29.124178 12543 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.133303 12686 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.134250 12423 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.108150  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.195750  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:29.147006  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 141064us
I20260709 12:27:29.144829  6384 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:29.152199  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.148296  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.205648  7335 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:29.173767  7319 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
W20260709 12:27:29.145371 12657 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.152s	user 0.015s	sys 0.017s
I20260709 12:27:29.180269 12694 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.163815 12703 consensus_queue.cc:1243] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:29.198032  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.186576  6452 rpcz_store.cc:276] 0709 12:27:28.077326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.147258 (+ 69932us) service_pool.cc:224] Handling call
0709 12:27:29.099620 (+952362us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b54002c8654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:29.099762 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32681216}
I20260709 12:27:29.153317 12619 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:29.199877  7320 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:29.194576  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.219532  6458 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:29.165675 12614 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:29.172499 12624 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.172410  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.187386  6301 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.208351  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.193027 12567 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.210453  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.169407 12611 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.231426  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.212779  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.217453  6455 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:29.221493  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.222859  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.224275  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.234715  7317 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.202728 12591 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:29.154449 12636 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:29.228549  6380 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:29.223742 12615 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.229048  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:29.234697  7314 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:29.238307  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.228248  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.211773 12506 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:29.253544  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.253513  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:29.230751 12606 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.222440 12694 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.266153  6380 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:29.266291 12606 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.267669  6380 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:29.267871 12567 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:29.268599 12606 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.210475 12686 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.257270 12591 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:29.270577  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.271242  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:29.271478 12591 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.272387 12692 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:29.274503 12614 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.275997 12614 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:29.213013 12423 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.211123  7329 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:29.264578  6307 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:29.256199  7317 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:29.268488 12567 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.264379  6377 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:29.282251  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.272063 12606 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:29.263967  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.283491 12606 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:29.284009 12606 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:29.284606  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.285395  6456 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:29.227969 12619 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:29.288301 12662 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.197s	user 0.001s	sys 0.003s
I20260709 12:27:29.289477 12619 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.290148 12567 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:29.291252 12518 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.291206 12318 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.292426 12588 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:29.292865 12588 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:29.293758 12588 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.296171 12577 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:29.297392  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 43961us
W20260709 12:27:29.300015 12685 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.403s	user 0.002s	sys 0.000s
I20260709 12:27:29.220148 12670 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:27:29.303956  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.304159  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.341344  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2793) took 2177 ms. Trace:
I20260709 12:27:29.236049 12624 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.309835  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.379519  7335 raft_consensus.cc:2368] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11.
I20260709 12:27:29.311172  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2798) took 1814 ms. Trace:
I20260709 12:27:29.286375  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.314854  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.380616 12624 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.380959  6307 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:29.381042  6379 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:29.381326  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.381999  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.382565  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.383359  6456 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:29.380537  6308 rpcz_store.cc:276] 0709 12:27:27.496386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.496536 (+   150us) service_pool.cc:224] Handling call
0709 12:27:27.500330 (+  3794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:27.843416 (+343086us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.843446 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:27.844694 (+  1248us) log.cc:844] Serialized 26 byte log entry
0709 12:27:27.968562 (+123868us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:28.008236 (+ 39674us) log.cc:844] Serialized 14 byte log entry
0709 12:27:28.012052 (+  3816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.247305 (+235253us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54002d28d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:28.247369 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.287601 (+1040232us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:29.310992 (+ 23391us) raft_consensus.cc:1685] finished
0709 12:27:29.311011 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.311122 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27137,"prepare.queue_time_us":25319,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":37794,"raft.run_cpu_time_us":583,"raft.run_wall_time_us":11355,"spinlock_wait_cycles":370035840,"thread_start_us":242507,"threads_started":2,"wal-append.queue_time_us":212500}
I20260709 12:27:29.223685 12527 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:27:29.383921  6307 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:27:29.384208  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.385288  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.320206  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:29.336692  7331 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:29.336796  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.342355  6310 rpcz_store.cc:276] 0709 12:27:27.163992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.164196 (+   204us) service_pool.cc:224] Handling call
0709 12:27:27.164644 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:28.386218 (+1221574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.341045 (+954827us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54000b3454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:29.341272 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2782112512}
I20260709 12:27:29.343827  6377 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:29.346133  7332 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:29.267818 12694 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:29.390444  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 176842us
I20260709 12:27:29.348232  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.348870  6458 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:29.259641 12657 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:29.352789 12318 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.354907  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:29.355247  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.358546  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:29.402019  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43570.7us
I20260709 12:27:29.360251 12627 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:29.402091 12694 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:29.403012 12624 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:29.403510 12627 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.403606 12624 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:29.405233 12624 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.406840  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.407788 12361 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.408434  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.407796 12624 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.264424 12636 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.376654 12543 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.378151 12610 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.220080 12706 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:27:29.412142 12694 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.387478  7314 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:29.415516  6376 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:29.417572 12636 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:29.387605  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67500us
I20260709 12:27:29.419158  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.342067  7320 raft_consensus.cc:2376] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20. Re-sending same reply.
I20260709 12:27:29.396581  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.409512 12361 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:29.424225  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.395807  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.459239  6458 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:29.459981 12610 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:29.460582  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:29.460973  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 72510.6us
I20260709 12:27:29.461372  6458 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.461921 12610 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:29.463187 12543 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:29.463574 12543 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:29.410338  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:29.413061 12694 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.360831  6301 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:29.403532  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:29.497088  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 93633.8us
I20260709 12:27:29.428797  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:29.456699  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.466109  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:29.467680  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:29.398584 12577 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:29.511468  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 82793.3us
I20260709 12:27:29.517004  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.512545  6458 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:29.498011  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:29.524848  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:29.526690  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 116405us
I20260709 12:27:29.516980 12578 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:29.399712 12657 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:29.533597  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:29.525863  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:29.535050  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 67434us
I20260709 12:27:29.465406 12445 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:29.545322  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.544852  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:29.548599  6377 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:29.333051 12615 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.421947 12117 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.135s
I20260709 12:27:29.503293 12636 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:29.389966 12606 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.565579  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:29.566018  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.565224  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:29.524251 12581 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:29.572926  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.577008  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.577744  6458 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:29.577025  6308 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:29.542605 12318 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.582703 12318 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:29.583192 12318 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.524087 12693 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.581687 12382 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:29.564967 12361 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.591385 12382 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.591542 12361 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.592720  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.593430  6379 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:29.593739 12382 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:29.593954 12361 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.579559 11996 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:29.496066 12619 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:27:29.585736 12429 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:29.594349  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.595108  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.536321 12695 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:29.522696 12707 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:27:29.541664 12665 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:29.542155 12684 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:29.609540 12657 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.607808 12695 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.611268 12429 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:29.612051 12615 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.549033 12703 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 12:27:29.607620 12690 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.528s	user 0.012s	sys 0.000s
I20260709 12:27:29.577934 12606 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.613768 12518 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.615288 12621 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:29.594458 12382 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:29.549219 12682 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.090s
I20260709 12:27:29.607133 12693 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:29.608656 12618 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:29.626088 12694 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:29.627331 12711 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:27:29.579728 12636 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.630865 12700 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:29.634128  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.648365 12703 consensus_queue.cc:1243] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:29.636494 12695 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:29.536698 12580 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:27:29.681710 12580 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:29.682358 12580 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:29.682574 12693 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:29.683650 12693 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.683521 12580 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.684656  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.685334 12580 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:29.685948  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.691000  6381 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:29.692168  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.692391 12636 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.693490  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.693724 12318 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.695469  7329 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:29.697847 12318 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:29.545187 12445 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:29.698668 12445 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:29.699402 12339 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.700014 12339 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.639755  6456 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:29.703804  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.704524 12572 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:29.705024 12572 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:29.705353  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:29.599115 12688 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
I20260709 12:27:29.706938  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.707191 12688 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:29.708140  6384 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:29.709331  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.710101 12669 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:29.710791 12669 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.711201  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.711869  6381 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:29.712766  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.655249 12621 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.713101 12669 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:29.713825  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.594666 11996 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.715041 11996 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:29.715327 12669 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:29.716279  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.717013  6455 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:29.600519 12581 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:29.645484 12665 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:29.723142  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.595773 12577 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:27:29.634842  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:29.612371  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.722592  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:29.719978  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.713629 12621 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.621683 12684 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.651829 12657 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:29.651455 12382 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:29.613622  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:29.708658  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.715978 12669 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.726461  6459 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:29.783054  6456 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.679569 12700 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:29.648980 12690 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.797652  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.791118  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.702412 12339 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:29.799657  6458 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:29.724035 12506 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:29.748071 12684 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:29.813311  6385 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:29.815348  6385 raft_consensus.cc:606] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.815788  6385 raft_consensus.cc:614] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:29.706102 12572 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.850929  6382 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:29.850086  6455 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
W20260709 12:27:29.852708  7287 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.007s	sys 0.019s
W20260709 12:27:29.855890  7287 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.008s	sys 0.019s
I20260709 12:27:29.855099 12572 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:29.865609  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.867082  6456 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:29.843001 12339 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:29.856439  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 547521us
I20260709 12:27:29.715324 12543 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.872859 12339 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.874287 12581 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.878189 12339 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:29.855712 12700 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.881582 12669 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:29.843581 12670 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:29.882700 12670 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.884742 12670 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:29.885005  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.846798 12615 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:29.873260 12543 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:29.887428  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.889211  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:29.844132 12715 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:29.896651 11996 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.898527  6458 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 11.
I20260709 12:27:29.901464  6458 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:27:29.892788  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.908530  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:29.904373  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.943338  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50655.2us
I20260709 12:27:29.855759 12621 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:29.918171  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.946982  6309 raft_consensus.cc:2368] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:29.947144  6454 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:29.947546  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 949) took 1023 ms. Trace:
I20260709 12:27:29.948077  6309 rpcz_store.cc:276] 0709 12:27:28.924113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.924247 (+   134us) service_pool.cc:224] Handling call
0709 12:27:29.918056 (+993809us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182eee 000000000036eb39 00007f6fa81f61e0 00007f6fa82bf0c4 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:27:29.947465 (+ 29409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3140992}
W20260709 12:27:29.921623  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.921836 11996 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:29.923367  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.922719  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.004s	sys 0.020s
I20260709 12:27:29.891126 12670 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:29.890686  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 356117us
I20260709 12:27:29.944159  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:29.924911 12526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.013s	sys 0.011s
I20260709 12:27:29.902441 12716 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:29.950806  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.004s	sys 0.020s
W20260709 12:27:29.961279  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 561940us
W20260709 12:27:29.961742  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 424386us
I20260709 12:27:29.962118  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:29.963263  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 429746us
I20260709 12:27:29.964972  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:29.891449 12615 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:29.951872  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:29.954339  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.967424 11996 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:29.923062 12361 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.070640  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:30.070986  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:30.072286  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:30.073676 11996 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:27:30.074118 11996 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:30.074882 11996 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.956921  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:29.954427 12578 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:30.080430 12578 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.954339  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:30.082073  7288 leader_election.cc:400] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:30.082500  7288 leader_election.cc:403] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:30.069897  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.990854  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.897397 12669 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:27:30.084154  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:30.092420  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:30.094084  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:29.951030 12117 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:30.085134  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 120211us
W20260709 12:27:30.105252  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 382782us
I20260709 12:27:30.108618  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:29.867177 12382 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.951498 12339 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:29.960512 12526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.013s	sys 0.011s
I20260709 12:27:30.086596  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:30.267436  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:30.115664  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:29.905031 12715 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:29.959875 12575 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.267s
I20260709 12:27:30.225318  6308 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:29.970180 12621 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:29.969448 12615 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:30.228259  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 237569us
I20260709 12:27:30.324321  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:29.971014 12716 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:30.236740  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.071120 12619 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:30.325866 12619 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.097878 12670 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:30.114609  6275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.004s	sys 0.021s
W20260709 12:27:30.326927  6275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.009s	sys 0.028s
W20260709 12:27:30.327239  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 761926us
I20260709 12:27:30.125852  6425 leader_election.cc:400] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11.
I20260709 12:27:30.329141  6425 leader_election.cc:403] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:29.968389 12721 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:30.284155  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 211738us
W20260709 12:27:30.330425  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.683s (SENT) [suppressed 15 similar messages]
I20260709 12:27:29.920725 12665 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.109547 12686 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:30.234215 12591 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.235527 12624 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.381495  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 275664us
I20260709 12:27:30.382261  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:30.379837  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:30.383733  6454 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:30.089874 12361 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.097430 12445 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:29.960613 12506 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.326643 12619 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:30.402639  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 448311us
W20260709 12:27:30.413306  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 319183us
I20260709 12:27:30.413679  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:30.418609  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:30.429286  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 105064us
I20260709 12:27:30.370879 12382 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:30.431753  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.364830 12117 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.430429  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:30.435972  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 105517us
W20260709 12:27:30.441967  6274 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.824s (SENT)
I20260709 12:27:30.372383 12614 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.447052  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.448158  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.442013  6454 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:30.366469 12716 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:30.381358 12632 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:30.370607  6202 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 7 to 8, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 8 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:30.477861  6309 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:30.479105  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:30.481045  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:30.392132 12721 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.363448 12707 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.110s
I20260709 12:27:30.490705  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2921) took 1159 ms. Trace:
I20260709 12:27:30.497557  6454 rpcz_store.cc:276] 0709 12:27:29.331002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.400976 (+1069974us) service_pool.cc:224] Handling call
0709 12:27:30.441850 (+ 40874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:30.462035 (+ 20185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.490640 (+ 28605us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:30.516294  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 134094us
I20260709 12:27:30.517668  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:30.517796  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:30.430645 12690 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:30.517946  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 87598.1us
I20260709 12:27:30.419041 12621 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:30.453559 12636 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:30.400622 12624 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.486636 12339 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 10. Current term is 11.
I20260709 12:27:30.493469 12669 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:30.556885  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75791.2us
I20260709 12:27:30.539717 12707 consensus_queue.cc:1243] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:27:30.501984 12614 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:30.627188  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.626929 12619 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.637748 12720 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.257s	user 0.000s	sys 0.001s
W20260709 12:27:30.639495 12741 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:27:30.629869 12526 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:30.643005 12714 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.403s	user 0.000s	sys 0.000s
W20260709 12:27:30.643901 12731 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
W20260709 12:27:30.653751  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.507109 12721 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:30.726260 12543 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.830s	user 0.006s	sys 0.001s
I20260709 12:27:30.673799 11996 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:30.677655  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:30.743857  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66271.8us
W20260709 12:27:30.686311  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.687551  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 170013us
I20260709 12:27:30.690518  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3051) took 1353 ms. Trace:
I20260709 12:27:30.492288 12568 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.670993 12747 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:27:30.544795 12581 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.706830 12746 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
W20260709 12:27:30.707394 12736 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 12:27:30.707669 12754 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:27:30.709232 12684 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.864s	user 0.009s	sys 0.002s
I20260709 12:27:30.709386 12624 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:30.631996 12686 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:27:30.727548  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2832) took 1539 ms. Trace:
I20260709 12:27:30.728065 12445 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:30.674871 12688 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.731315  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.014s	sys 0.011s
I20260709 12:27:30.591542 12670 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:30.752154  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.756937  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.757854  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.743016 11996 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.759011 12715 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:30.692229 12738 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:27:30.760192  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.761240  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3342) took 1563 ms. Trace:
W20260709 12:27:30.717013 12743 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
W20260709 12:27:30.761251  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 921) took 1431 ms (client timeout 1748 ms). Trace:
I20260709 12:27:30.714570 12619 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:30.848302  6377 rpcz_store.cc:269] 0709 12:27:29.329376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.499216 (+169840us) service_pool.cc:224] Handling call
0709 12:27:30.761076 (+1261860us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:30.761198 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57539968}
W20260709 12:27:30.776252  6455 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.004s	sys 0.036s
I20260709 12:27:30.807890  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:30.853965  6455 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.007s	sys 0.036s
I20260709 12:27:30.718797 12339 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:30.817343  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.009s	sys 0.009s
I20260709 12:27:30.815353  6300 rpcz_store.cc:276] 0709 12:27:29.188357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.193312 (+  4955us) service_pool.cc:224] Handling call
0709 12:27:29.193615 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.205755 (+ 12140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.205785 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.212673 (+  6888us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.348695 (+136022us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b4800002958. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:27:29.676589 (+327894us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:29.676638 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.676752 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.727142 (+1050390us) raft_consensus.cc:1685] finished
0709 12:27:30.727166 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.727469 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":170,"raft.queue_time_us":845,"raft.run_cpu_time_us":1931,"raft.run_wall_time_us":139063,"spinlock_wait_cycles":13634944,"thread_start_us":421601,"threads_started":1,"wal-append.queue_time_us":718565}
I20260709 12:27:30.718534 12526 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:30.826233  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.014s	sys 0.011s
I20260709 12:27:30.859978  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.860641  7318 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:30.761741 12608 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:30.847812 12657 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.056s	user 0.012s	sys 0.002s
I20260709 12:27:30.866703  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.867707  7318 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:30.839871 12721 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:30.837772 12730 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:30.869310  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:30.840674 12760 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:30.869957  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:30.871014  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:30.871707  6377 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:30.872480  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:30.873644  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:30.874614 12608 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.877045 12608 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:30.877909 12716 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:30.840402 12614 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:30.878847 12619 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:30.858821 12738 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:30.879688  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.880229  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.880319 12738 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:30.843317 12580 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.157s	user 0.015s	sys 0.000s
I20260709 12:27:30.799335 12568 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:30.882799  6305 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:30.884505 12568 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:30.885666  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:30.820256 12725 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.886803 12725 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:30.887318 12568 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:30.837782 12686 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:30.888197 12686 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.888511  7318 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:30.747316  6381 rpcz_store.cc:276] 0709 12:27:29.337336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.795151 (+457815us) service_pool.cc:224] Handling call
0709 12:27:29.795538 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.795846 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:30.274064 (+478218us) log.cc:844] Serialized 14 byte log entry
0709 12:27:30.575954 (+301890us) log.cc:844] Serialized 14 byte log entry
0709 12:27:30.689915 (+113961us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:30.690006 (+    91us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:30.690082 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.690197 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.690401 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":215699,"prepare.queue_time_us":795509,"raft.queue_time_us":172617,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":31654,"spinlock_wait_cycles":29767936,"thread_start_us":677305,"threads_started":2}
I20260709 12:27:30.889231 12725 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:30.847407 12670 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.889906  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:30.889989 12670 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.891924  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.892436 12670 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:30.854976  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3341) took 1657 ms. Trace:
I20260709 12:27:30.893235  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:30.880247 12716 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:30.893898  7318 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:30.728910 12361 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:30.861184  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.834830  6456 rpcz_store.cc:276] 0709 12:27:29.197750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.868197 (+670447us) service_pool.cc:224] Handling call
0709 12:27:29.868959 (+   762us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.869677 (+   718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.869707 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.870184 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.409094 (+538910us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:30.409178 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.409308 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.760997 (+351689us) raft_consensus.cc:1685] finished
0709 12:27:30.761022 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.761176 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":454883,"threads_started":1,"wal-append.queue_time_us":888274}
I20260709 12:27:30.895502  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:30.846882 12715 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:30.843876 12572 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.979s	user 0.011s	sys 0.000s
I20260709 12:27:30.888608 12686 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.824520 12684 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.826324 12581 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:30.860819  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.009s	sys 0.009s
W20260709 12:27:30.868312  6383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.963s	user 0.007s	sys 0.035s
W20260709 12:27:30.922789  6383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.008s	sys 0.035s
W20260709 12:27:30.922688  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 807072us
W20260709 12:27:30.864454  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 334557us
I20260709 12:27:30.882169 12612 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:30.886396  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:30.900210  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:30.901095  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:30.901468  6273 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.000s	sys 0.025s
I20260709 12:27:30.924240  6377 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:30.924749  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:30.925204  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
W20260709 12:27:30.924496  6273 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.000s	sys 0.025s
I20260709 12:27:30.833475 12688 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:30.931618  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2757) took 1822 ms. Trace:
I20260709 12:27:30.893074  6455 rpcz_store.cc:276] 0709 12:27:29.197535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.857168 (+659633us) service_pool.cc:224] Handling call
0709 12:27:29.891050 (+ 33882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.891289 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:29.917614 (+ 26325us) log.cc:844] Serialized 14 byte log entry
0709 12:27:30.854441 (+936827us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:30.854613 (+   172us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:30.854668 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.854760 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.854908 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29283,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":29430,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":228,"thread_start_us":936426,"threads_started":1}
I20260709 12:27:30.863298 12526 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:30.925838  6377 raft_consensus.cc:606] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:27:30.933863  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.934052  6377 raft_consensus.cc:614] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:30.896190  7319 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.846362 12615 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:30.937425  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:30.932052  6383 rpcz_store.cc:276] 0709 12:27:29.109258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.258046 (+148788us) service_pool.cc:224] Handling call
0709 12:27:29.339537 (+ 81491us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.339868 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:29.896497 (+556629us) log.cc:844] Serialized 14 byte log entry
0709 12:27:30.931192 (+1034695us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:30.931263 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:30.931331 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.931412 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.931556 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14728,"prepare.queue_time_us":1166072,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":6810,"thread_start_us":1523711,"threads_started":2}
I20260709 12:27:30.938664  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58396 (request call id 43) took 1050 ms. Trace:
I20260709 12:27:30.939412  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:30.938393  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:30.976678  6377 rpcz_store.cc:276] 0709 12:27:29.884105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.925067 (+1040962us) service_pool.cc:224] Handling call
0709 12:27:30.934453 (+  9386us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:30.977006  6455 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.913102 12694 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:30.984912  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46617.7us
I20260709 12:27:30.985477  6381 raft_consensus.cc:2410] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:27:30.986078  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:30.987933  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:30.991446  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 105121us
I20260709 12:27:30.935611 12657 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:30.934893 12577 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:30.931069 12578 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.998411  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:30.925833 12624 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:27:31.014050  6455 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:30.927417 12612 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:30.936568 12117 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:30.977900 12619 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:30.938802 12636 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.934784 12669 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:31.011584  7317 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:31.070945  6455 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:31.009529  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:30.993412  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:30.983886 12693 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:30.983754 12726 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:31.006747 12577 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:31.033694 12695 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.347s	user 0.013s	sys 0.000s
I20260709 12:27:30.999120 12694 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.068521 12723 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:31.043273 12728 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.079653 12706 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:31.061442  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.005750 12749 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:31.000898 12585 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:27:31.072391  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3351) took 1690 ms. Trace:
I20260709 12:27:31.076990  6383 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:31.077404 12722 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:31.130043  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74283.8us
I20260709 12:27:31.044226 12506 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:31.077507 12669 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.043545 12657 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.015972 12578 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:31.091929  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 98640.8us
I20260709 12:27:31.069051 12724 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:31.092711  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 102813us
I20260709 12:27:31.095455 12762 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:31.096949  6379 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:31.098567 12577 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:31.100561 12774 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:31.155889  6305 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:31.156111  6300 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:31.104627  7319 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:31.109078 11996 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:31.110708  6382 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2745) took 2023 ms (client timeout 1661 ms). Trace:
I20260709 12:27:31.158967  7318 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 12:27:31.158687  6382 rpcz_store.cc:269] 0709 12:27:29.086976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.087098 (+   122us) service_pool.cc:224] Handling call
0709 12:27:31.110441 (+2023343us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b540023ebd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.110627 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1695752448}
I20260709 12:27:31.138221  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:31.110735  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3352) took 1728 ms. Trace:
I20260709 12:27:31.112268 12588 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:31.113271 12712 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:31.113902 12611 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:31.091912 12693 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.008312 12725 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.043875 12636 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.120864 12694 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.121394  6377 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:31.115686 12695 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.175715 12581 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.124464  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2922) took 1793 ms. Trace:
I20260709 12:27:31.126272  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3353) took 1743 ms. Trace:
I20260709 12:27:31.126384  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.118352  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:31.187904  7317 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:31.130527  6455 rpcz_store.cc:276] 0709 12:27:29.381878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.043683 (+1661805us) service_pool.cc:224] Handling call
0709 12:27:31.068969 (+ 25286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.072064 (+  3095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.072324 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.075007 12515 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:27:31.115954 12665 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:31.111370  6385 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:31.170171 12688 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.161154  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.160326  6451 rpcz_store.cc:276] 0709 12:27:29.382171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.109889 (+1727718us) service_pool.cc:224] Handling call
0709 12:27:31.110221 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.110408 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.110437 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.110507 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:31.110537 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.110583 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.110681 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.167332 12684 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.137671 12506 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.162978 12527 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.198748 12578 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.199072 12527 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.199538 12527 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.200443 12506 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.201638  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 3) took 2030 ms. Trace:
I20260709 12:27:31.160983 12762 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.163360  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.131677 12722 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.162766  6303 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:31.202102  6380 rpcz_store.cc:276] 0709 12:27:29.185703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.269308 (+ 83605us) service_pool.cc:224] Handling call
0709 12:27:31.201333 (+1932025us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b540023f5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ad735 00007f6fa826dd29 00007f6fa17ec1d4 00007f6fa17ec162 00007f6fa17ec091 00007f6fa17ec00c 00007f6fa17eb2c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.201525 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1587325824}
I20260709 12:27:31.165970  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.202198 12695 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.179673  7319 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:31.144721 12749 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:31.181427  7320 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:31.183424 12693 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.190122 12615 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.217716  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:31.101603 12726 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.124117  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.222020 12749 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.222913  7314 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:31.224607  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2903) took 2945 ms. Trace:
I20260709 12:27:31.224985  6453 rpcz_store.cc:276] 0709 12:27:28.279574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.623974 (+344400us) service_pool.cc:224] Handling call
0709 12:27:29.099894 (+475920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.100110 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.100135 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:29.100184 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:29.100221 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.100285 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.224427 (+2124142us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54002c8650. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.224535 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1087236480}
I20260709 12:27:31.160710 12774 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.185798  6308 pending_rounds.cc:85] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 7
I20260709 12:27:31.226191  6308 pending_rounds.cc:107] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:27:31.226557  6308 raft_consensus.cc:2889] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:31.130342 12728 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.187594  6452 rpcz_store.cc:276] 0709 12:27:29.382418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.125302 (+1742884us) service_pool.cc:224] Handling call
0709 12:27:31.125667 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.125923 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.125963 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.126012 (+    49us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:31.126045 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.126104 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.126215 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.155892  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:31.230634  7314 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:31.231235  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:31.144168 12657 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.166709 12712 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:31.192687  7317 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:27:31.207367 12585 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:31.194614  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:31.159373  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:31.240535  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 81232.9us
I20260709 12:27:31.194267  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:31.196563  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:31.197360  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:31.198278 12773 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
W20260709 12:27:31.153842 12382 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.698s	user 0.008s	sys 0.004s
I20260709 12:27:31.208325  6305 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 12:27:31.209336  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.210294  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.210443 12606 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:31.210578 12762 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.211632 12518 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:31.211710  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2752) took 2104 ms (client timeout 1881 ms). Trace:
W20260709 12:27:31.253960  6376 rpcz_store.cc:269] 0709 12:27:29.107669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.107828 (+   159us) service_pool.cc:224] Handling call
0709 12:27:31.211497 (+2103669us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b54003d0c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.211611 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":484768512}
W20260709 12:27:31.212735  7329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1148) took 1670 ms (client timeout 2003 ms). Trace:
I20260709 12:27:31.191576 12515 consensus_queue.cc:1243] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:31.156949 12760 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.220899  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.221263 12506 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.230340 12774 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.187021  6454 rpcz_store.cc:276] 0709 12:27:29.331268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.500408 (+1169140us) service_pool.cc:224] Handling call
0709 12:27:30.522946 (+ 22538us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:30.680283 (+157337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:30.680325 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:30.684463 (+  4138us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.808402 (+123939us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:30.808442 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.808518 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.124268 (+315750us) raft_consensus.cc:1685] finished
0709 12:27:31.124289 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.124405 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":320903,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":263342,"threads_started":2,"wal-append.queue_time_us":438022}
I20260709 12:27:31.237709  7317 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.169044 12783 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:31.184880 12725 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.238893  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.251605 12606 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:31.241946  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:31.271499  6456 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:31.222939 12423 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:31.242889  6307 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:31.247732  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.280054 12762 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:31.280618 12762 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.253657 12518 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:31.280982 12526 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.287329  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.288368  6383 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:31.287606 12606 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:31.265554 12591 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.853s	user 0.014s	sys 0.000s
I20260709 12:27:31.252233  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.290966  7335 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:31.224210 12693 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:31.255378  7329 rpcz_store.cc:269] 0709 12:27:29.542422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.542638 (+   216us) service_pool.cc:224] Handling call
0709 12:27:31.212536 (+1669898us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54007a0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.212671 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287609088}
I20260709 12:27:31.226243 12750 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:31.259904  6380 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:31.222316 12692 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:31.238412 12578 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:31.290259 12591 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:31.351500 12581 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.352246 12726 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.352140 12581 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.352785 12527 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.353614  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.266813 12506 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.353654 12608 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.355392 12527 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.356537 12506 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.355392 12693 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.357232 12577 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.281052 12762 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.248061 12627 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:31.263700  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.264266  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2822) took 2221 ms. Trace:
I20260709 12:27:31.365945  6304 rpcz_store.cc:276] 0709 12:27:29.042982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.043153 (+   171us) service_pool.cc:224] Handling call
0709 12:27:29.043562 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:29.115513 (+ 71951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.115548 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:29.115596 (+    48us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:29.115641 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.263980 (+2148339us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b54000942d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:31.264044 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.264196 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164963712}
I20260709 12:27:31.244503 12574 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.248392 12636 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.269461  6452 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:31.269836  7317 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:27:31.210752 12722 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:31.247638 12684 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:27:31.289407  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.374854  6383 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
W20260709 12:27:31.310465  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.289163 12606 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:31.344097  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.342353 12692 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:31.355979 12608 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.204854 12665 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.354332  6454 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:31.361104  6455 raft_consensus.cc:2368] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20.
W20260709 12:27:31.362159  6384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2764) took 2134 ms (client timeout 2408 ms). Trace:
I20260709 12:27:31.248283 12585 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:31.294447 12781 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:27:31.371268  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:31.388943  6457 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:31.354393 12750 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.373436  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.390554  6380 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:31.391539  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.385576 12627 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:31.270309 12783 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.291256 12712 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:31.376686  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.370411 12574 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:31.384325  6384 rpcz_store.cc:269] 0709 12:27:29.228201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.252558 (+ 24357us) service_pool.cc:224] Handling call
0709 12:27:31.362024 (+2109466us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400109ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.362111 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004639104}
I20260709 12:27:31.387588  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:31.388028  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:31.282058 12518 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.391999 12577 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:31.417752  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.398275  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:31.413053  6383 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:31.415688  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.419018  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.416661  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.418365  6451 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:31.395169  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:31.420274  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.424976  6454 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.424470  6303 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:31.425058  6457 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:31.423678  7317 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:31.383589 12636 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:31.393661 12785 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:27:31.431743 12703 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.432705  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.397993 12665 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.437814  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:31.439266  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.396413 12712 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:31.440126  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.446051  6305 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:31.439549  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.395764 12783 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.442337  6457 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:31.435953  6303 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:31.443249  7317 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:31.418026 12574 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.444851  6301 pending_rounds.cc:85] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 11
I20260709 12:27:31.455803  6301 pending_rounds.cc:107] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:27:31.448074  6384 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:31.428301 12518 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:31.456213  6301 raft_consensus.cc:2889] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:31.457746  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.392489 12684 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:27:31.458832  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.434913 12636 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:31.464977  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 30efe8bf06814d23bf20818fb23139be: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:27:31.420456 12515 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:27:31.467435  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:31.468688  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.473829  7317 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:31.483000  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.483681  6384 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:31.486889  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.452198 12703 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.392966 12692 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.492745  7329 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:27:31.491912  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:31.494668  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.494876  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3354) took 2112 ms. Trace:
I20260709 12:27:31.495361  6383 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:31.495292  6451 rpcz_store.cc:276] 0709 12:27:29.382769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.482692 (+2099923us) service_pool.cc:224] Handling call
0709 12:27:31.494368 (+ 11676us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.494582 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.494610 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.494648 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:31.494682 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.494728 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.494825 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.497707  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:31.498374  6384 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
W20260709 12:27:31.488773 12686 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.004s
W20260709 12:27:31.499699  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.482385 12624 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.484669 12657 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.500672  6457 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:31.394171 12627 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.503103 12578 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:31.503649 12581 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.511966  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:31.507108 12636 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.492178 12591 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:27:31.507670 12728 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.521884  6384 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:31.507993 12750 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:31.508968 12624 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.510246  6451 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:31.526648  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:31.528936  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2925) took 2000 ms. Trace:
I20260709 12:27:31.529443  6451 rpcz_store.cc:276] 0709 12:27:29.528104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.509727 (+1981623us) service_pool.cc:224] Handling call
0709 12:27:31.510086 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.528551 (+ 18465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.528846 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.483228 12783 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.511864  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2924) took 1984 ms. Trace:
I20260709 12:27:31.531497  6451 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:31.513360 12725 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.532626  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2931) took 1552 ms. Trace:
I20260709 12:27:31.515863 12692 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:31.517668 12703 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:31.506461 12526 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:31.522786  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3056) took 2131 ms. Trace:
I20260709 12:27:31.509292 12786 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:31.538233  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:31.515666  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:31.533172  6451 rpcz_store.cc:276] 0709 12:27:29.980484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.530637 (+1550153us) service_pool.cc:224] Handling call
0709 12:27:31.531039 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.532396 (+  1357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.532574 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.532075  6457 rpcz_store.cc:276] 0709 12:27:29.527252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.492091 (+1964839us) service_pool.cc:224] Handling call
0709 12:27:31.498942 (+  6851us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.511590 (+ 12648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.511805 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.535764  6384 rpcz_store.cc:276] 0709 12:27:29.390956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.500873 (+2109917us) service_pool.cc:224] Handling call
0709 12:27:31.521613 (+ 20740us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.522410 (+   797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.522711 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.539487 12728 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:31.524690 12588 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:31.544368 12789 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:27:31.546967  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.549538 12728 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.512506 12686 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.511581 12782 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:27:31.550931 12728 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.549396 12160 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:27:31.555573  6384 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:31.555673 12160 consensus_queue.cc:1243] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:27:31.556349 12160 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:27:31.556710 12160 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:31.556360  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3057) took 2165 ms. Trace:
I20260709 12:27:31.557657  6384 rpcz_store.cc:276] 0709 12:27:29.391246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.554999 (+2163753us) service_pool.cc:224] Handling call
0709 12:27:31.555363 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.556103 (+   740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.556301 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.546420 12724 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:31.549706 12588 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.536098 12783 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.567939  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3059) took 2168 ms. Trace:
I20260709 12:27:31.568392  6384 rpcz_store.cc:276] 0709 12:27:29.399371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.558646 (+2159275us) service_pool.cc:224] Handling call
0709 12:27:31.567491 (+  8845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.567662 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.567685 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.567726 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:31.567754 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.567800 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.567889 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.566195  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:31.574649  6454 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:31.577755  6451 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
W20260709 12:27:31.598292  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.546099 12786 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.563030 12724 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.664492  6384 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:31.665819  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3061) took 2239 ms. Trace:
I20260709 12:27:31.666226  6384 rpcz_store.cc:276] 0709 12:27:29.426759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.569310 (+2142551us) service_pool.cc:224] Handling call
0709 12:27:31.664317 (+ 95007us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.665534 (+  1217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.665753 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.567443 12588 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:31.576244 12765 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:31.692921  6383 raft_consensus.cc:606] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:27:31.676199 12790 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:27:31.660612 12606 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.696557  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3054) took 2358 ms. Trace:
I20260709 12:27:31.706132  6385 rpcz_store.cc:276] 0709 12:27:29.338292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.876602 (+538310us) service_pool.cc:224] Handling call
0709 12:27:31.111159 (+1234557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.191246 (+ 80087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.696216 (+504970us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540023ebd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.696457 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2838374272}
I20260709 12:27:31.708712  6385 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:31.702698  6383 raft_consensus.cc:614] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:31.703374  6454 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:31.711578 12783 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.712930  6454 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:31.713315 12786 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:31.714465 12778 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.603s	user 0.007s	sys 0.006s
W20260709 12:27:31.715045  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.715404  6385 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:31.669363 12724 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:31.724027  6385 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:31.725611  7319 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:31.729449 12693 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.733183 12750 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.733924 12750 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.565593 12762 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.720044  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.739787  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3062) took 2312 ms. Trace:
I20260709 12:27:31.717164  6454 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 10. Current term is 11. Ops: []
I20260709 12:27:31.741091  7335 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:31.659801 12160 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:31.740995  6384 rpcz_store.cc:276] 0709 12:27:29.427294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.696409 (+2269115us) service_pool.cc:224] Handling call
0709 12:27:31.739118 (+ 42709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.739336 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.739374 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:31.739431 (+    57us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:31.739517 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.739597 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.739719 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:31.679363 12722 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.742645 12160 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:27:31.722204 12774 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.766021  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2834) took 1696 ms. Trace:
W20260709 12:27:31.766422  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.741792 12765 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:31.768222  6309 rpcz_store.cc:276] 0709 12:27:30.069222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.069602 (+   380us) service_pool.cc:224] Handling call
0709 12:27:31.765819 (+1696217us) spinlock_profiling.cc:243] Waited 395 ms on lock 0x7b540010f754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.765947 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":907805056}
W20260709 12:27:31.778761  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.738405 12606 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:31.790319  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:31.798080 12776 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.766s	user 0.003s	sys 0.000s
W20260709 12:27:31.905755 12764 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.888s	user 0.000s	sys 0.005s
I20260709 12:27:31.924703  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.925652  6310 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:31.919014  6453 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:31.919510  6450 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:31.920045  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 929) took 1151 ms. Trace:
W20260709 12:27:31.927858  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.927690  7318 rpcz_store.cc:276] 0709 12:27:30.768565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.913341 (+144776us) service_pool.cc:224] Handling call
0709 12:27:31.158754 (+245413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.159543 (+   789us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.799992 (+640449us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b54005da2d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.919975 (+119983us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563348736}
I20260709 12:27:31.891377 12765 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.801667 12572 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:31.921634  6455 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:31.932212  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3071) took 1077 ms. Trace:
W20260709 12:27:31.922302  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.927069  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3053) took 2586 ms. Trace:
W20260709 12:27:31.933584  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3060) took 2533 ms (client timeout 1683 ms). Trace:
I20260709 12:27:31.933682  6379 rpcz_store.cc:276] 0709 12:27:29.337971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.796050 (+458079us) service_pool.cc:224] Handling call
0709 12:27:29.796335 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.152978 (+1356643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.815730 (+662752us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54003d2cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:31.924321 (+108591us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2990617856}
I20260709 12:27:31.930303  6308 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 12:27:31.929592  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 171163us
W20260709 12:27:31.929234  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.926397  7285 leader_election.cc:400] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20.
I20260709 12:27:31.938277  7285 leader_election.cc:403] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:31.938587  6381 rpcz_store.cc:276] 0709 12:27:30.854942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.889639 (+ 34697us) service_pool.cc:224] Handling call
0709 12:27:31.912877 (+1023238us) spinlock_profiling.cc:243] Waited 94.5 ms on lock 0x7b540023e6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.932134 (+ 19257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":217316096}
I20260709 12:27:31.940798  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:31.940740  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:31.934451  6378 rpcz_store.cc:269] 0709 12:27:29.399691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.544406 (+144715us) service_pool.cc:224] Handling call
0709 12:27:31.806814 (+2262408us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:31.933504 (+126690us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3141063936}
I20260709 12:27:31.942046  7319 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:31.941644  7335 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:31.943557  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.947234  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.948426  6459 raft_consensus.cc:2410] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:27:31.949666  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.950407  6459 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:31.951669  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.952504  6452 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:31.962282  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:31.941231 12318 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.970345  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.978978  7318 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:31.980176  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.969623 12728 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:31.925452 12627 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.977303  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:31.991616  6310 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:31.985493  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:31.974503  6378 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:31.992467  6378 pending_rounds.cc:85] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 8
I20260709 12:27:31.993088  6378 pending_rounds.cc:107] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:27:31.993462  6378 raft_consensus.cc:2889] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:31.993851 12762 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:31.995012  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:31.995668 12712 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.540s	user 0.012s	sys 0.000s
I20260709 12:27:31.996371  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:31.998039  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:31.945716 12572 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:31.999526  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54983.4us
I20260709 12:27:32.006055  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:32.001432  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:31.998847  6378 raft_consensus.cc:2376] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15. Re-sending same reply.
I20260709 12:27:31.966528 12700 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:31.962182 12506 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.962430 12749 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:31.969878 12791 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.267s	user 0.000s	sys 0.005s
I20260709 12:27:32.008073  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:31.996381 12712 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:27:32.004204  7318 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:32.001482  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.038416  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 265531us
W20260709 12:27:32.035740  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.043365  6303 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:32.044822  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:32.010675  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 71467.5us
I20260709 12:27:32.047209  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:32.029697  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:32.050213  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.050215  6383 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:32.052495  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:32.052591  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 46653.6us
I20260709 12:27:32.006171 12686 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.053558  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:32.009447 12657 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:32.060045  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.061225  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:32.070354  6302 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:32.062002  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:32.063740  6303 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:32.007073 12624 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:31.976537 12526 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:27:32.010411 12688 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.973739 12574 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.037765  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:32.100097  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62446.8us
I20260709 12:27:31.986340 12612 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.072916  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:32.073326 12572 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.095510 12615 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:32.073704  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:32.073933  6305 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:32.071746 12591 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:32.107836 12591 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.108389 12591 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:32.079595 12782 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:27:32.071756  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:32.114688 12581 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:32.114997 12614 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:32.115195 12581 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.115618 12581 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:32.116798  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45243.4us
I20260709 12:27:32.118621  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.118856  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:32.119416 12614 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:32.119877 12614 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.120227 12614 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:31.988752 12318 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:32.097777  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:32.028138 12756 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:27:32.006629 12627 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.105230  6302 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.106979 12615 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.074189 12684 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:32.056541 12748 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:32.098590 12767 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.025s	user 0.000s	sys 0.005s
I20260709 12:27:32.133621  6380 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:32.094579 12725 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:32.034428 12506 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.079973 12750 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:32.104682 12771 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:27:32.137972  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 852) took 2917 ms. Trace:
I20260709 12:27:32.129416 12722 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.165442  7316 rpcz_store.cc:276] 0709 12:27:29.220591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.221165 (+   574us) service_pool.cc:224] Handling call
0709 12:27:29.712986 (+491821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.717667 (+  4681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.717719 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.719082 (+  1363us) log.cc:844] Serialized 26 byte log entry
0709 12:27:30.320664 (+601582us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:30.320987 (+   323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:30.321540 (+   553us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.895119 (+573579us) raft_consensus.cc:1685] finished
0709 12:27:30.895141 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.137612 (+1242471us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54003bcc50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:32.137883 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4218,"raft.queue_time_us":80074,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11034,"spinlock_wait_cycles":1130358400,"thread_start_us":489556,"threads_started":1,"wal-append.queue_time_us":1057115,"wal-append.run_cpu_time_us":6346,"wal-append.run_wall_time_us":1078528}
I20260709 12:27:32.096021 12657 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.143244 12585 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:32.144650  7320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1151) took 1665 ms (client timeout 1780 ms). Trace:
I20260709 12:27:32.105247 12688 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:32.169533  7320 rpcz_store.cc:269] 0709 12:27:30.479604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.479841 (+   237us) service_pool.cc:224] Handling call
0709 12:27:32.144471 (+1664630us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b54005c1054. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:32.144582 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198632832}
I20260709 12:27:32.074334 12728 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.125659 12686 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:32.160156  6300 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:32.149967 12612 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:32.153354 12684 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.149117 12774 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.121938 12665 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 12:27:32.146268 12793 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:27:32.075127 12606 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:32.170955  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:32.175112  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1009) took 1049 ms. Trace:
W20260709 12:27:32.175297  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 206911us
I20260709 12:27:32.208202  6453 rpcz_store.cc:276] 0709 12:27:31.126174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.225767 (+ 99593us) service_pool.cc:224] Handling call
0709 12:27:32.174887 (+949120us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54000fc654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:32.175049 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167174528}
I20260709 12:27:32.161706 12588 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.174903 12686 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:32.213304  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2783) took 1163 ms. Trace:
I20260709 12:27:32.211787  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:32.192809  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 247759us
I20260709 12:27:32.199337 12684 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.218294  7317 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.222043  7316 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:27:32.228749  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.229357  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.229848  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.204846 12615 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:32.231875  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 160788us
I20260709 12:27:32.232056  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:32.232632  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:32.237846  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:32.212806 12608 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:32.246249  7317 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:32.262717 12615 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:32.263913 12615 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:32.264251 12615 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:32.265048 12615 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.271971  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2929) took 2306 ms. Trace:
I20260709 12:27:32.272428  6450 rpcz_store.cc:276] 0709 12:27:29.965310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.527433 (+1562123us) service_pool.cc:224] Handling call
0709 12:27:31.538209 (+ 10776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.946364 (+408155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.271696 (+325332us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54001c1954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:32.271903 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":557249920}
I20260709 12:27:32.276810 12615 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:32.274149  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.250365 12807 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:27:32.280055 12117 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:32.280069 12506 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:32.283921  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:32.284202  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.287705 12627 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:32.288497  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.288797  6454 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:27:32.289120  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.290071 12588 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:32.290843  6454 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:27:32.294030  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.294700 12686 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.296478 12686 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.297168  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.297958  6455 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:32.299335  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.287710  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.302551  7335 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
W20260709 12:27:32.303833  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47144.7us
I20260709 12:27:32.304673  6302 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:32.309942  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:32.310565  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 280990us
I20260709 12:27:32.311416  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:32.314157  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 198058us
I20260709 12:27:32.308831  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:32.336699  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:32.337355  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.320174  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.339236  6455 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:32.338119  6450 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:32.255442 12160 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:32.324335  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:32.300199 12738 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.225s	user 0.005s	sys 0.021s
I20260709 12:27:32.253036 12808 raft_consensus.cc:2764] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:32.345484  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:32.374213  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:32.403945  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.493606  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.507995  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.538307  6378 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:27:32.538904  6378 pending_rounds.cc:85] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:27:32.539207  6378 pending_rounds.cc:107] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:27:32.539503  6378 raft_consensus.cc:2889] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:32.540925  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:32.340910 12588 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:32.560683  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 430995us
I20260709 12:27:32.310395 12608 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:27:32.347640 12694 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:32.347952 12160 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:32.349407 12738 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.278s	user 0.005s	sys 0.021s
I20260709 12:27:32.569079  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:32.578768  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 407868us
I20260709 12:27:32.612216 12807 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.612802 12807 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.345081 12712 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.614862 12712 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.615430 12807 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:32.615985 12807 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.616405 12807 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.617126 12712 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:32.617677 12712 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:32.588482  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 467382us
I20260709 12:27:32.618414 12807 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:32.345965 12801 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:32.620041 12801 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.622061 12801 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:32.336052 12574 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.355003 12627 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:32.574710 12700 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.532s	user 0.006s	sys 0.006s
I20260709 12:27:32.440615 12722 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.626924  7332 raft_consensus.cc:2410] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 12:27:32.627159  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.349272 12728 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.241s
I20260709 12:27:32.317056 12591 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.620010  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:32.645684  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.659142  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:32.595456 12694 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.579555 12588 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.579974 12765 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:27:32.652508  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 328253us
W20260709 12:27:32.605342 12804 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 12:27:32.660863  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.653328 12578 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.622738 12627 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.618103 12712 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.598300 12160 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.644214 12117 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:32.667389 12574 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.691037  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:32.691179  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:32.692139  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:32.644541 12706 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.649914  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.676229  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.674423 12657 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.666891 12694 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.667022 12774 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:32.703399 12774 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:32.704030 12774 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:32.685509 12117 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:32.707239 12117 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.695271 12574 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.709906 12574 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:32.710248 12612 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.510s	user 0.007s	sys 0.000s
W20260709 12:27:32.685961  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66906.6us
I20260709 12:27:32.670698 12588 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:32.720712  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.671192  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:32.688089  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.694391  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:32.725714  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:32.663720 12506 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:32.657462 12722 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.695382  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.675278 12688 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.735608  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:32.736469  7316 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:32.698267 12830 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:32.699156 12822 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.671979 12807 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:32.702831 12591 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.671918  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2784) took 1621 ms. Trace:
W20260709 12:27:32.728050 12755 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.712s	user 0.001s	sys 0.012s
I20260709 12:27:32.685001 12803 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:32.729076  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.732499  6378 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:32.730312  7314 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:32.742061  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:32.744016  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 382399us
I20260709 12:27:32.762739  7320 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.758208 12813 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:32.745469  6382 rpcz_store.cc:276] 0709 12:27:31.050976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.920680 (+869704us) service_pool.cc:224] Handling call
0709 12:27:31.960605 (+ 39925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.960882 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:32.223141 (+262259us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.671242 (+448101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:32.671359 (+   117us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:32.671434 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.671557 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.671785 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9681,"prepare.queue_time_us":28864,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":4913,"raft.queue_time_us":145466,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":8250,"thread_start_us":678323,"threads_started":3,"wal-append.queue_time_us":496479}
W20260709 12:27:32.765144  6425 leader_election.cc:336] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.589s (SENT)
I20260709 12:27:32.730435 12657 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:32.740741 12749 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.000s
W20260709 12:27:32.740999 12614 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.620s	user 0.012s	sys 0.001s
I20260709 12:27:32.767773  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3358) took 1849 ms. Trace:
I20260709 12:27:32.730365 12506 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.768611  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.774052  6378 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:27:32.760159 12811 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:27:32.719975 12817 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:27:32.764655 12813 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:32.771189  6451 rpcz_store.cc:276] 0709 12:27:30.918259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.546471 (+628212us) service_pool.cc:224] Handling call
0709 12:27:31.575145 (+ 28674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.662504 (+ 87359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.767480 (+1104976us) spinlock_profiling.cc:243] Waited 25.5 ms on lock 0x7b54002c8b50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:32.767696 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58654080}
I20260709 12:27:32.697563 12706 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.775972  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.776302  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:32.770428  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46586.1us
W20260709 12:27:32.771644  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 192524us
I20260709 12:27:32.761343 12803 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:32.789299  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.775578 12506 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:32.794359  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3069) took 1940 ms. Trace:
I20260709 12:27:32.775060 12657 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:32.740018 12665 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:27:32.773487 12749 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20.
I20260709 12:27:32.792411 12627 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:32.794121  7320 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:32.785645  6382 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:32.795017  6385 rpcz_store.cc:276] 0709 12:27:30.854115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.725199 (+871084us) service_pool.cc:224] Handling call
0709 12:27:31.725614 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.751328 (+ 25714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.751366 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.755268 (+  3902us) log.cc:844] Serialized 26 byte log entry
0709 12:27:32.222390 (+467122us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:32.245802 (+ 23412us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.246115 (+   313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.246211 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.794138 (+547927us) raft_consensus.cc:1685] finished
0709 12:27:32.794161 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.794295 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":237326,"prepare.queue_time_us":24646,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":27830,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":12609,"spinlock_wait_cycles":12104960,"thread_start_us":258789,"threads_started":1,"wal-append.queue_time_us":574115}
I20260709 12:27:32.759119 12822 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.805477  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.810226  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:32.809044  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114637us
I20260709 12:27:32.812211  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.795090 12803 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:32.825311 12812 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:27:32.826932 12627 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:27:32.827876 12821 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:27:32.828166 12160 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.830626 12588 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.833108 12588 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:32.836585 12722 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.837491 12722 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:32.787935 12813 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.775295 12614 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.768031 12807 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:32.841166  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.842432  6451 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:32.845695  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2785) took 1793 ms. Trace:
W20260709 12:27:32.853706  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67396.6us
I20260709 12:27:32.852833  6379 rpcz_store.cc:276] 0709 12:27:31.052223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.937539 (+885316us) service_pool.cc:224] Handling call
0709 12:27:31.937978 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:32.052475 (+114497us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.052573 (+    98us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:32.078924 (+ 26351us) log.cc:844] Serialized 52 byte log entry
0709 12:27:32.231659 (+152735us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:32.231723 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.231824 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.845392 (+613568us) raft_consensus.cc:1685] finished
0709 12:27:32.845422 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.845621 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113821,"raft.queue_time_us":113964,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":12666,"thread_start_us":152236,"threads_started":1,"wal-append.queue_time_us":527789}
I20260709 12:27:32.860390  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:32.863648  6379 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:32.863849  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:32.863986  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.864544  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:32.867108  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:32.867292  7319 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:32.864472  6304 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:32.865198  6379 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:32.862857  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:32.874434  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89580.4us
I20260709 12:27:32.877605  6302 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:32.849392 12722 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:32.851200 12833 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:32.853443 12822 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.882081  6379 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:32.880381  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:32.882200  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.888392 12748 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.734s	user 0.010s	sys 0.000s
I20260709 12:27:32.888868 12748 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.889768 12572 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:32.891117 12722 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.894721 12839 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:32.894384 12833 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.895259 12839 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.896889 12833 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.897267 12839 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:32.897929 12833 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:32.898366 12822 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:32.897758 12839 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:32.899171 12703 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.899669 12703 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.901721 12703 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:32.902921  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 90789.2us
I20260709 12:27:32.904021  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:32.902289 12615 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.000s
I20260709 12:27:32.879824 12843 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:32.919781 12686 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.623s	user 0.006s	sys 0.008s
I20260709 12:27:32.864953 12749 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:27:32.921792  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:32.885777 12712 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.935684 12807 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:32.937073 12614 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:32.938505 12824 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:27:32.938831  6308 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:32.216164  6377 rpcz_store.cc:276] 0709 12:27:31.049654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.919424 (+869770us) service_pool.cc:224] Handling call
0709 12:27:31.919766 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.934648 (+ 14882us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:31.987717 (+ 53069us) log.cc:844] Serialized 14 byte log entry
0709 12:27:32.212954 (+225237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:32.213008 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:32.213058 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.213130 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.213246 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2104,"prepare.queue_time_us":35883,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":46,"raft.queue_time_us":522,"thread_start_us":224313,"threads_started":1}
I20260709 12:27:32.902796 12657 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:32.921108 12817 consensus_queue.cc:1243] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:27:32.923804 12835 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:32.932806 12684 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.001s
W20260709 12:27:32.960187  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97395.5us
I20260709 12:27:32.965806  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:32.966400  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44715.1us
I20260709 12:27:32.928366 12665 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:27:32.929867 12843 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.968930 12846 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:32.954617 12842 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.951264 12686 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:32.954221 12835 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:32.960024 12712 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:32.973649  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.973941 12835 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:32.973126 12842 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.965489 12684 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:32.968953 12724 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.246s	user 0.013s	sys 0.002s
W20260709 12:27:32.981870  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.975497 12712 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.984129  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:32.984688 12712 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:32.985174  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:32.990619 12817 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.991495 12817 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:32.978847 12842 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:32.992512 12846 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:32.975557 12686 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:32.992923 12846 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.993287 12846 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:32.994755 12845 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:32.991941 12817 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.996001 12817 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:32.996706 12725 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.997282 12725 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:32.981297 12684 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:33.000564 12581 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.884s	user 0.010s	sys 0.003s
I20260709 12:27:33.001070 12581 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:33.002199  6380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2771) took 2066 ms (client timeout 2205 ms). Trace:
W20260709 12:27:33.002808  6380 rpcz_store.cc:269] 0709 12:27:30.935754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.391235 (+455481us) service_pool.cc:224] Handling call
0709 12:27:33.001914 (+1610679us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:33.002116 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1704792448}
I20260709 12:27:33.004230 12724 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.006759  6378 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18.
I20260709 12:27:33.006932  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1155) took 2157 ms. Trace:
I20260709 12:27:33.007292  7332 rpcz_store.cc:276] 0709 12:27:30.849262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.879943 (+ 30681us) service_pool.cc:224] Handling call
0709 12:27:33.006690 (+2126747us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b5400565754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:33.006849 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4014451968}
I20260709 12:27:32.942282 12803 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:33.023613  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 119680us
I20260709 12:27:33.026016  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.027386  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.016359 12724 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.032534  6304 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:32.972376 12813 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.016611 12686 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.024627  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.033572  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.016005 12665 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:27:33.050376  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.050396 12848 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.061116  6300 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:33.059743  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.062928  6301 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:27:33.065212  6307 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:33.069370 12694 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.071687 12694 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.073362  6385 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17.
I20260709 12:27:33.051662 12848 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:33.080214  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.048795 12506 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.080206  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.088132 12813 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:33.088421 12806 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.490s	user 0.002s	sys 0.002s
I20260709 12:27:33.086882 12835 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.094767  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.088609 12581 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.101745 12835 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.078017 12848 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.103993 12318 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.083505 12843 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.106421  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2781) took 2059 ms. Trace:
I20260709 12:27:33.108351  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.107829  6384 rpcz_store.cc:276] 0709 12:27:31.047478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.769514 (+722036us) service_pool.cc:224] Handling call
0709 12:27:31.923796 (+154282us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:31.979193 (+ 55397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.979244 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:31.980209 (+   965us) log.cc:844] Serialized 52 byte log entry
0709 12:27:33.005351 (+1025142us) spinlock_profiling.cc:243] Waited 9.66 ms on lock 0x7b1800107e60. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:33.035770 (+ 30419us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:33.035804 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.035854 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.106095 (+ 70241us) raft_consensus.cc:1685] finished
0709 12:27:33.106119 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.106356 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80833,"raft.queue_time_us":61092,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":15392,"spinlock_wait_cycles":22209280,"thread_start_us":30169,"threads_started":1,"wal-append.queue_time_us":1089774}
I20260709 12:27:33.109262  6458 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 11.
I20260709 12:27:33.114045  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.112407  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:33.092595 12786 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.018s	sys 0.031s
I20260709 12:27:33.116251  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.118902  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.127149 12774 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.129908 12774 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.130599 12618 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:33.130770  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2779) took 2192 ms. Trace:
I20260709 12:27:33.132344 12618 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:33.131114  6376 rpcz_store.cc:276] 0709 12:27:30.937883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.727138 (+789255us) service_pool.cc:224] Handling call
0709 12:27:31.727464 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:31.753714 (+ 26250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.753749 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:31.759889 (+  6140us) log.cc:844] Serialized 52 byte log entry
0709 12:27:32.152829 (+392940us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b180041b480. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:32.503846 (+351017us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:32.651679 (+147833us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.129864 (+478185us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b180041b480. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:27:33.130236 (+   372us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.130418 (+   182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.130542 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.130583 (+    41us) raft_consensus.cc:1685] finished
0709 12:27:33.130594 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.130717 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":366499,"prepare.queue_time_us":153266,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"raft.queue_time_us":29715,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":9617,"spinlock_wait_cycles":21144704,"thread_start_us":350709,"threads_started":1,"wal-append.queue_time_us":942000}
I20260709 12:27:33.132740 12618 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:33.133576 12618 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.134970 12618 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.137552 12160 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:33.141927  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.117396 12786 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.043s	user 0.018s	sys 0.031s
I20260709 12:27:33.146231  6305 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:33.115020 12803 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.204052 12688 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.178479 12700 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.158066  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:33.215804  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.227197  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49019.4us
I20260709 12:27:33.204632  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1003) took 3862 ms. Trace:
W20260709 12:27:33.182114  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 511034us
I20260709 12:27:33.232118  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.233256  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:33.231765  6456 rpcz_store.cc:276] 0709 12:27:29.342459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.905887 (+1563428us) service_pool.cc:224] Handling call
0709 12:27:30.906269 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.271928 (+365659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.204378 (+1932450us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b54000eb354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:33.204559 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839419264}
I20260709 12:27:33.235013  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.235229  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.179349 12728 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.236578  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.175132  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.240424  6303 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:33.119808 12318 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.242668  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.214648 12591 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:33.247566 12827 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.228048 12688 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.253147  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.180966 12829 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:33.257225  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 118985us
I20260709 12:27:33.257382  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:33.255472  6301 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:33.258975  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.243670  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.253701  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.260643  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.260839  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.261299  6301 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:33.236692 12728 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.261719  6300 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:33.262115  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:33.263057  6274 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.823s (SENT)
I20260709 12:27:33.130043 12848 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.242921 12318 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.263934  6383 raft_consensus.cc:606] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.269658  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.263111  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.250911 12810 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.248189 12827 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.241060 12700 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:33.268085  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.268731  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.277987 12848 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.292022  6300 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:33.271458  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:33.297070  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.297766  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:33.298830  7320 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:33.266209  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:33.260658 12694 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:33.242040 12846 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:33.277977 12849 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:27:33.304672  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.303797  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.281327 12318 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.269405 12809 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.303862  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.306385  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.271505 12786 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:33.269850 12765 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:33.306104  6274 leader_election.cc:336] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.096s (SENT)
I20260709 12:27:33.308262  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.271692 12693 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:27:33.312006  6301 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:33.312939 12506 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.314249  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:33.315016  6309 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:33.303771 12830 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:33.271759 12608 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
W20260709 12:27:33.318005  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.261802 12829 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.310185 12574 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:33.346685  6459 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:33.317274  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:33.349534  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 955) took 1308 ms. Trace:
I20260709 12:27:33.349910  6458 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:33.351346 12827 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.351541  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:33.352497 12859 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:27:33.319815  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.353293  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.353627  6301 raft_consensus.cc:2410] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:33.353894  6452 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:33.354458  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.355489  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.356159  6301 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:33.350020  6310 rpcz_store.cc:276] 0709 12:27:32.040984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.045911 (+  4927us) service_pool.cc:224] Handling call
0709 12:27:32.069964 (+ 24053us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.187631 (+117667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.187684 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.201913 (+ 14229us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.052069 (+850156us) spinlock_profiling.cc:243] Waited 883 us on lock 0x7b1800369a60. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:33.059665 (+  7596us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:33.059706 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.349137 (+289431us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54002067d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:33.349204 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.349263 (+    59us) raft_consensus.cc:1685] finished
0709 12:27:33.349281 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.349458 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28354,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10634,"spinlock_wait_cycles":271434880,"thread_start_us":7393,"threads_started":1,"wal-append.queue_time_us":896690}
I20260709 12:27:33.358026  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.318958  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.359091  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.310859 12725 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.323697  7317 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:33.360504 12834 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:33.360935 12834 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:33.361097 12725 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:33.355347  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.363108 12706 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.324505 12728 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.363485  6310 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:33.364392  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.326992 12765 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.324908 12688 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:33.328178 12657 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:33.313407 12786 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.332672  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 859) took 2343 ms. Trace:
I20260709 12:27:33.366851  7314 rpcz_store.cc:276] 0709 12:27:30.989557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.231653 (+242096us) service_pool.cc:224] Handling call
0709 12:27:31.232013 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.762176 (+1530163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.332435 (+570259us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54005c1a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:33.332616 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3441171200}
I20260709 12:27:33.334466  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:33.314026 12851 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:27:33.335206  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.352437  6458 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:33.359148  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.315300 12694 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:27:33.376480  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.319136  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.376977  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.364554  6301 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:33.378027  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.378731  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.378605  6303 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:33.368923  6450 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:33.379602  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:33.379594 12578 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:33.368948  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:33.322862  6274 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.231s (SENT)
I20260709 12:27:33.372185  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.381348  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.381734  6302 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:27:33.382107  6302 pending_rounds.cc:85] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 14
I20260709 12:27:33.382225  6381 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:27:33.382545  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.327677  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.383409  6456 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:27:33.384228 12794 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.374s	user 0.010s	sys 0.007s
I20260709 12:27:33.373780  6453 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:33.382356  6302 pending_rounds.cc:107] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:27:33.385028  6453 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:33.385214  6302 raft_consensus.cc:2889] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:33.366796 12786 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.388717  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:33.348356  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.389643  6459 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
W20260709 12:27:33.390168  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 956) took 1348 ms (client timeout 1646 ms). Trace:
W20260709 12:27:33.390587  6308 rpcz_store.cc:269] 0709 12:27:32.041362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.042584 (+  1222us) service_pool.cc:224] Handling call
0709 12:27:33.390007 (+1347423us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b54000a9e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:33.390115 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1926137088}
I20260709 12:27:33.391456  6304 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:33.382230 12694 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.386098  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68942.2us
I20260709 12:27:33.392994  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.394510  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.394870  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:33.395347  6381 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:33.295542 12810 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.397774  6301 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:33.361809 12834 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.387602  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.399253  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.399434  6381 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:33.375638  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:33.400013  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.414330  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.415294  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:33.396982  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74153.4us
I20260709 12:27:33.314606 12712 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.314174 12809 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.434018  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:33.373731  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 89522.2us
I20260709 12:27:33.420280  6459 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:33.399776  6303 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:33.416210  6379 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.401803  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:33.396847  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:33.438462  6458 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:33.440362  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.426208 12827 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:27:33.440896 12506 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:33.388348 12803 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:27:33.441524  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.441756 12826 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:33.414438 12578 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:33.442543 12712 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.442108  6380 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.376364 12774 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:27:33.409108 12694 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.444368 12774 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:33.444772 12774 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:33.444957  6376 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:33.431020  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.445562 12826 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.446159 12826 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:33.445621 12774 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.416880  6301 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:33.445163 12827 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:33.442070 12803 consensus_queue.cc:1243] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:33.417269  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:33.450361  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.402261 12822 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:33.385946 12756 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:27:33.437309  7287 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.613s (SENT)
I20260709 12:27:33.451839  6308 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:33.480351  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.355259 12817 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.460114  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.464705  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58396 (request call id 45) took 1431 ms. Trace:
I20260709 12:27:33.413887 12688 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:33.466606  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.467159  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3080) took 1089 ms. Trace:
I20260709 12:27:33.397293 12848 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:33.495321  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.474674  6380 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:33.482147  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:33.482434  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45182.2us
I20260709 12:27:33.473879  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.496398  6456 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:33.496635  6459 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:33.435129  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.484503  6383 rpcz_store.cc:276] 0709 12:27:32.033093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.051913 (+ 18820us) service_pool.cc:224] Handling call
0709 12:27:33.464515 (+1412602us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003d2cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:33.464608 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659995008}
I20260709 12:27:33.484318  6381 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.480050  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:33.466622 12798 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.394s	user 0.008s	sys 0.003s
I20260709 12:27:33.491983  6385 rpcz_store.cc:276] 0709 12:27:32.377686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.809882 (+432196us) service_pool.cc:224] Handling call
0709 12:27:33.466427 (+656545us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b540022f454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:33.467082 (+   655us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602169728}
I20260709 12:27:33.475374  6458 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:33.415673 12686 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.075s
W20260709 12:27:33.495448  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61919.9us
I20260709 12:27:33.497326  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:33.497638  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.483250 12756 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:33.500586  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.509138  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.467031 12700 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.506525  6380 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.499177  6378 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.458405 12827 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:33.505357  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.489538 12734 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:33.484732 12703 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:33.548276  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.490396 12688 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:33.516498  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.593039  6381 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.581768  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.600605  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.603819  6380 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:33.604506  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.604641  6383 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:27:33.494910 12774 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.495127 12608 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:33.514672  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:33.613026  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 98454.6us
I20260709 12:27:33.614351  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:33.588366  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75429.4us
I20260709 12:27:33.503192 12803 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:33.588083  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 82806.3us
I20260709 12:27:33.503305 12712 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:33.500590 12706 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:33.605276  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.623843  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.624467  6380 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:33.606103  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.626370  7288 connection.cc:256] RPC call timeout handler was delayed by -1.2358491189997949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1646s for a response
I20260709 12:27:33.603031  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:33.628392  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:33.607899  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.630358  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.631289  6383 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:33.610041  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.517004 12830 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:33.582904 12835 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:27:33.619267  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.503901 12627 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.096s
I20260709 12:27:33.584350 12700 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:33.607333 12688 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.585050 12725 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 12:27:33.621695  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 124438us
I20260709 12:27:33.662961  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 870) took 2101 ms. Trace:
I20260709 12:27:33.663144  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:33.636629  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.625689 12876 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:33.583228 12848 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:33.641252  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.655215  6454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:33.679692  6454 raft_consensus.cc:606] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.680259  6454 raft_consensus.cc:614] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:33.679190  6383 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:33.663597  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:33.634382  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.666220  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.621168 12850 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:33.670746 12606 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:27:33.749086  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47222.3us
I20260709 12:27:33.673525 12318 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.674602 12614 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:33.637346 12827 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:33.675202 12797 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.602s	user 0.001s	sys 0.007s
I20260709 12:27:33.645957 12700 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.769802 12827 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:33.770726 12700 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:33.773947 12863 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:27:33.775008  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1008) took 2649 ms. Trace:
I20260709 12:27:33.682551 12876 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:33.769883 12871 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:27:33.676055 12591 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:33.607093 12703 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.620891 12608 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.647326 12762 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:33.645749 12845 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:33.810683  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 110230us
I20260709 12:27:33.647640 12813 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:33.652235 12692 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.663702  7329 rpcz_store.cc:276] 0709 12:27:31.561198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.720844 (+159646us) service_pool.cc:224] Handling call
0709 12:27:31.721189 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:31.721535 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.721563 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:31.722172 (+   609us) log.cc:844] Serialized 52 byte log entry
0709 12:27:31.895684 (+173512us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:31.895752 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.895856 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.465942 (+1570086us) spinlock_profiling.cc:243] Waited 23.4 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:33.658072 (+192130us) raft_consensus.cc:1685] finished
0709 12:27:33.658103 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.662867 (+  4764us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19320,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":3465,"spinlock_wait_cycles":53767552,"thread_start_us":166897,"threads_started":1,"wal-append.queue_time_us":308477}
I20260709 12:27:33.681927  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.823892  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.825312  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.826198  6383 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.677466 12840 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:33.691290  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:27:33.829457  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 149391us
I20260709 12:27:33.679160 12734 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.680752 12657 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:33.679304 12846 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:33.699630  7319 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 933) took 1770 ms (client timeout 1755 ms). Trace:
W20260709 12:27:33.699774  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 65473.4us
W20260709 12:27:33.700953  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.687475 12829 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.715397  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:33.732829  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:33.738881  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.759526 12817 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:33.761169 12868 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:27:33.828121 12845 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.638823 12830 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.768846 12810 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.637984 12706 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:33.779507  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78645.4us
I20260709 12:27:33.752104 12850 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.778746 12506 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:33.685004 12756 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:33.820863  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 192569us
I20260709 12:27:33.678804 12848 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.898772 12848 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.899660 12608 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.900388  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.901073  6376 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:33.901937 12826 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.888979 12850 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.904284 12826 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.905090 12694 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.880731 12877 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:33.905804 12686 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:33.906127 12877 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:33.906638 12877 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.906951 12877 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.910671 12808 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.911350 12808 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.912693  6453 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.913745  6453 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:33.913698 12808 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.914582  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.915246  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.916303  6452 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:33.917163  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.917725  6452 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:33.919833 12636 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.918617  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:33.922281 12870 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.246s	user 0.001s	sys 0.000s
I20260709 12:27:33.922866 12756 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.924878  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.832705  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:33.926122  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 93543.7us
I20260709 12:27:33.831943  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:33.855777  7319 rpcz_store.cc:269] 0709 12:27:31.935162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.962015 (+ 26853us) service_pool.cc:224] Handling call
0709 12:27:33.699432 (+1737417us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:33.699555 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018651520}
I20260709 12:27:33.859793  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.861102  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.927558  6455 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:33.786458 12591 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:33.928484  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.930307  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:33.930709  6455 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:33.930883  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:33.931540  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.931910  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:33.856246  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:33.933156  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76989.1us
I20260709 12:27:33.860345  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.775436  6457 rpcz_store.cc:276] 0709 12:27:31.125871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.558893 (+433022us) service_pool.cc:224] Handling call
0709 12:27:31.568495 (+  9602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.568969 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.569015 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.576939 (+  7924us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.927573 (+350634us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:31.927645 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.927756 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.459648 (+1531892us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:33.774573 (+314925us) raft_consensus.cc:1685] finished
0709 12:27:33.774599 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.774918 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1856,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":86063,"spinlock_wait_cycles":19117312,"thread_start_us":233801,"threads_started":1,"wal-append.queue_time_us":407231}
I20260709 12:27:33.882366  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:33.870334  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:33.868580 12871 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:27:33.856384 12692 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.931552  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:33.929534  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.934953  6456 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:33.864931  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.939785  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.939935  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.935338 12318 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.941094 12318 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:33.941571 12318 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:33.933272  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:33.915236 12803 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.942585 12318 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.881824 12830 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:33.857357 12734 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:33.943711  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61476.6us
I20260709 12:27:33.919863 12876 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.944900 12876 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:33.945581 12734 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:33.945730 12813 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:33.946167  6378 raft_consensus.cc:2410] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:27:33.930954  7318 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:33.947881  7314 raft_consensus.cc:2376] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18. Re-sending same reply.
I20260709 12:27:33.949007  6450 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:33.881745 12845 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:33.936753  7329 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:27:33.952039  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60455.2us
I20260709 12:27:33.952605 12834 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.953161  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.895490 12839 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:33.954146 12839 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:33.855443 12657 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.954824 12657 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:33.955122 12839 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.956486 12839 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:33.938895  6380 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.958168 12703 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.943261 12803 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:33.942955  6452 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:33.958699 12828 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.889416 12506 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.992385 12506 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:33.959345 12877 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:33.894847 12872 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:27:33.897022 12706 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:33.947557  6457 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:27:33.898213 12866 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.241s	user 0.003s	sys 0.000s
I20260709 12:27:33.963532  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:34.021777  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:34.023306  6380 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.969914 12712 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:33.972257  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:33.935643  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:33.915398  6382 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.932349  6455 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:33.926750 12688 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.983608  6376 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:34.032256  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:33.987222  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.033212  7332 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:34.034294  6379 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:33.987793  6453 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:33.988457  6381 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:33.988751  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:34.036104  7329 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.036451  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.032286  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:34.037132  6381 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:33.989831  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:33.991526 12828 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:33.999828  7314 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:34.019928 12877 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.022109  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.041811  7316 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:34.041666  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:33.855065 12627 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:27:34.027384  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55196.5us
W20260709 12:27:34.028220  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 92660.3us
I20260709 12:27:34.047925  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:34.049769  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.050689  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:34.050711  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:34.053475  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.054826  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.030721  6456 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:34.037303  6380 raft_consensus.cc:2368] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20.
I20260709 12:27:34.056082  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:34.056504  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.056836  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.060953  6384 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:34.062008  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.038235  6383 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:33.919076 12117 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.025606 12636 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:34.038763  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.039048  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.040655  7319 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:34.045092  7314 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:34.058296  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.064525  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.079311  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:34.065546  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:34.049098 12423 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
W20260709 12:27:34.038836  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 906fbea0458f4a28bfad3b9dde8c6b35: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:34.071292  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:34.092504  6383 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:34.061677  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.073009  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:34.073570  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.075434  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:34.075688  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.032724 12809 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.063894  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:34.134402  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:34.135296  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71486.3us
W20260709 12:27:34.136994  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71563.9us
I20260709 12:27:34.081090  6385 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:34.039152 12813 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:34.039553 12835 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:34.070812 12694 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.046334 12627 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:34.083989 12830 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.087779 12850 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:34.133692 12786 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.001s
I20260709 12:27:34.138995  7287 leader_election.cc:400] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20.
W20260709 12:27:34.143245  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 331421us
I20260709 12:27:34.046552 12712 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:34.155210  6380 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:34.030692 12827 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 12:27:34.187222  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:34.025344 12700 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.161300  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.079635 12774 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:34.033859 12578 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:27:34.175987  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.175220 12886 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:34.088979  6300 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:34.195636 12786 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.207208  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.207548  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:34.195173 12822 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.000s
I20260709 12:27:34.208163  6382 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:34.184567  7329 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
W20260709 12:27:34.187156  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48330.2us
I20260709 12:27:34.128654  7332 raft_consensus.cc:2376] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20. Re-sending same reply.
I20260709 12:27:34.081632 12686 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.098s
I20260709 12:27:34.156718 12813 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.167244 12693 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:34.144057 12803 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.156256 12692 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.201750  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.203639 12712 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.176708 12627 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:34.160197 12835 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:34.174055  6383 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:34.212466  7320 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:34.198546 12774 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.155897 12848 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.213516  6451 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:34.213753  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:34.217831  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.024s	sys 0.040s
W20260709 12:27:34.241225  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.024s	sys 0.040s
W20260709 12:27:34.241564  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 578016us
I20260709 12:27:34.222360  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:34.229843  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 162364us
I20260709 12:27:34.227495  6202 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 14 to 15, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 15 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.230340  6454 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:27:34.239167  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.249359  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:34.239728  6382 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:34.245278  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:34.264030  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:34.269392  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.267777  6376 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:34.264559  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:34.260675  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:34.270953  6305 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:34.254832  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:34.218711  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:34.226725  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:34.279009  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:34.279984  6385 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:27:34.284451  6385 pending_rounds.cc:85] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 10
I20260709 12:27:34.284763  6385 pending_rounds.cc:107] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:27:34.285068  6385 raft_consensus.cc:2889] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:34.285658  7329 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:34.283967  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:34.209654 12886 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:34.288815  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:34.214432 12813 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:34.288781  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:34.215018 12803 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.232450 12845 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.173832 12694 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:34.287753  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.227907 12850 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:34.218037 12835 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:34.219769 12774 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:34.239734 12822 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:34.237303 12423 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:34.215586 12700 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.297658  6381 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:34.318634  6381 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:34.214672 12693 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:34.299723  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 112590us
I20260709 12:27:34.246987 12692 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:34.300475  6378 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:34.230031 12846 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:27:34.289510 12809 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:34.291793  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.309646  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:34.280153 12688 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.229441 12848 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:34.313066  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 63843.3us
I20260709 12:27:34.337508  6310 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.337991  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:34.306017  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:34.314342  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:34.339425  6304 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:34.223990 12627 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.293509 12886 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:34.324653 12878 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:34.299604 12606 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.329139  6379 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:34.326932  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:34.338163 12809 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.307438 12762 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:34.372282  6307 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:34.298451 12834 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.366506  6303 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:34.336112  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:34.309751 12803 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.327939 12591 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.348135 12827 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:34.333112 12880 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:34.382635  6310 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:34.348229 12688 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:34.339964 12765 raft_consensus.cc:2764] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:34.337908 12692 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.375025 12809 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:34.430799 12879 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:34.373888 12878 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:34.375984 12690 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.365100 12822 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:27:34.415556 12887 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:34.449189  6377 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 12: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:34.371665 12606 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.459549  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2641) took 1253 ms. Trace:
W20260709 12:27:34.460063  6459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.026s	sys 0.049s
I20260709 12:27:34.460191  6300 rpcz_store.cc:276] 0709 12:27:33.209300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.376752 (+167452us) service_pool.cc:224] Handling call
0709 12:27:34.459297 (+1082545us) spinlock_profiling.cc:243] Waited 711 ms on lock 0x7b5400140d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:34.459463 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1635956096}
I20260709 12:27:34.463896  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3094) took 1085 ms. Trace:
I20260709 12:27:34.464457  6384 rpcz_store.cc:276] 0709 12:27:33.378287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.393186 (+ 14899us) service_pool.cc:224] Handling call
0709 12:27:33.398065 (+  4879us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.061360 (+663295us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.463599 (+402239us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b54001090d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:34.463819 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1524023552}
I20260709 12:27:34.467406 12318 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.468230 12318 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.471267 12318 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:34.460949  6459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.026s	sys 0.049s
W20260709 12:27:34.482163  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.478806  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:34.433632 12827 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:34.488895  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:34.390336 12834 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.476871 12690 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.494354 12828 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:34.495085  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.497393  7329 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:34.439823 12879 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:34.416178 12840 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 12:27:34.516918  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 443986us
W20260709 12:27:34.536197  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 485975us
I20260709 12:27:34.541093  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:34.515631 12762 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.544179  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.545368  7329 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 19 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:27:34.551249 12879 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.552850 12879 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:34.501135 12822 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:34.554677 12822 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:34.555189 12822 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:34.477300 12887 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.561080 12887 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:34.562682 12712 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:34.564531 12606 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.566169 12591 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.566951 12845 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.567832 12692 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.563050 12694 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.568966 12834 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.574062 12712 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:34.574559  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:34.578228  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:34.488397 12786 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.592172  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:34.593798  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50278us
I20260709 12:27:34.420188 12693 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.597368  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:34.597779  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:34.598398  7329 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:34.598696  7314 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:34.600795  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:34.517372 12627 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.528594 12725 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:34.576928 12839 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.620s	user 0.010s	sys 0.001s
I20260709 12:27:34.606956  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:34.612421  6450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.001s	sys 0.018s
I20260709 12:27:34.486613 12506 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:34.616225  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.034s
W20260709 12:27:34.615787  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 404948us
I20260709 12:27:34.626212  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:34.624930  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.010s	sys 0.034s
I20260709 12:27:34.594540 12712 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:27:34.624012  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 282783us
I20260709 12:27:34.516201 12827 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.621094  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:34.605225 12606 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:34.515000 12889 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.105s
W20260709 12:27:34.629757  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.604418 12423 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:34.649617  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:34.635927  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.652136  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:34.619992  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 336078us
W20260709 12:27:34.619488  6450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.002s	sys 0.019s
I20260709 12:27:34.650002  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:34.654029  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:34.621270 12834 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:34.721249  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:34.619014 12840 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:34.653975  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:34.720001  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.721894  6451 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:34.619486 12841 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:34.767474 12657 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.812s	user 0.009s	sys 0.003s
I20260709 12:27:34.781231  6454 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:34.636178 12690 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.726605  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.782797  6455 raft_consensus.cc:2410] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:27:34.721390 12834 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.727241 12888 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:34.642102 12606 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:34.731191  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:34.731971  6383 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:27:34.732721 12627 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:34.790552  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:34.790958  6385 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:34.631464 12712 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:34.717634 12850 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:34.735720  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2815) took 1050 ms. Trace:
I20260709 12:27:34.801404  6377 rpcz_store.cc:276] 0709 12:27:33.685654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.924598 (+238944us) service_pool.cc:224] Handling call
0709 12:27:34.735539 (+810941us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:34.735666 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176395136}
I20260709 12:27:34.737627 12749 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.741690  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2884) took 1367 ms. Trace:
I20260709 12:27:34.651660 12423 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:34.742360 12117 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.661s	user 0.005s	sys 0.005s
I20260709 12:27:34.732190 12809 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.782109 12841 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.758090 12686 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:34.624054 12878 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.610800 12845 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:27:34.765997 12848 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:34.766446 12898 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:34.720921  6458 raft_consensus.cc:2368] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:34.761873 12822 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.625476 12725 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:34.778899 12897 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:34.786640 12834 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:34.801411 12712 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:34.785588 12803 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:34.784778 12826 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:34.747347 12830 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.786506 12888 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:34.785161 12657 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.034744  6308 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:35.034863 12506 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.036686 12657 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:34.287289  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:34.737138  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:34.805584  6310 rpcz_store.cc:276] 0709 12:27:33.374286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.375431 (+  1145us) service_pool.cc:224] Handling call
0709 12:27:34.741488 (+1366057us) spinlock_profiling.cc:243] Waited 921 ms on lock 0x7b54000942d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:34.741621 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2117350272}
W20260709 12:27:34.975914  6204 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 266) took 26263 ms (client timeout 15000 ms). Trace:
I20260709 12:27:34.977159  6202 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:34.780555  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126641us
I20260709 12:27:35.048532  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:34.769337  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 459795us
I20260709 12:27:34.982687  6381 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:27:34.990058  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2819) took 1289 ms. Trace:
I20260709 12:27:34.801491  6451 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:34.768573  6452 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:27:34.999327  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 378143us
I20260709 12:27:35.001979  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 2950) took 1142 ms. Trace:
I20260709 12:27:35.058315  6458 rpcz_store.cc:276] 0709 12:27:33.859217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.859424 (+   207us) service_pool.cc:224] Handling call
0709 12:27:35.001146 (+1141722us) spinlock_profiling.cc:243] Waited 737 ms on lock 0x7b54001e32d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:35.001563 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1693985152}
W20260709 12:27:34.998304  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 371979us
I20260709 12:27:35.060633  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:35.061244  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:34.853755 12756 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.930s	user 0.010s	sys 0.002s
I20260709 12:27:34.829227  7331 raft_consensus.cc:2410] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:27:34.812914 12423 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:34.851296 12606 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.032271 12318 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.088589 12318 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.054545  6383 rpcz_store.cc:276] 0709 12:27:33.700447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.274787 (+574340us) service_pool.cc:224] Handling call
0709 12:27:34.731525 (+456738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.790430 (+ 58905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.989318 (+198888us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b54003e7950. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:34.989747 (+   429us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1049607040}
I20260709 12:27:34.813122 12749 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.785465 12840 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.016736  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:34.843753 12883 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:34.904500 12886 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
I20260709 12:27:34.883617 12888 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:34.870270 12887 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:34.869918 12897 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:34.892477 12903 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:27:34.893126 12908 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:27:34.881870 12827 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.090814 12692 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:35.096421  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 80331.3us
W20260709 12:27:35.118784  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.119251  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.121271 12828 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:35.097383 12883 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.122849 12690 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:35.123857 12690 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.124444  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.125780 12883 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.098577 12886 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.129663 12886 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.099714 12888 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:35.043203  6204 rpcz_store.cc:269] 0709 12:27:08.712510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.712715 (+   205us) service_pool.cc:224] Handling call
0709 12:27:34.975597 (+26262882us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:24.366296 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:24.384839 (+ 18543us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:24.384898 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:27:24.384914 (+    16us) tablet.cc:662] Decoding operations
0709 12:27:24.386909 (+  1995us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:24.387033 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:24.387091 (+    58us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:27:24.389904 (+  2813us) tablet.cc:701] Row locks acquired
0709 12:27:24.389959 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:27:24.390125 (+   166us) write_op.cc:270] Start()
0709 12:27:24.390276 (+   151us) write_op.cc:276] Timestamp: P: 1783600044390058 usec, L: 0
0709 12:27:24.390297 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:27:24.392063 (+  1766us) log.cc:844] Serialized 10752 byte log entry
0709 12:27:24.930819 (+538756us) op_driver.cc:464] REPLICATION: finished
0709 12:27:25.192697 (+261878us) write_op.cc:301] APPLY: starting
0709 12:27:25.223832 (+ 31135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:25.644056 (+420224us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:25.644211 (+   155us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:26.092997 (+448786us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b18002bcb40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2fe60ec 00007f6fa2fe8af7 00007f6fa2fe9fb0 00007f6fa6cfe1bb 00007f6fa6d455d4 00007f6fa6bf18da 00007f6fa6bf43cb 00007f6fa6bf3fd3 00007f6fa6cf2e34
0709 12:27:26.348893 (+255896us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:26.349411 (+   518us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 12:27:26.349480 (+    69us) write_op.cc:312] APPLY: finished
0709 12:27:26.350127 (+   647us) log.cc:844] Serialized 244 byte log entry
0709 12:27:26.549747 (+199620us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:26.794718 (+244971us) write_op.cc:454] Released schema lock
0709 12:27:26.833250 (+ 38532us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:26.833797 (+   547us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:26.833832 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5945132,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":203968,"apply.run_cpu_time_us":74377,"apply.run_wall_time_us":1665656,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121223,"num_ops":28,"prepare.queue_time_us":684652,"prepare.run_cpu_time_us":22613,"prepare.run_wall_time_us":301621,"raft.queue_time_us":196291,"raft.run_cpu_time_us":26000,"raft.run_wall_time_us":127521,"replication_time_us":540274,"spinlock_wait_cycles":13760768,"thread_start_us":990834,"threads_started":5,"wal-append.queue_time_us":616188,"wal-append.run_cpu_time_us":28599,"wal-append.run_wall_time_us":2371330}]]}
W20260709 12:27:35.131703  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 71419.3us
W20260709 12:27:35.132829  7331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 923) took 4376 ms (client timeout 1589 ms). Trace:
I20260709 12:27:35.132951  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:35.133617  7331 rpcz_store.cc:269] 0709 12:27:30.756439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.756716 (+   277us) service_pool.cc:224] Handling call
0709 12:27:35.132276 (+4375560us) spinlock_profiling.cc:243] Waited 4 s on lock 0x7b54004547d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:35.132516 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9202858240}
I20260709 12:27:35.134795 12923 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.115021 12688 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:35.095006 12749 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.117800 12910 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.143312 12910 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.144804 12841 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:35.114461  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 377217us
I20260709 12:27:35.145802 12850 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:35.147082  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:35.148104 12318 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.109927  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 2966) took 1112 ms. Trace:
I20260709 12:27:35.115383 12840 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.150900  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:35.151737 12840 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:35.108911 12725 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.152410 12840 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:35.153015 12840 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:35.115892 12889 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:35.108883 12706 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:35.155102 12706 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.155517 12117 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:35.117228 12506 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:35.112042  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64120.4us
I20260709 12:27:35.156396 12117 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:35.112398  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 61235.3us
I20260709 12:27:35.157779  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:35.140170  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79191.8us
I20260709 12:27:35.159286  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:35.160283 12827 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:35.161700  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:35.117586 12627 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.104193 12809 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.163897  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:35.166381 12749 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:35.112368 12692 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.118057 12845 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.163542 12627 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.139436 12817 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:35.163430 12506 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:35.155603 12706 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.163764  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:35.183377  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.183884 12922 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:35.164258 12850 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 12:27:35.186802 12786 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:35.157944 12117 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.110967 12822 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.145747 12910 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.113111 12887 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.198899  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.174190 12817 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:35.193754 12606 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:35.150935  6450 rpcz_store.cc:276] 0709 12:27:33.997380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.997524 (+   144us) service_pool.cc:224] Handling call
0709 12:27:33.998451 (+   927us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.022572 (+ 24121us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:34.022635 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.023180 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.678571 (+655391us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:34.678626 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.678714 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.109375 (+430661us) raft_consensus.cc:1685] finished
0709 12:27:35.109425 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.109696 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23546,"raft.queue_time_us":25076,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7776,"thread_start_us":655018,"threads_started":1,"wal-append.queue_time_us":764051}
W20260709 12:27:35.211360  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53828.5us
I20260709 12:27:35.197016 12690 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:35.212482  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62658.2us
I20260709 12:27:35.197523 12506 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:35.224660  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.226406  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65174.3us
I20260709 12:27:35.217983 12692 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:35.229557  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.104288 12897 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:35.231976  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.238698  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.237766 12922 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:35.236044 12899 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:35.250319  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.252628 12506 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:35.257004  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 591711us
W20260709 12:27:35.266809  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 103175us
I20260709 12:27:35.268692 12692 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.274019 12846 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.275094 12846 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.276250  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:35.277402  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:35.277732 12846 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.278219 12809 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:35.278744 12809 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:35.267246  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.220229 12845 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.240813 12922 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:35.279186 12809 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:35.298267 12923 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:35.309190  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:35.337275  7318 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:35.311280  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:35.339318  7317 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:35.340655  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:35.342983  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 113815us
I20260709 12:27:35.343144  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:35.343462  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:35.345003  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:35.346957  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:35.312947 12774 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.985s	user 0.007s	sys 0.012s
W20260709 12:27:35.315477  6425 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.679s (SENT)
I20260709 12:27:35.346753  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:35.316274 12827 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:35.319023 12627 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:35.243974 12880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.005s	sys 0.013s
W20260709 12:27:35.351440 12880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.005s	sys 0.014s
I20260709 12:27:35.351498 12827 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:35.353240 12922 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.328068 12817 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.330096  6450 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:27:35.273940 12835 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.946s	user 0.011s	sys 0.008s
I20260709 12:27:35.349370 12774 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:35.363438 12813 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.057s	user 0.003s	sys 0.020s
I20260709 12:27:35.360584  7320 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:35.363986 12813 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.274972 12692 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.368511 12692 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.369390  6451 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:35.370702  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:35.372251  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.372426  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:35.373019  6454 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:35.373310  6308 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:35.374250  6348 leader_election.cc:400] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:35.374622  6348 leader_election.cc:403] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:35.375319  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:35.376349  6307 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:27:35.355787 12827 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:35.366240 12845 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:35.373091 12712 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.009s
W20260709 12:27:35.412431 12834 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.004s
I20260709 12:27:35.413607 12880 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:35.415567 12846 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.416178 12423 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.421205  6459 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:35.439728  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.438189 12690 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:35.382433 12774 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.456115  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:35.450742  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67372.7us
W20260709 12:27:35.465942  7288 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.629s (SENT) [suppressed 38 similar messages]
I20260709 12:27:35.450037 12840 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.492902  6452 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:35.470671 12845 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.503329 12845 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.451413 12712 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.499748  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.501659 12846 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.411671 12835 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.483407 12817 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:35.499511  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 154644us
W20260709 12:27:35.508167  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.499401  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.517449  6457 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 13: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:35.453925 12834 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.485282 12923 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:27:35.506393 12845 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.533116  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:35.539433 12783 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.540522 12783 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:35.542275  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.543251  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:35.548125  7329 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:27:35.548699 12923 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:35.549364 12923 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:35.553332  6310 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:35.553982 12923 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.559082 12827 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.560324 12923 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:35.564900 12835 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.569240 12834 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.572392 12423 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.576781 12846 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.582207  6456 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:35.590207 12813 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.548710  7329 pending_rounds.cc:85] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 13
I20260709 12:27:35.533994  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:35.628410  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.597043  7329 pending_rounds.cc:107] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:27:35.634690  7329 raft_consensus.cc:2889] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:35.633867 12665 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:35.650344 12909 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:35.651592  6383 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:35.651289 12909 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.643443  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:35.689639  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:35.674295  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.700033  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 67704.7us
I20260709 12:27:35.675777  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:35.706236  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:35.720990  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.654359 12909 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.671495 12783 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.738864  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.742250  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.717020 12813 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.769070  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.784006  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:35.790705  6450 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:35.740839 12923 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.791285  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:35.771893  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96378.6us
I20260709 12:27:35.794957 12692 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:35.795068  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.795598 12692 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.796159 12692 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:35.796638  6383 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 15: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:35.796095  6459 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:27:35.799141  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.654404 12706 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.805908  7320 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:35.810707  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.810859  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:35.751667 12712 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.783672 12922 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.810873  6451 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:35.812808 12922 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.813742  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.815047  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.815985  6451 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:35.822945  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:35.824086  6309 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:35.823052 12846 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:35.824983  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:35.824779 12846 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.825397  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:35.812101  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.812031  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.826440  6301 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:35.831964  6452 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:35.827893  6459 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:35.826416  6304 raft_consensus.cc:1217] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Deduplicated request from leader. Original: 16.13->[17.14-17.14]   Dedup: 17.14->[]
I20260709 12:27:35.850710  6304 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:35.824201  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:35.863451 12940 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:27:35.864801  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 330939us
I20260709 12:27:35.865703  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:35.866254  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:35.867674  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:35.867957 12606 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:35.867692 12665 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:35.832866 12922 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.879034  7319 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:27:35.880029 12926 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:35.887382  7335 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17.
I20260709 12:27:35.858510  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:35.895538  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:35.896165  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:35.896937  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:35.897755  6456 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:35.902915 12606 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:35.903411 12606 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:35.904311 12606 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.905022 12706 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.905671 12606 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:35.907596  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:35.908267  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:35.909579 12923 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.909865  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.912117 12923 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:35.920886 12809 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.000s
I20260709 12:27:35.921420 12809 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.924124  6309 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:35.934952  6457 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:35.936269  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:35.937054  6457 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:35.939049  6301 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:35.940434 12845 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.940689 12817 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:35.941005 12845 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.943454 12845 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.944650  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:35.945259 12931 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:27:35.946259 12688 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:35.947618  6458 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:35.950706 12840 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:35.856624 12423 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.095s
I20260709 12:27:35.951902 12846 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:35.856706 12117 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.954557 12117 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:35.957309 12117 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.852010 12813 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:35.959750  6459 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 12: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:35.961175 12813 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:35.964514  6309 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:35.941478 12817 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:35.967396  6383 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:35.953789  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:35.993072  6454 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.619s	user 0.018s	sys 0.001s
I20260709 12:27:35.993520  6454 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:35.994288  6450 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
W20260709 12:27:35.984208  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 495510us
I20260709 12:27:35.999482  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.001583  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.007215  6459 raft_consensus.cc:2410] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:27:36.008145  6308 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:36.000260  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:36.007272 12786 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:27:36.034933  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1032) took 1383 ms. Trace:
I20260709 12:27:36.035409  6456 rpcz_store.cc:276] 0709 12:27:34.652062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.653824 (+  1762us) service_pool.cc:224] Handling call
0709 12:27:36.034724 (+1380900us) spinlock_profiling.cc:243] Waited 846 ms on lock 0x7b5400344754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:36.034872 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1946786176}
I20260709 12:27:36.043853  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:35.998514 12809 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.007897 12117 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.047641  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:36.046136  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:36.050971  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.051431  6450 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:36.051685  6308 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.066184  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.067183  6301 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:27:36.071702  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.080354  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 126664us
I20260709 12:27:36.087664  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.059437 12117 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:36.104173  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 109685us
I20260709 12:27:36.099566  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:36.106966  6453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.025s	sys 0.045s
I20260709 12:27:36.094123 12922 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:36.113102  6453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.026s	sys 0.045s
I20260709 12:27:36.109354  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:36.115079  6304 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17.
I20260709 12:27:36.099623 12897 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.122674  6309 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:36.122618  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.112861 12926 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:27:36.128664  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.130862  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.127424  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.132452  6307 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:36.127151 12690 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.671s	user 0.014s	sys 0.000s
W20260709 12:27:36.142956  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48181.6us
I20260709 12:27:36.131803 12827 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:36.104923 12817 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.138556 12926 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:36.152647  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.154392  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:36.155889  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.145994 12690 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:36.134053 12897 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:36.173808  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51668.9us
I20260709 12:27:36.140358 12922 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.167770 12692 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.170770  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:36.171133 12809 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:36.150630 12827 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:36.184626  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.191037  6304 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.192252  6425 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17.
I20260709 12:27:36.192538  6425 leader_election.cc:403] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:36.170794 12693 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:36.187772  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:36.195897  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75344.3us
I20260709 12:27:36.186213 12827 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:36.215167  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.228667  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.229663  6305 raft_consensus.cc:2410] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:27:36.229902  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.230697  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.231865  6305 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.230839  6302 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
W20260709 12:27:36.232673 12949 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:27:36.233393  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.234192  6302 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.235049  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.232717  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.236099  6305 raft_consensus.cc:2410] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:27:36.236188  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.238971  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68282.9us
I20260709 12:27:36.240170  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:36.194144 12423 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:36.241593 12423 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.245384 12423 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:36.247200 12117 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:36.248243 12117 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:36.248850 12117 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:36.249375 12117 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:36.258294  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.258765 12926 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.266812 12897 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:36.269384 12692 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:36.270932  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:36.272435  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56168.5us
I20260709 12:27:36.202270 12809 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:36.202587 12923 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17.
W20260709 12:27:36.287034  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 243294us
I20260709 12:27:36.286274  6308 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:36.196442 12813 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.079s
I20260709 12:27:36.220248 12783 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:36.287554 12946 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:27:36.282388 12693 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.302930  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:36.308928  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.289882 12690 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:36.313095  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 304264us
I20260709 12:27:36.307010 12688 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:36.281900 12665 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:27:36.333066  6304 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:27:36.324472 12948 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
W20260709 12:27:36.339838  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 99888.3us
I20260709 12:27:36.301337 12783 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.332543  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:36.344301  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.348752 12117 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:36.353166  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 139470us
I20260709 12:27:36.322268 12690 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.316629 12693 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.342967 12688 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:36.392429 12117 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.393466  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:36.394968  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.395622  6310 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:36.398439  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:36.400084  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 176154us
I20260709 12:27:36.406200  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:36.414577 12606 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:36.415046 12606 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:36.416888  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:36.429744  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:36.417805  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.431221  7329 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:27:36.434321  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 240401us
I20260709 12:27:36.437587 12688 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.418563 12909 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.455298  7335 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:36.435472 12923 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.460677  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:36.437160 12786 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:27:36.443246 12943 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:27:36.491621  6300 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.495981  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.497148  6308 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:36.500720  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63863.1us
I20260709 12:27:36.509444 12834 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:36.489565 12786 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:36.515182 12922 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:36.496053 12954 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.172s
I20260709 12:27:36.517004 12922 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:36.457481 12953 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:36.533870  6310 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:36.540886 12693 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.542552 12690 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:36.552054  6307 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
W20260709 12:27:36.555087  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 149949us
I20260709 12:27:36.433741 12606 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.482173 12909 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.487498 12897 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:27:36.593809  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.595988  6308 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.589829  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:36.601085  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 101431us
I20260709 12:27:36.502183 12923 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:36.614369  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75659.3us
W20260709 12:27:36.613673  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 104610us
I20260709 12:27:36.612123  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:36.616005  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3400) took 1193 ms. Trace:
I20260709 12:27:36.545334 12953 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:36.625078  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69318.9us
W20260709 12:27:36.623108  6301 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.547s	user 0.008s	sys 0.006s
I20260709 12:27:36.626197  6301 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:36.619144  6453 rpcz_store.cc:276] 0709 12:27:35.422316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.422603 (+   287us) service_pool.cc:224] Handling call
0709 12:27:35.423088 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.423725 (+   637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.423768 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.424361 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.116288 (+691927us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:36.116342 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.116423 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.615653 (+499230us) raft_consensus.cc:1685] finished
0709 12:27:36.615682 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.615916 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":195,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":132,"thread_start_us":691405,"threads_started":1,"wal-append.queue_time_us":764077}
I20260709 12:27:36.639158  6381 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:36.623998 12840 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:27:36.638397  7287 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.547s (SENT)
W20260709 12:27:36.643573  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.632923 12690 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.655882  6301 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:36.640355 12693 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:27:36.658501 12958 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:36.667280  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.642257 12783 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:36.672138  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.674911  7317 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
W20260709 12:27:36.654302 12955 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:27:36.655032 12606 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:36.654518 12950 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
I20260709 12:27:36.684806  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.685635  6308 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:36.654734 12959 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:27:36.689404  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.691175  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.691810  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.693188  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.693984  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.694880 12968 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.695250  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.695988  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.715935  6310 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.656543 12840 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:36.624507 12786 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.670682 12965 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:36.707758  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.695250  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:36.713974  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.737517 12968 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.745080  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.745395  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.746084  6308 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.746337  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.746866  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:36.743405  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.747457  6308 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:36.716910  6300 raft_consensus.cc:2410] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:27:36.730528  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.736534  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.715106  6307 raft_consensus.cc:2376] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21. Re-sending same reply.
I20260709 12:27:36.737744  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.748912  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:36.743428  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:36.743454  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.751250  6385 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:36.751400  6301 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:36.688282 12693 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:36.751792  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.712409 12959 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.750401  6303 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:36.662791 12690 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:36.758862  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.762836  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.763521  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:27:36.765621  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 130356us
I20260709 12:27:36.764232  7329 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:36.754577  6453 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:36.769134 12968 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:36.769613 12969 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.770147 12969 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.772632 12969 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:36.773785 12973 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:27:36.743541 12953 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.776306 12840 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.776958 12923 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:36.758893 12959 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:36.779295  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.783603 12909 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:36.785321 12971 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:36.786732 12971 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:36.787848 12971 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:36.789021 12922 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:36.792636 12922 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:36.792971 12786 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:36.790786  6310 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:36.793725 12922 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.791807  6301 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:36.795070 12922 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:36.797076 12961 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:36.797490 12961 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:36.800292  6307 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:36.804216  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:36.715719 12897 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:27:36.818542 12972 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.819160 12972 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.821233 12972 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.825038  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.831043 12693 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.802635 12817 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.835455 12817 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.838030 12817 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.842253  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:36.751937 12965 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.844120  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:36.845633  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:36.846076  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.848692  6385 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:27:36.853246  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.854249  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.855195  6379 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:36.806274 12845 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:36.860291  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50956.9us
W20260709 12:27:36.851022 12809 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.560s	user 0.008s	sys 0.004s
I20260709 12:27:36.861722 12834 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:36.798410 12961 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.852758 12909 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.863332 12834 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:36.863773 12834 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:36.864708 12834 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.865944 12834 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:36.866318 12845 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.863790 12909 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.868314  6380 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:36.868795 12845 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:36.873476 12954 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:36.876065  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.876924  6380 raft_consensus.cc:2368] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16.
I20260709 12:27:36.876896  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:36.878363  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.878851  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.879700  6376 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:36.879851  6385 raft_consensus.cc:2410] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:27:36.879913  6378 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:36.884357  6384 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:36.884759  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.883998 12834 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:36.885318  6379 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:36.852322 12783 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:36.860996 12972 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:36.891073 12972 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:36.891465 12972 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:36.895705 12690 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.896628  6379 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:36.901846 12974 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:36.902807 12813 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:36.903512 12974 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:36.905090 12974 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.905635  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:36.903940 12813 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:36.907748 12813 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:36.909045 12813 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:36.865996 12827 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.591s	user 0.010s	sys 0.001s
I20260709 12:27:36.911448  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.911693  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:36.910552 12827 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.912281  6457 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:36.913301  6377 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:36.913839  6456 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:36.915249 12827 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.919533  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:36.924335 12809 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:36.931473  6377 raft_consensus.cc:2410] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 16 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:27:36.922101  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:36.918828 12974 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:36.919320  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:36.929029  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:36.930255 12965 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:36.939564  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:36.940466 12706 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:36.946201  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:36.930449 12712 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:36.947446 12712 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.954015 12959 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:36.955534  6381 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:36.958918 12959 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:36.959904  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 101115us
I20260709 12:27:36.961521  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:36.989130  6383 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
W20260709 12:27:36.992482  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 207639us
I20260709 12:27:36.993687  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:36.985598  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:36.967226  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.972127 12423 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:36.972435  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.000573  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:37.016340  7288 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.520s (SENT)
I20260709 12:27:37.018612 12117 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:37.019433  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.020153  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.020953  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:37.020846 12117 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.021917  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.023042  6383 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.023437  6310 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:36.999146 12813 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:37.023063  7331 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:37.029037  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.029717  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.030627  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:37.033283  6425 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17.
I20260709 12:27:37.033794  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:37.034982  7335 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:37.035280  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.030212  6300 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:37.039062  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.040259  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.040445  6308 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:37.048897 12117 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:37.015129 12827 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:37.050112  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:37.026223 12897 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.056712  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.004926 12423 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.028264 12813 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:37.062388  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.053737  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.075205 12712 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:37.088780  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 143) took 54302 ms (client timeout 15000 ms). Trace:
I20260709 12:27:37.089540  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.052156 12827 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.028687 12923 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:27:37.102784  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:37.104231  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87921.4us
W20260709 12:27:37.104996  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.547s (SENT)
I20260709 12:27:37.105347  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:37.091070 12970 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:37.090365 12813 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.108696  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.091411 12827 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:37.109256  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55579.9us
I20260709 12:27:37.092918 12712 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.111655  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.112486  6382 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:37.112905  6302 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:27:37.113794  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.075789 12897 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:37.089977  6202 rpcz_store.cc:269] 0709 12:26:42.753718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.754055 (+   337us) service_pool.cc:224] Handling call
0709 12:27:37.055866 (+54301811us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:35.216278 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:35.219857 (+  3579us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:35.219953 (+    96us) write_op.cc:435] Acquired schema lock
0709 12:27:35.221713 (+  1760us) tablet.cc:662] Decoding operations
0709 12:27:35.228014 (+  6301us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:35.228166 (+   152us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:35.228198 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:27:35.232169 (+  3971us) tablet.cc:701] Row locks acquired
0709 12:27:35.233082 (+   913us) write_op.cc:260] PREPARE: finished
0709 12:27:35.233336 (+   254us) write_op.cc:270] Start()
0709 12:27:35.236880 (+  3544us) write_op.cc:276] Timestamp: P: 1783600055233235 usec, L: 0
0709 12:27:35.236944 (+    64us) op_driver.cc:348] REPLICATION: starting
0709 12:27:35.278756 (+ 41812us) log.cc:844] Serialized 1275 byte log entry
0709 12:27:36.140592 (+861836us) op_driver.cc:464] REPLICATION: finished
0709 12:27:36.443460 (+302868us) write_op.cc:301] APPLY: starting
0709 12:27:36.481392 (+ 37932us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:36.545932 (+ 64540us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:36.600032 (+ 54100us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:36.788366 (+188334us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:36.788745 (+   379us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:27:36.788820 (+    75us) write_op.cc:312] APPLY: finished
0709 12:27:36.789156 (+   336us) log.cc:844] Serialized 41 byte log entry
0709 12:27:36.792594 (+  3438us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:36.792727 (+   133us) write_op.cc:454] Released schema lock
0709 12:27:36.792805 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:36.793427 (+   622us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:36.793460 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5019777,"reported_tablets":29,"spinlock_wait_cycles":277376,"child_traces":[["op",{"apply.queue_time_us":259356,"apply.run_cpu_time_us":11047,"apply.run_wall_time_us":367589,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":157897,"prepare.run_cpu_time_us":38284,"prepare.run_wall_time_us":279501,"raft.queue_time_us":483839,"raft.run_cpu_time_us":43022,"raft.run_wall_time_us":269644,"replication_time_us":887826,"thread_start_us":774683,"threads_started":4,"wal-append.queue_time_us":344091}]]}
I20260709 12:27:37.118235 12817 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:27:37.122123 12817 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:37.122819 12817 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:37.123798 12817 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.125427  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:37.126938  6308 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.126765  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.124955 12817 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:37.128108  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.128793  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.129283  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.130270  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.129361  6385 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:37.130854  6384 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:37.130903  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.131364  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.132155  6383 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.132244  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.132994  6377 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:37.133716  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.133960 12909 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.106374 12984 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:37.143841 12985 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:27:37.145640 12423 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:37.146196 12897 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:37.131697  6379 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:37.148133  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:37.148726  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43770.5us
I20260709 12:27:37.149976  6382 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:37.151551  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:37.152954  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3: Illegal state: Replica 4e0e521cb08c420f981a29576eee90a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 18 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:37.155180  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.155460  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:37.155929  6383 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:37.156173  7317 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:37.158321  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.158535  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.158900  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.159075  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.115942 12712 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.159621  6383 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:37.159847  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.160684  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.161240 12712 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:37.163784 12972 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.166956  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.169486  6455 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:37.170130  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.170576  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.168799  6383 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:27:37.168529  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:37.166431  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.171828  6377 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:37.171973  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.171182  6384 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:37.171154  6378 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:27:37.168574  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.170713  6382 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:37.172822  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57830us
I20260709 12:27:37.174851  6385 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.115680 12953 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:37.171133  6383 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 6
I20260709 12:27:37.174435  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:37.175323  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56636.1us
I20260709 12:27:37.175915  7335 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:37.176452  6453 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.114038 12693 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.177605  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:37.175355  6383 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:27:37.179065  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.184240  6383 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:37.181854  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.187289  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.188230  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:37.188752  6455 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:37.182919  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:37.189625  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.190179  6456 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.191407  6385 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:37.193280  6450 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:37.187469  6459 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:37.195150 12117 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:37.200899  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.204103  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:37.203110 12974 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:27:37.210186 12897 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:37.217340  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:37.220292  6384 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:37.225853  6382 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:37.230288  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.175628 12972 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.230875  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:37.233932  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59570.3us
I20260709 12:27:37.214994 12817 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:37.234122  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:37.237645  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:37.238477  6382 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:37.239354  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.229102 12984 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:27:37.182286 12813 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.241541  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:37.244354  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:37.247691  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.248584  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.247875  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:37.237224 12817 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.244892  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.235759 12897 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:37.246383  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:37.236622 12423 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:37.202971 12693 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.254861  6382 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:27:37.255800  6454 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:37.258571  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:37.246929 12972 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:37.260476  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:37.263222  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.263301  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.285171  6452 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:37.264633  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:37.266647  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.262019  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:37.275329 12117 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:37.276820  6453 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:27:37.280970  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66602.7us
I20260709 12:27:37.292318  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:37.253880 12923 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:37.253541 12974 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:37.270726 12423 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:37.257762 12817 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:37.304620 12827 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.305968  6381 raft_consensus.cc:2368] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20.
I20260709 12:27:37.306830  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:37.247414 12909 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:27:37.308770  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.275378 12897 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.313558 12817 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.301798 12968 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:37.301590 12423 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.311167  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.301239 12969 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:37.314893 12909 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:37.299948  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:37.259441 12961 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:37.263403 12954 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:37.320339 12827 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.272632 12813 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.271315 12991 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:37.280486 12693 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.259961 12712 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:37.288419 12786 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:37.278844 12972 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:37.331588  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:37.262235 12965 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
W20260709 12:27:37.334322  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71140.3us
I20260709 12:27:37.338804 12954 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.337263  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:37.338356  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 89989.8us
I20260709 12:27:37.325459 12606 raft_consensus.cc:2764] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:37.365532  6457 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:37.345988  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:37.332953 12968 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:37.351284  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89340.3us
I20260709 12:27:37.371115  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.329285 12840 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:37.383786 12813 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:27:37.383713 12840 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.357894  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:37.280616 12690 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:27:37.341552 12783 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:37.354979 12786 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.358137 12972 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:37.367388  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56300.5us
I20260709 12:27:37.344849 12809 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:37.369860  6424 leader_election.cc:400] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16.
I20260709 12:27:37.398505  6424 leader_election.cc:403] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:27:37.374207 12970 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:37.375939 12117 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:37.398772  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.399245 12117 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.391196  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.401216  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.361228 12953 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:37.364915 12954 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:37.363583  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:37.386583 12968 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.415056  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:37.415305  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.411412  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:37.418529  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48847.8us
I20260709 12:27:37.420519  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:37.401155 12786 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:37.358816 12712 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.396960 12926 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:37.396229 12974 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:37.397166 12984 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.402205 12970 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:37.408291 12953 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:37.408115 12606 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.388706 13002 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:27:37.498014 12117 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:37.498680  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97578us
I20260709 12:27:37.464442  6453 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:37.442209  6455 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:37.426280 12840 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:37.412282 13001 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:37.462790  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.434757 12923 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:37.463784  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 127355us
I20260709 12:27:37.516963 12117 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:37.461493 12827 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:37.465638 12953 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:37.521801  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76158.3us
I20260709 12:27:37.438782 12986 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:37.527125  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:37.497323 12926 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:37.530699  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68171.2us
I20260709 12:27:37.501353 12997 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16.
I20260709 12:27:37.532536 12970 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.541116 12956 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:37.541905 12834 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:37.542296 13007 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:37.543589 12897 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:37.462833 12984 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.501556 12606 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.568446  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:37.528643 12986 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:37.578420  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:37.535979 13010 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:37.582857  7329 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:37.584504  6453 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:37.567515 12970 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:37.568130 12956 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:37.615187  6300 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:37.608546  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.619210  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:37.629837  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.598222 12974 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.578610 12986 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.636888  6302 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:37.638214  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:37.641732  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 221285us
I20260709 12:27:37.643798  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:37.645133  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.645637  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:37.646435  7317 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:37.616927 12956 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.628718 12965 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:37.628472 12783 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.597882 12897 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.640547 13010 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:37.675107  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 490561us
I20260709 12:27:37.679728  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:37.681473  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 79900.5us
W20260709 12:27:37.681689  7288 leader_election.cc:336] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.562s (SENT)
I20260709 12:27:37.682235  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.682276  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:37.716549 13008 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
I20260709 12:27:37.713791 12998 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:37.714380 13002 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:37.715690 13015 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:37.720696  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:37.607476 12923 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.724345 12840 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:37.729241  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.729128  6380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:37.729841  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:37.794047  6380 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:27:37.794261  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 64485.5us
I20260709 12:27:37.794598  6380 raft_consensus.cc:614] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:37.796769 12997 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:37.797354 12997 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:37.798277 12997 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.800278 12922 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.800498 12961 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:37.800973 12961 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.730464 13014 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:37.801839 13007 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:37.729775 13001 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:37.685293 12783 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.802415 13007 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.803018 13001 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:37.803498 13001 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:37.804822 12783 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:37.804472 13001 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.805743 12840 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:37.730783 13011 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:27:37.645272 12974 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:37.738627 13005 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:27:37.739634 12999 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:27:37.740125 12909 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:37.740334 12996 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:27:37.746994 12984 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.747152 12606 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:37.853392  6381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:37.748003 12712 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:37.731287 12813 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:27:37.753099 12817 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:37.754733  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73074.8us
I20260709 12:27:37.726760 12971 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:27:37.862457 12971 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:37.862939 12971 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:37.863824 12971 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.865140 12971 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:37.865337 12897 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.848090 12909 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.865623 12953 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.677115 13010 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.772228  7317 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:37.621100 12926 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:37.800940 12922 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:37.872195  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 151603us
I20260709 12:27:37.872998  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:37.805822 13001 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.760319 12983 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:37.806349 12840 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.735214 13018 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:37.877645  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.877802  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.877892  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:37.809672 12986 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.650840 12827 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:37.773026 12834 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:37.811198 13003 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:37.821440  6204 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 16. New cstate: current_term: 16 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:27:37.832993  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 194870us
I20260709 12:27:37.833201  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:37.835568  6451 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:37.676924 13012 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.131s
I20260709 12:27:37.804724 13007 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:37.860639  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.759369  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.866338 12909 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:37.871483 13010 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:37.875336  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.875909 12817 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:37.877084 12922 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:37.878322 12840 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:37.892066 12986 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:37.892565  6425 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.614s (SENT)
I20260709 12:27:37.896209  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:37.897563  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:37.897804  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.897982  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.921263  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:37.921912  6310 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:37.898895  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.899715  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.879082 12983 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:37.900666  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.924069  6300 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:27:37.924106  6305 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:37.902263  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.889487  6302 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:37.843324 12809 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:37.911038  6204 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:27:37.917153  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 84054.5us
I20260709 12:27:37.896198 12959 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:37.918792  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.918217  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 158919us
I20260709 12:27:37.925236  6304 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:37.929104  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.921731  6303 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
I20260709 12:27:37.934420  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:37.929152  6204 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 7 to 11, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 11 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:27:37.936136  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43606.9us
I20260709 12:27:37.937146  6204 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:37.940133  6204 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 15, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:37.943984  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.944375  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.920729 12817 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.949713  6204 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:37.948705  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.955229  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.962929  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:37.957484 12809 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.952492 12986 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:37.931912 13019 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:37.955753 12834 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.984277  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:37.912360 12786 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:37.933596 12959 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:37.989881  6204 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:37.926549 13024 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:37.992513  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:37.942308 12922 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:38.000265  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3419) took 1088 ms. Trace:
W20260709 12:27:38.001374  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.961926 12817 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:38.037388  6451 rpcz_store.cc:276] 0709 12:27:36.911436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.961270 (+ 49834us) service_pool.cc:224] Handling call
0709 12:27:38.000015 (+1038745us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54001e23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:38.000183 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1876062464}
I20260709 12:27:38.011758  6301 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:37.985985 12827 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:38.013257  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:37.989166 12845 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:27:38.017254 12968 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.581s	user 0.013s	sys 0.000s
I20260709 12:27:37.951195 13020 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.075s
I20260709 12:27:37.988234 12983 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.018801  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:38.023847  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.028002  6303 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.029294  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:38.063035  6305 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 13: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:38.060910  7332 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:38.030866  6304 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:38.029289 12953 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:38.034431  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50245us
I20260709 12:27:38.038364  6204 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 19, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:38.064941  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.065398  6204 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 15, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:38.067893  6204 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:38.039944  6309 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:38.041913  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 98011.7us
I20260709 12:27:38.069932  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:38.053260  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 105633us
I20260709 12:27:38.026845 12690 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:38.080263 12690 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:38.081259  7288 leader_election.cc:336] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.629s (SENT)
I20260709 12:27:38.083031 12786 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.063103  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:38.087082  7318 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:38.088392  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:38.089082  7331 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
I20260709 12:27:38.073913  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.090322 12845 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:38.091897 12959 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.092334 12840 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.078719 12606 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:38.079344  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.063674 12968 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:38.100936 12954 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.004s
I20260709 12:27:38.036577 13024 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.036653 12986 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:38.075145 13022 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:27:38.082278  7288 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.896s (SENT)
I20260709 12:27:38.128319  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:38.104640  6204 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:38.120751  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.121197  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.127166  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.131553  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.132508  6377 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:38.144896  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.145716  6384 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:38.146883  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.147030  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.148098  6451 raft_consensus.cc:2410] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:27:38.101858 12956 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.149585 12817 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.131996 12954 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.132321  6204 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 19, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.152972  7329 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:38.153260  6204 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 16, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 16 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:38.154791  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.155596  6380 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:38.157619  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.158246  6454 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:38.157850  6204 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:38.159207  6303 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:38.161384  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.162753  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.086335 12983 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:38.163828  6455 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
W20260709 12:27:38.163975  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 226853us
I20260709 12:27:38.164393  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:38.170797  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60327.9us
I20260709 12:27:38.178030  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.178709  6456 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:38.179133  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.181241  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:38.181622  6303 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:38.181864  6377 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.183903  6453 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:38.187743  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.180172  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:38.191717  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:38.186100  6204 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 16, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 16 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.193455  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.194303  6301 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:38.194764  6382 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:38.194453  6204 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:38.195952  6204 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 20, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 20 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.198980  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.201692  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.203310  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:38.203938  6380 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:38.204454  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.209805  6305 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:38.210134  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 127876us
I20260709 12:27:38.166172 12926 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:38.213148  6204 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 16, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 16 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:38.127734 13024 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:38.214547  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.177639 12953 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:38.217854  6204 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 8 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:38.221815 13021 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:38.222719 13009 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:38.222581 13030 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:27:38.199658 12922 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:38.233480  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:38.234961  6377 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:38.236454 12786 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.237002 12786 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:38.237998  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 252dcff751be4a3d948b0f83cb081100: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.239087 12809 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:38.239651 12809 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.240300 12693 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:38.242920 13018 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:38.248858  6380 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:38.225986  6454 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:38.229758  7319 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:38.199000 13012 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:38.231379  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:38.234081  6376 raft_consensus.cc:2376] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18. Re-sending same reply.
I20260709 12:27:38.251011  6204 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 14, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 14 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.219434 13019 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:38.254392  6204 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 19, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 19 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.225117 12690 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:38.256177  6204 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 9 to 14, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 14 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:38.281400  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.299018  6451 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:38.251164 13031 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.231998 12922 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.304114  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.253088 12706 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:38.271984 13002 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:38.276669 12959 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:38.321108  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.322077  6451 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 12.
I20260709 12:27:38.317378  6454 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:38.323729  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.280020 12786 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.286725 12968 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:38.324715  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69265.3us
I20260709 12:27:38.330307  6454 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:38.325667  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 129536us
I20260709 12:27:38.318008 12922 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.302428 12956 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.302361 12954 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:38.338289  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:38.330003  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 115557us
I20260709 12:27:38.334458  6305 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:38.331878  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:38.334863  6304 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:38.334887  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:38.337033  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.325946  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.287773 13009 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:27:38.339280  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.339727  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:38.343175  6301 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:38.339607  7319 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:38.340137  6308 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:38.340647 12840 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.340745  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.341190 13002 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:38.314415 13031 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.315121 13010 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:38.335927  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 97214.9us
I20260709 12:27:38.342881  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.351207  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.352258  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:38.353328  6308 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:27:38.347052  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:38.347429  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.353854  6303 raft_consensus.cc:2376] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21. Re-sending same reply.
I20260709 12:27:38.354108  6301 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:38.354000  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.359562  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:38.360707  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:38.342684 13009 consensus_queue.cc:1243] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:38.370870  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:38.332207 12909 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.351353 12970 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:27:38.674450 12693 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:38.420269 13010 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.578783  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.676896 12693 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.575829  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:38.678063  6310 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:38.380828  6454 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:38.679294 12970 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:38.582990  6301 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:27:38.680260  6301 pending_rounds.cc:85] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 13
I20260709 12:27:38.376165  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:38.620450  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3001) took 1272 ms. Trace:
I20260709 12:27:38.678471  6307 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.513790  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:38.601018  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.679694 12693 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.523685  6309 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
W20260709 12:27:38.421236 12972 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.008s	user 0.012s	sys 0.001s
I20260709 12:27:38.681277  6452 rpcz_store.cc:276] 0709 12:27:37.348280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.348477 (+   197us) service_pool.cc:224] Handling call
0709 12:27:38.620145 (+1271668us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54001e2dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:38.620360 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2544031488}
I20260709 12:27:38.378523  6381 raft_consensus.cc:606] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:38.684705  6381 raft_consensus.cc:614] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:38.425241 12968 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:38.679754 12970 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:38.685912 12968 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:38.680691  6301 pending_rounds.cc:107] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:27:38.531131 12923 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.686731  6301 raft_consensus.cc:2889] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:38.686983 12923 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.683455 12972 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.357360 13039 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:38.354871 13007 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.334s
I20260709 12:27:38.689429 12923 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:38.689850 12972 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.477272 13018 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.216s
I20260709 12:27:38.681022  6454 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:27:38.690618 12970 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.691305 12909 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.417752 13031 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.347648 13002 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:38.512781  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:38.597054 12813 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:38.763743 13002 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:38.764014  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 251344us
I20260709 12:27:38.764143 12813 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.764787  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:38.599478  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.766623 12813 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:38.556950  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.769954  6377 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 11. (term mismatch)
I20260709 12:27:38.609473  6303 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:38.523412 12706 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:38.388376  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.505244 13001 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.190s
I20260709 12:27:38.618268 12809 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:38.572247  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.776919  6379 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:38.426925  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:38.433897  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57178.1us
I20260709 12:27:38.616074  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.779569  6378 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:27:38.779855  6450 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.468112 12840 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.437204  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:38.785244  6304 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:38.625229  6455 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:27:38.533295  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:38.531898 13036 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 12:27:38.512122  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:38.530543 13038 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.189s	user 0.003s	sys 0.000s
I20260709 12:27:38.430835 12712 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:38.418013  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:38.793144  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 375194us
I20260709 12:27:38.794826  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:38.795892  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:38.360880 12817 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:38.361140 13020 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.342s
I20260709 12:27:38.537518 12922 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.610231 12961 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:38.483520  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 150059us
W20260709 12:27:38.433429  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50972.9us
I20260709 12:27:38.444864  7319 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:38.426355  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.599823  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:38.500802  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84841.4us
W20260709 12:27:38.808790  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 209088us
I20260709 12:27:38.713449  7335 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:38.809561  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.694751 13007 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:38.687300 12968 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.830039  6450 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:38.832772  7319 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:27:38.755613  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:38.758627 13024 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:38.585282  6458 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:27:38.770392  6377 pending_rounds.cc:85] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 10
I20260709 12:27:38.844458  6377 pending_rounds.cc:107] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:27:38.844872  6377 raft_consensus.cc:2889] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:38.705641 12971 raft_consensus.cc:2764] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:27:38.712747 12974 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:38.767503 13050 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:38.773252  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.780530  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:38.784252  6382 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:38.787246  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.855443  6455 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 14: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:38.789433  6456 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:38.791946  6380 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:38.772603 12997 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:38.797946  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:38.862831  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64978.9us
I20260709 12:27:38.773924 12706 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:38.710551  6307 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:38.816039  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.790956 13018 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:27:38.772826 13031 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:27:38.748663  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.833532  6383 raft_consensus.cc:606] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:38.832772  7329 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:38.834214  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:38.836253  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:38.800921 12986 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:38.802894 13039 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:38.867087  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:38.870285  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97805us
I20260709 12:27:38.819082 13007 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:38.830473 12961 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:38.800768 12840 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:38.887956  6454 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:38.888048  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.902271  6456 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:38.888554  6383 raft_consensus.cc:614] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:38.884644  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:38.904929  6458 raft_consensus.cc:2368] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:38.877462  6453 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 11.
I20260709 12:27:38.886479  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:38.906096  6455 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:38.891579  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55425us
I20260709 12:27:38.849131 12923 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:38.895684  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:38.851629 13047 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:38.862253 12997 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.881668 12817 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:38.903730  6376 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 11.
I20260709 12:27:38.881343 12693 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:38.868316 13050 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:38.908303  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:38.912688  6307 pending_rounds.cc:85] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:27:38.925585  6307 pending_rounds.cc:107] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:27:38.926067  6307 raft_consensus.cc:2889] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:38.915614  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 149993us
I20260709 12:27:38.927768  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:38.916700  6304 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:38.899194 13015 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:38.897929 13039 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.892225 12986 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.936715  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.931969  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:38.899678 12961 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.930902 13009 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:38.886950 13040 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:27:38.937965  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:38.921854 12809 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:38.890045 12974 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:27:38.922796 12997 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:38.949723  6454 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:38.943948  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.013520  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:39.035842  6456 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:27:39.037075  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:39.038053  6452 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:38.930192 13050 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:38.936357 12972 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:39.079547  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 261539us
W20260709 12:27:39.079670  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 135778us
I20260709 12:27:39.080627  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.081655  6348 leader_election.cc:400] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:39.081946  6348 leader_election.cc:403] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:38.933024 12817 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:38.953229 13043 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:38.936805 13039 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:38.957734 12961 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:38.943926 12986 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:38.956378 13020 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:38.926378 13047 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.047593 12922 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:39.048929 13024 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:39.012241 12809 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.956534 12693 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:27:39.018116 12954 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:39.016871 13007 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.090530 13031 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.059101 12840 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.064618  6452 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:39.034066 12423 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:39.029593 12970 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.090284  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:39.092077  6377 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:39.116983  6380 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:39.092914  6385 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:39.117359  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:39.093713  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 166053us
I20260709 12:27:39.095011  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:39.098618  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:39.118249  6379 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:39.081995 12834 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:27:39.090571 12909 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:27:39.091008 12968 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:39.120914 12968 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.121331 12968 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:39.058038  6455 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
W20260709 12:27:39.114457  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.109517 12809 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:39.074007 13028 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:39.095782 13043 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:39.125342 12909 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:39.147327 12909 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:39.147475 13043 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.149524 12834 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:39.150022 12834 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:39.119900  6452 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:39.108970 12606 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:39.124660  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.092491  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 225522us
I20260709 12:27:39.128942  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:39.145545 12997 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.116173 12423 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.154604  7288 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:39.119971 13007 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:39.141793 12817 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.170114  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.162984 12423 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:39.156744 13028 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:39.166142 12922 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:39.178557 13052 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:27:39.185792 12922 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:39.183352 13040 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.195561 12706 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.196333 12834 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.197961 12834 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:39.198474 12706 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.199296 12706 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:39.195502 12922 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:39.181111  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:39.210909  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56410.3us
I20260709 12:27:39.211962  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:39.223639  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.184002 12897 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:39.174213 12606 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.199720 13047 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:39.186640 12693 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:39.216383 12706 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:39.196291 13040 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.249403 12954 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:39.268278  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54898.8us
I20260709 12:27:39.225327 13063 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:39.227311 12991 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:27:39.257540 12897 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:39.265687 13047 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.250221 12961 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.289461 12968 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.291947 12968 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.293007 12897 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.296299 13047 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.278479 13028 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:39.302928 13066 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:27:39.302660 12986 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.269235 12693 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.306787 12693 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.310264  6303 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:27:39.311393  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:39.312589  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:39.313148  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.313766  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:39.314572  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.315174 13039 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.315961 12606 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.273850 12997 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:39.317404  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.317453 13039 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.317909 13040 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.318698  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:39.319077 12961 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:39.319200 12991 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:39.319927 13047 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:39.319787 12991 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.321259  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3430) took 1036 ms. Trace:
I20260709 12:27:39.321669  6457 rpcz_store.cc:276] 0709 12:27:38.285380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.426027 (+140647us) service_pool.cc:224] Handling call
0709 12:27:39.321063 (+895036us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400163ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:39.321198 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1173033216}
I20260709 12:27:39.325635  6304 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:39.304939 12986 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.327662  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.328504  6300 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:39.329322 13062 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:27:39.332396 12712 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:39.333154 12813 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:39.333343 12712 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:39.333964 12813 consensus_queue.cc:1243] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
W20260709 12:27:39.334976  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51191.5us
I20260709 12:27:39.313397 12909 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.336098  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:39.336663  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:39.332448 13009 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:39.338706 13061 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:39.339140 13061 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:39.339509 13061 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.344024 13070 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:39.337363  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:39.347420  6450 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:39.347573  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:39.349123 13071 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:39.349617  6455 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:39.350049 13071 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:39.350716 13015 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.351245 13015 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.353382 13015 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:39.355657  6455 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 12:27:39.338287  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:39.339615  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:39.360347  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 574467us
I20260709 12:27:39.362118  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:39.364778  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.313097  6458 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:39.372046  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:39.372913  6457 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:39.374764 12986 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:39.375241  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:39.375510 12986 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:39.376199  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.377318 12968 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:39.360484  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.350726 13043 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:39.379520 13043 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:39.380018  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.380640 12840 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:27:39.380405 13043 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.381011  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.383219  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:39.383442 12606 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:39.384056  7329 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:39.384974 12974 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:39.385133  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.385433 12974 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.385984 12974 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.387367  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:39.388116  7331 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:39.389278  6308 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:27:39.389683  6303 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:27:39.386137  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:39.390479 13001 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:39.382330 12840 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:39.391580 12840 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:39.391980 12840 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:39.394847 13040 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:39.383960 12606 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:39.366094  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.376946 12986 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.399101 12606 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.399930  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.401196 12968 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:39.401793 12968 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:39.382171  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.402738  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:39.403681  6307 raft_consensus.cc:2410] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:27:39.403780  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:39.404663  6453 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:39.405221 13059 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:27:39.402817 12968 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.403774 12845 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:39.406066 12845 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.406454 12845 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:39.406796  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3013) took 1279 ms. Trace:
I20260709 12:27:39.407189  6459 rpcz_store.cc:276] 0709 12:27:38.128045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.579602 (+451557us) service_pool.cc:224] Handling call
0709 12:27:38.580080 (+   478us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.580616 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:27:38.833561 (+252945us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.932629 (+ 99068us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.935963 (+  3334us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.936109 (+   146us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:38.936163 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.936231 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.353246 (+417015us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b18001ed780. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:27:39.404599 (+ 51353us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:39.406535 (+  1936us) raft_consensus.cc:1685] finished
0709 12:27:39.406571 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.406732 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372885,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":24810,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":9774,"spinlock_wait_cycles":159485568,"thread_start_us":314380,"threads_started":2,"wal-append.queue_time_us":226180}
I20260709 12:27:39.409811  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:39.410342  6459 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:27:39.411296  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.353890 13015 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:39.412056 13068 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:39.412707 13068 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.413125 13068 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:39.414239  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:39.412369 13015 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.404542  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.416661  6451 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:39.418740  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:39.420511  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59062.8us
I20260709 12:27:39.424448  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:39.426779  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:39.435105  6308 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:39.428720  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.437146  6310 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:39.438580  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:39.439571  6310 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:39.443871  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.445747  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1073) took 1109 ms. Trace:
I20260709 12:27:39.337420 12909 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.446451  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.447038  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:39.487887  6302 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:39.489415  6457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "402a1967982740658c70049fd62be44e"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:39.490404  6457 raft_consensus.cc:606] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:39.446326  6451 rpcz_store.cc:276] 0709 12:27:38.336820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.352259 (+ 15439us) service_pool.cc:224] Handling call
0709 12:27:39.445581 (+1093322us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54000fc654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:39.445688 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1216976768}
I20260709 12:27:39.425354 12991 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:39.492168  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:39.421888 13002 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.658s	user 0.012s	sys 0.000s
W20260709 12:27:39.487246  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.491016  6457 raft_consensus.cc:614] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:39.495088  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:39.425940 13047 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:39.436676 13024 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:39.531409  6308 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:39.504354 12991 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.532964  7335 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:39.533141  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:39.445768 13001 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.423799 13058 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:27:39.503441 13009 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:39.570024  7329 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:39.435442 12712 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.117s
I20260709 12:27:39.423980 13069 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:27:39.527379 13047 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.578939  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:39.580904  7335 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:39.447631 12423 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.493517 12972 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
W20260709 12:27:39.592506  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 143834us
I20260709 12:27:39.554983 12693 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.487154 13039 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
W20260709 12:27:39.602005  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 106344us
I20260709 12:27:39.602365  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:39.602886  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:39.604712  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:39.607852  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:39.503504 13002 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.609143  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:39.608526  6385 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:39.612504  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:39.617710 13083 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:39.565671 13001 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.577579 13032 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.620399  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:39.585319 13047 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:39.624034  7335 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:39.622524  6382 pending_rounds.cc:85] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 9
I20260709 12:27:39.596202 13020 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:39.664474  6381 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:27:39.597959 13050 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.509s	user 0.011s	sys 0.000s
I20260709 12:27:39.665504  6382 pending_rounds.cc:107] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:27:39.631311  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:39.669315  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:39.671751  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67591.1us
I20260709 12:27:39.670840  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:39.596351 13009 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:39.669584  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 103623us
I20260709 12:27:39.700002  6382 raft_consensus.cc:2889] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:39.560293 12991 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.704452  6456 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:39.674703  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:39.705410  6383 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:39.611059 13075 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:39.585417 12809 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.609874 13024 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:39.779009  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:39.781728  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71620.2us
I20260709 12:27:39.791934  6384 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
W20260709 12:27:39.783748  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.614871 13058 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:39.675447 13069 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:39.568564 12954 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.792922  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:39.793149 13031 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.677s	user 0.008s	sys 0.003s
I20260709 12:27:39.704581 13064 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 12:27:39.886868  6453 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:39.805439 12706 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:39.806591 13076 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:27:39.808475 13009 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.794685 13040 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.813791  6458 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:39.675110 12974 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.848353 13002 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:39.923455  6383 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:39.887307 12909 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:39.853142 12986 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.855398  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3018) took 1156 ms. Trace:
I20260709 12:27:39.706634 13050 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.868283 12809 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.669229 12970 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.134s
W20260709 12:27:39.878611  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86137.4us
I20260709 12:27:39.614521 12693 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.797288 13007 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:39.889989  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.906268  6453 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
W20260709 12:27:39.907917  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 238644us
I20260709 12:27:39.788044 13039 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:39.660676 12922 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.814864 12817 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.911712  6451 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
W20260709 12:27:39.971868  6307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.002s	sys 0.005s
W20260709 12:27:39.929131  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.869750 13069 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:39.675866 13032 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.851343 13024 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:39.901268 13031 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:39.944916  6452 rpcz_store.cc:276] 0709 12:27:38.698731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.153319 (+454588us) service_pool.cc:224] Handling call
0709 12:27:39.168937 (+ 15618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.169452 (+   515us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:39.183849 (+ 14397us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.560995 (+377146us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.582386 (+ 21391us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.582596 (+   210us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:39.582643 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.582732 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.855136 (+272404us) raft_consensus.cc:1685] finished
0709 12:27:39.855163 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.855327 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2384,"prepare.queue_time_us":14971,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":4750,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":32848,"thread_start_us":352201,"threads_started":1,"wal-append.queue_time_us":407650}
I20260709 12:27:39.955960  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:39.958140  6453 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:27:39.966696  6301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.006s	sys 0.000s
W20260709 12:27:39.999934  6301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.006s	sys 0.000s
I20260709 12:27:39.912317  6381 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:27:39.997453  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:39.998059  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47342.8us
I20260709 12:27:40.003432  6453 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:27:40.007586  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51659.3us
W20260709 12:27:39.997424  6349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.002s	sys 0.014s
I20260709 12:27:40.012902  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.014676  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:40.017208  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:40.002247  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:39.931711 12909 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:39.911929 12706 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:39.952387 13007 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.945935 13001 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:39.878616 13058 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:40.002105 12845 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.000s
I20260709 12:27:40.037314  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:40.037660  6459 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.000s
I20260709 12:27:40.038319  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.038844  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:39.922001 13040 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.076001  6459 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 12.
I20260709 12:27:40.074429  7318 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:40.076840  6302 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:40.077538  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:40.080312  6458 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:40.080722  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:40.017201 12809 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:40.089574  7331 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:40.089589  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39690 (request call id 3) took 1383 ms. Trace:
I20260709 12:27:40.024983 13094 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:40.090795  6453 rpcz_store.cc:276] 0709 12:27:38.700169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.022669 (+1322500us) service_pool.cc:224] Handling call
0709 12:27:40.083312 (+ 60643us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:40.038185 13007 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.953708 12986 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:40.095377  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 205493us
I20260709 12:27:40.018680 13015 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.095921 12606 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:39.999501 12817 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.095961 12423 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:40.120582  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 4e75b2c5204044d2847e78c187388fd3: Illegal state: Replica 4e0e521cb08c420f981a29576eee90a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:40.080010 13063 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:40.015543 13039 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.128489  6302 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:40.085525 12961 raft_consensus.cc:2764] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:27:40.087143 12809 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.095765 12897 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:40.088653 12997 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:40.074837 13002 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:27:40.080245 12845 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.196905 12845 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:40.199540 13080 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:27:40.202682 12986 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.203749  6309 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:27:40.204622 13040 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.204970 12991 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.161460 13031 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:40.166951 12809 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.207121 12817 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.167249 12974 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:40.168201 13092 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:27:40.210961 13043 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.168927 13039 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.116314 12972 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:40.160936  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60496.6us
I20260709 12:27:40.212530 12972 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:40.212987  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:40.213447 12972 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.219224 13099 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:27:40.219820 13099 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:40.220882 13099 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.206167 13031 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:40.223886 13031 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:40.232303  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 192403us
W20260709 12:27:40.179430  6385 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.001s
I20260709 12:27:40.167006 12971 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:40.181422 13086 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:40.182161 13090 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:40.191632 13050 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:40.125149 12909 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:40.193284 13079 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:27:40.193552 13032 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.194334 12983 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:40.124898 13094 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:40.235841  6382 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.017s	sys 0.033s
W20260709 12:27:40.259910  6382 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.017s	sys 0.034s
I20260709 12:27:40.233098 13071 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.153884 12606 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.237735  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:40.239714  6385 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:40.245368  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3432) took 1883 ms. Trace:
I20260709 12:27:40.261401  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.159682 13012 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:40.251951  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:40.261117  6378 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:40.262099  6456 rpcz_store.cc:276] 0709 12:27:38.362169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.071038 (+708869us) service_pool.cc:224] Handling call
0709 12:27:39.071423 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.715851 (+644428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.245119 (+529268us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b54001621d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:40.245304 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444902784}
I20260709 12:27:40.232276 12897 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:40.233016 12845 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:40.096453 13069 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.271389  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:40.250887 12971 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:40.268249 13061 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.928s	user 0.014s	sys 0.000s
I20260709 12:27:40.279742  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:40.280491  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.263660 12817 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.273159 12840 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.881s	user 0.007s	sys 0.007s
I20260709 12:27:40.274904 13050 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:40.270413 12909 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:40.270615 13012 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.290419 12897 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:40.315624 13087 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:27:40.316671 13012 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.271710 13071 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.294014  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:40.290712 12971 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.274962  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:40.272799 13090 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:40.290594 13068 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.877s	user 0.005s	sys 0.007s
I20260709 12:27:40.288226 13069 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.256364 13086 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.317747  6307 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:40.291637 13047 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.573s	user 0.012s	sys 0.000s
I20260709 12:27:40.290310 13002 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:40.309855  6350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.008s	sys 0.008s
I20260709 12:27:40.302232 13024 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.333056  6301 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:40.312062  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
W20260709 12:27:40.343211  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68462.4us
I20260709 12:27:40.253291 13031 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.288456 12991 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:40.352969 12991 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:40.354058  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:40.353958 12991 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.299098 12693 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.296386 12706 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:40.354728  6385 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:40.354662 13032 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.355545 13032 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.356437  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:40.296446 13104 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:40.319157  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 481d3efac77d41e6bba0961a140626dc: Illegal state: Replica 4e0e521cb08c420f981a29576eee90a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:40.357699 13032 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.358143  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:40.349458  6350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.730s	user 0.008s	sys 0.008s
I20260709 12:27:40.359711 13032 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:40.295599 13088 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:40.280062 12845 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.359970  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:40.361269 13088 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.361826 13050 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:40.362396 13050 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:40.363255 12845 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.364013  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:40.364535 13104 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.365644 12423 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:40.363303 13050 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.366218 12423 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:40.367352 12423 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.365123 13104 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.372418 13104 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.282321 13028 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:27:40.318439 12971 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:40.385656 13108 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:40.386123 13086 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.386080 13084 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:40.342604 13071 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.318692 13090 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.392499 13096 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:40.347338 12922 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:40.394765 13108 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.462194 12712 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:40.462705 12712 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:40.423046  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3026) took 1269 ms. Trace:
I20260709 12:27:40.464105  6450 rpcz_store.cc:276] 0709 12:27:39.154152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.353163 (+199011us) service_pool.cc:224] Handling call
0709 12:27:39.353492 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.355146 (+  1654us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:39.424847 (+ 69701us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.170103 (+745256us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b18000ee060. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:27:40.266311 (+ 96208us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.268709 (+  2398us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.278449 (+  9740us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.278833 (+   384us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:40.282385 (+  3552us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.287079 (+  4694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.309166 (+ 22087us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.422622 (+113456us) raft_consensus.cc:1685] finished
0709 12:27:40.422661 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.422944 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2845,"prepare.queue_time_us":74735,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":113,"raft.queue_time_us":5097,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":45807,"spinlock_wait_cycles":3870976,"thread_start_us":92510,"threads_started":1,"wal-append.queue_time_us":851801}
I20260709 12:27:40.296150 13094 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.382743 12706 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:40.431015 13031 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:40.432283  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.351460 13047 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.466894 13108 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.477197  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:40.478175 13047 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:40.303041 12817 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.478991 13097 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.479552 13097 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.480814 12817 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.481544 12817 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:40.481842 13097 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.482014 12817 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:40.383374 13024 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.451151 13084 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:40.454183 13089 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.294s	user 0.001s	sys 0.000s
I20260709 12:27:40.456221 12834 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:40.472194  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:40.473264  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:40.476732  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.308481 12840 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.348788 13068 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.483345 12986 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:40.486599 12956 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:40.492264  6308 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:40.493000  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:40.482659 13040 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:40.546072  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.568869  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:40.570727  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 77821.9us
I20260709 12:27:40.473656  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:40.288336 13061 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.350966 13083 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:40.476444  6376 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:27:40.508142  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:40.550092  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.578657  6301 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:40.581063  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.556257  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:40.585078  6377 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:40.557714  6457 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:40.548192 13032 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.548413 12834 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:40.571622  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:40.545027 13108 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.576138  6380 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:40.576327  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.548923  6381 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:40.604287  6452 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:40.493607 13097 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:27:40.495662 13007 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.583670  6303 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:40.548062 13018 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:40.463970 13100 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:40.589178  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:40.572475  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 218407us
I20260709 12:27:40.593986  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:40.595387  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.599117  7329 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:40.599593  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:40.603009  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:40.616142  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:40.616698 12834 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.617540  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.617650  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.619130  6452 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:40.619777  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.577322 13105 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:40.620572  6452 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:40.621156  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2867) took 1021 ms. Trace:
I20260709 12:27:40.468297 12712 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.612308  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3438) took 1443 ms. Trace:
I20260709 12:27:40.621548  6378 rpcz_store.cc:276] 0709 12:27:39.599762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.599979 (+   217us) service_pool.cc:224] Handling call
0709 12:27:40.621014 (+1021035us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54001f4fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:40.621106 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844155776}
I20260709 12:27:40.625890  6452 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [14.10-16.11]
I20260709 12:27:40.470770 12706 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.625828  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.559065 12809 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.628943  6455 rpcz_store.cc:276] 0709 12:27:39.168506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.357100 (+188594us) service_pool.cc:224] Handling call
0709 12:27:39.424394 (+ 67294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.445073 (+ 20679us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.445107 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.478717 (+ 33610us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.667672 (+188955us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:27:39.667834 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.668079 (+   245us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.828991 (+160912us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b48000cccd8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:27:40.453408 (+624417us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:40.611964 (+158556us) raft_consensus.cc:1685] finished
0709 12:27:40.612000 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.612212 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62556,"raft.queue_time_us":23376,"raft.run_cpu_time_us":3612,"raft.run_wall_time_us":346286,"spinlock_wait_cycles":141794048,"thread_start_us":98224,"threads_started":1,"wal-append.queue_time_us":497200}
I20260709 12:27:40.590648 13083 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:40.607523 13001 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:40.665127  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50664.2us
I20260709 12:27:40.573977 12897 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:27:40.661784 12954 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:40.490221 13096 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.629668 12834 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:40.633078 13075 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:40.604897 13032 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.610534 13070 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 12:27:40.701854  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71954.2us
I20260709 12:27:40.675516 13001 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.675429 12954 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:40.708871  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:40.674434 13097 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:40.679491 12423 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.675958 12991 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.676765 13071 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.677230 12712 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.710032  6459 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:40.614701 13108 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.623610 13018 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:40.721105  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47975.4us
I20260709 12:27:40.727349 13024 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.729693 12809 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.734767 13083 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:40.736307 13098 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.538s	user 0.005s	sys 0.003s
I20260709 12:27:40.628371 13100 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.711074 13001 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:40.713027 12954 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:40.726186  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:40.698890 13040 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:27:40.713788 13097 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:40.716499 13075 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:40.723706 12606 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:40.790448  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 64554.5us
W20260709 12:27:40.771930 13128 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:40.709523 13020 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
W20260709 12:27:40.827908  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 43930.7us
W20260709 12:27:40.774930 13103 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.477s	user 0.000s	sys 0.005s
I20260709 12:27:40.685456 12817 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:40.801112  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.775664 13125 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:27:40.775719 12706 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.777199 13108 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.773883 13032 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:40.872843  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44176us
I20260709 12:27:40.768642 13133 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.723366 12423 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.739163 12991 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.898789 12840 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.899302 13120 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:40.900095  7331 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:40.900266 12991 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.744004 13071 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:40.901018 13123 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:27:40.901275  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:40.890631  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:40.902453 13122 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:27:40.898676  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:40.875442 12922 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:40.891386  6308 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:40.908420  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:40.909024  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:40.891544  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.864024 12897 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:40.877636 12909 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.002s
I20260709 12:27:40.896801  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.903409  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:40.891004  7335 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:40.918656  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:40.903851  7319 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:40.921983  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:40.922657  6307 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:40.923560  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:40.924460  6376 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:40.902419 13071 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.927414  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:40.928081  6377 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:40.929750 13069 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.578s	user 0.004s	sys 0.004s
W20260709 12:27:40.930642 13129 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:27:40.931787  6300 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:27:40.932533 12423 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:40.916163 12922 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.916421 12991 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:40.919656 12897 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.935177 12991 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.936753 12897 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.937028  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:40.937328 12991 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.929644 13064 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:40.938131 13064 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:40.939229 13124 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:27:40.940447 13064 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.943012 13137 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:40.935197 12922 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.943693 13137 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.906891 13131 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.156s
I20260709 12:27:40.925648 13007 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.964706  6377 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:40.967243  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:40.968281  6377 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:27:40.969190  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:40.969960  6377 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:40.970825 12897 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.971448 12897 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.973646 12897 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.921200 13104 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.981176  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:40.982302  6308 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:40.984328  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:40.985433  6308 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:40.956545 12606 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:40.956174 13137 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:40.886173 13075 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.989255  6384 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:40.999809  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:40.987576 12817 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:40.990245 13141 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.946353 12954 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:40.996999 13100 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:40.907928 13097 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:40.904611 13133 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.003075  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.004422  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.003116  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.005932  7332 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:41.007872  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:41.015978  7331 raft_consensus.cc:2376] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18. Re-sending same reply.
I20260709 12:27:41.017088  7317 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:40.983618 13104 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:41.021239  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.022280  7332 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:41.029538 13133 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.002553 12606 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.004025 13141 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.004729 12817 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.007766 13071 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:41.074834 13071 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.077045 13071 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.015432 13100 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:41.044209 13104 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.078033 13100 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.080916 13100 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.081456 13058 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.082046 13134 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.082690 13134 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:41.083117 13117 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.597s	user 0.005s	sys 0.000s
I20260709 12:27:41.084893 13134 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.085429 13043 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:41.085958 13043 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.089722 13043 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:41.090509  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2877) took 1481 ms. Trace:
I20260709 12:27:41.090975  6382 rpcz_store.cc:276] 0709 12:27:39.609484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.618206 (+  8722us) service_pool.cc:224] Handling call
0709 12:27:39.622296 (+  4090us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:40.260290 (+637994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.260322 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:40.260933 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 12:27:40.360285 (+ 99352us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:40.360326 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.360402 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.082428 (+722026us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:41.090090 (+  7662us) raft_consensus.cc:1685] finished
0709 12:27:41.090118 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.090393 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":288689,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":22982,"spinlock_wait_cycles":135715584,"thread_start_us":632058,"threads_started":2,"wal-append.queue_time_us":212303}
I20260709 12:27:41.034466  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.036562  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.040241  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.093652  7335 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:41.094292  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:41.094713  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60345.4us
I20260709 12:27:41.095028  6381 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:41.095161  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.095595 13104 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:41.095849  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:41.096735  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.095914  7335 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:41.036540  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:41.047595  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:41.097016  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.049240  6452 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:41.051098  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:41.052062  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:41.098734  6451 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:41.036454  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:41.053447  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.100980  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:41.099081  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.046003  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.051427  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:41.101970  6382 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:41.037667  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.102805  7317 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:27:41.051749  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:41.052709  6307 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:27:41.036809  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.071609  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.104893  6307 pending_rounds.cc:85] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:27:41.105212  6307 pending_rounds.cc:107] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:27:41.105530  6307 raft_consensus.cc:2889] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:41.106879  7316 raft_consensus.cc:2410] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:41.071785  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.093683 12693 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:41.040948  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.098861  6455 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:41.128964  6378 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:41.100203  7320 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:41.109452  6453 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:41.108207  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.052825  6456 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:27:41.132251  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.132571  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.078090  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.076993 12817 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.134507  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.051677 12954 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:41.133627  6381 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:41.137473  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:41.098685 13097 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:41.139268  6308 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:27:41.140949  6379 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
I20260709 12:27:41.139220  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.136404  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:41.142289  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43939.5us
I20260709 12:27:41.144699  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.145558  6378 raft_consensus.cc:2410] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:41.146158  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3444) took 1543 ms. Trace:
I20260709 12:27:41.146695  6458 rpcz_store.cc:276] 0709 12:27:39.603093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.920847 (+317754us) service_pool.cc:224] Handling call
0709 12:27:40.080105 (+159258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.080744 (+   639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.145983 (+1065239us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54002c9050. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:41.146084 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365395200}
I20260709 12:27:41.143998  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.130671 13090 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:41.151221  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:27:41.073161 13047 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:27:41.142666 12817 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:41.155606  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:41.158821  6381 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:41.137804 12909 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:27:41.159898  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.158070 13099 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:41.152349 13090 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:41.163350 13118 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.683s	user 0.004s	sys 0.004s
I20260709 12:27:41.107590 13031 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:27:41.165042  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:41.311391  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.335951  6382 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:41.337157  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 181623us
I20260709 12:27:41.345191  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.145722 13120 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.149148 12834 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:41.358552  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:41.356062  6381 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: []
W20260709 12:27:41.353135  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 203027us
I20260709 12:27:41.347463  6377 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
W20260709 12:27:41.149089 12972 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.018s	sys 0.023s
I20260709 12:27:41.371121  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1071) took 3041 ms. Trace:
W20260709 12:27:41.372459  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 207494us
I20260709 12:27:41.373950  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.197661 13132 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:41.372012  6454 rpcz_store.cc:276] 0709 12:27:38.314991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.060599 (+745608us) service_pool.cc:224] Handling call
0709 12:27:39.323211 (+262612us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:39.323580 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.323611 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:39.324186 (+   575us) log.cc:844] Serialized 52 byte log entry
0709 12:27:39.565425 (+241239us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:39.565493 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.565605 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.316903 (+751298us) raft_consensus.cc:1685] finished
0709 12:27:40.316938 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.347578 (+1030640us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400163ad0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:41.356185 (+  8607us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1339,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"spinlock_wait_cycles":603135744,"thread_start_us":240783,"threads_started":1,"wal-append.queue_time_us":380631,"wal-append.run_cpu_time_us":12708,"wal-append.run_wall_time_us":1624373}
W20260709 12:27:41.386649 13138 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
W20260709 12:27:41.378819 12972 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.162s	user 0.018s	sys 0.023s
W20260709 12:27:41.388284 13147 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:27:41.394728 13135 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:41.395483 13028 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:41.398591  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:41.404975  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.337276 13007 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.238s
I20260709 12:27:41.408191 13141 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.351096 12845 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:41.412793 12834 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:41.356513 13090 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.348752 13099 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.449163 13099 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.376993 13120 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:41.387714 13146 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 12:27:41.422884  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 64505us
W20260709 12:27:41.423846  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50011.7us
I20260709 12:27:41.462379  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:41.380589 13100 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:41.438181  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.438926 13135 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.438810  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.437907  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:41.439762 12817 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:41.470327  7332 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.467924 13132 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:41.439077 13028 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:41.445609 12845 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:41.437382 13144 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:41.455155  6379 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 11. Current term is 12. Ops: []
I20260709 12:27:41.418955  6381 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:41.457643  6382 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:41.441092  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:41.442745 13094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.037s	sys 0.062s
I20260709 12:27:41.466118  7316 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:27:41.466812  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:41.501402  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:41.501690  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136224us
I20260709 12:27:41.504611  7317 raft_consensus.cc:2410] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:27:41.471436 13132 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:41.536020  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.510604  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.539916  6304 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:41.471601 13108 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:41.508747 13132 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:41.546142  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.472827 13135 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:41.520397  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.553182  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:41.547770 13028 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.560081  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.504469 12845 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.556051  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.538193 12817 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:27:41.514400 13094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.038s	sys 0.063s
I20260709 12:27:41.572937  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1013) took 2153 ms. Trace:
I20260709 12:27:41.561003 13028 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:41.565558 13071 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:41.503909 12972 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.570163  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.503944 13144 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:41.572927  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.573668  6309 rpcz_store.cc:276] 0709 12:27:39.419465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.419710 (+   245us) service_pool.cc:224] Handling call
0709 12:27:39.442395 (+ 22685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.204322 (+761927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.572642 (+1368320us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54002df5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:41.572850 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750226816}
I20260709 12:27:41.570535 13100 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:41.586967 13071 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.570021 12968 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:27:41.562844 13108 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.587332 13144 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:41.608654  7332 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:41.610143  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:41.610389  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.564023 12834 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.590528 12972 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:41.618521  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.620405  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.616816 12972 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.625895  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.628376  6454 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:41.627326  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.632681  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.630455 13084 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.631932  6309 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:41.638680  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:41.640691  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.640750  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.642556 12983 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:41.643083 12983 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:41.643312 13119 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.158s	user 0.013s	sys 0.025s
I20260709 12:27:41.643497 12983 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:41.644279  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.158s (SENT) [suppressed 29 similar messages]
I20260709 12:27:41.645756 13088 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.646342 13088 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:41.641790  6309 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:41.656229  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.657471  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.622893 12809 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:41.661032  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:41.660895  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.664608  6305 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.640903 13024 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:41.641913 13075 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.683346  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.655853 13097 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.696416  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:41.678965 13040 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.001s	sys 0.019s
I20260709 12:27:41.670390 13086 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:41.700222  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:41.699033  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54775.1us
W20260709 12:27:41.694743 13001 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.869s	user 0.008s	sys 0.005s
I20260709 12:27:41.708967  6309 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:41.709565 13094 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.673944 12809 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.711437  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.713387  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:41.690189 13120 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:41.707086 13040 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.001s	sys 0.020s
I20260709 12:27:41.725024  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2879) took 1253 ms. Trace:
I20260709 12:27:41.722623  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:41.734709  6305 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:41.735338  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.732338  6385 rpcz_store.cc:276] 0709 12:27:40.471461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.473354 (+  1893us) service_pool.cc:224] Handling call
0709 12:27:41.724813 (+1251459us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b5400038754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:41.724944 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":745528064}
I20260709 12:27:41.676960 12909 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:27:41.742051  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.727025 13071 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:41.742398 13007 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:41.713624 13001 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.747527 13007 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.749065 13001 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.749956  7319 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:41.750123 13007 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.690514 13075 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.751816  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:41.755674  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.713380 13086 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.776081 13099 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.778288 12970 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.731189 13094 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.779111 12970 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.779642 12986 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.780264 12986 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.780835 13094 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.781411 13094 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.781785 12970 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.736701 13115 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:41.726300 13068 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.716149 12809 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.742700 13047 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:41.784127 13047 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.785120  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.785784  7332 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:41.736552 13153 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.087s
I20260709 12:27:41.786686  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.796837  6305 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 7. (term mismatch)
W20260709 12:27:41.797101  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.797384  6305 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 6
I20260709 12:27:41.797762  6305 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 15.7
I20260709 12:27:41.798064  6305 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:41.800838  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59032.2us
I20260709 12:27:41.800755  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2882) took 1328 ms. Trace:
I20260709 12:27:41.802300  6380 rpcz_store.cc:276] 0709 12:27:40.472329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.472989 (+   660us) service_pool.cc:224] Handling call
0709 12:27:41.800533 (+1327544us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:41.800691 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58469760}
I20260709 12:27:41.803471  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:41.739171 13101 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.516s	user 0.015s	sys 0.034s
I20260709 12:27:41.699043 13097 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.813001  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.819453  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.820226  6308 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:41.821237  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:41.771078 13095 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.547s	user 0.009s	sys 0.004s
I20260709 12:27:41.736658 13158 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:41.827378  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.827812  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.828418  6385 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:41.830901  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.782387 12986 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:41.817889 13101 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.594s	user 0.015s	sys 0.034s
I20260709 12:27:41.812419  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1199) took 1968 ms. Trace:
I20260709 12:27:41.794073 12970 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.834787  7329 rpcz_store.cc:276] 0709 12:27:39.824589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.824818 (+   229us) service_pool.cc:224] Handling call
0709 12:27:39.825220 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:40.618583 (+793363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.792186 (+1173603us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b54006801d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:41.792364 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1779068288}
I20260709 12:27:41.837385  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:41.824210  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:41.778338 13099 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.810469  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:41.812142 13007 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.834394  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:41.769989 13108 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.727984 12845 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.838600  6308 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.814368  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.721676 12983 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.854466  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.826510 13047 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:41.851274 12970 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.781831 13094 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.768867 13132 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:41.857942  6304 raft_consensus.cc:2368] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21.
I20260709 12:27:41.864054  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.865391  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:41.872058  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2714) took 2441 ms. Trace:
W20260709 12:27:41.863076  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52719.5us
I20260709 12:27:41.867924  6308 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:41.888983  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.892716  6305 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:41.895066  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:41.895593  6305 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:41.895886  6307 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:27:41.893241  7316 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:41.921046  6305 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:27:41.922678  6305 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:41.852710 13009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.384s	user 0.009s	sys 0.013s
I20260709 12:27:41.924219  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:41.933317  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 291047us
I20260709 12:27:41.996044  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.996157  7317 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:41.893471  6310 rpcz_store.cc:276] 0709 12:27:39.431058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.443082 (+ 12024us) service_pool.cc:224] Handling call
0709 12:27:39.443382 (+   300us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:39.601429 (+158047us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.601466 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:39.612638 (+ 11172us) log.cc:844] Serialized 52 byte log entry
0709 12:27:39.931866 (+319228us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:39.932290 (+   424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.932908 (+   618us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.770424 (+1837516us) spinlock_profiling.cc:243] Waited 6.62 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:41.871668 (+101244us) raft_consensus.cc:1685] finished
0709 12:27:41.871706 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.871977 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30041,"raft.queue_time_us":112606,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":26335,"spinlock_wait_cycles":15334656,"thread_start_us":256362,"threads_started":2,"wal-append.queue_time_us":378379}
I20260709 12:27:41.925261  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:41.998395  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73213.3us
I20260709 12:27:41.925817  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:41.999738  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:42.000026  7329 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:42.002244  7316 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
W20260709 12:27:42.003674  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:41.924053 13009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.456s	user 0.010s	sys 0.013s
I20260709 12:27:42.008311  6307 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:42.008881  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 153588us
I20260709 12:27:41.858913 13063 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:41.863116 12809 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:41.867825 13097 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.870733 13094 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:41.877738 13061 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:42.051208  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2716) took 2608 ms (client timeout 2158 ms). Trace:
I20260709 12:27:42.027550 12961 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:42.032249  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:42.008061 13050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.636s	user 0.029s	sys 0.046s
W20260709 12:27:42.067516 13050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.695s	user 0.029s	sys 0.047s
I20260709 12:27:42.068219 13050 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:42.068616 13050 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:42.068981 13050 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.041656  6301 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:27:42.006414  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.039527  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:42.073479  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.075184  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.076259  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:42.005213 13137 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.862849 13075 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:41.858980 13024 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:41.857625 12845 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.868748 13018 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:41.860829 12983 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:41.867444 12970 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:42.054133  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:42.106014  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51943us
I20260709 12:27:42.107456  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:42.035993 13096 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:42.051736  6303 rpcz_store.cc:269] 0709 12:27:39.443774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.446130 (+  2356us) service_pool.cc:224] Handling call
0709 12:27:42.041117 (+2594987us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001b2bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:42.041254 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2517067136}
I20260709 12:27:41.860980 13090 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.113346  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:41.855252 13007 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:41.925261 13135 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.058185  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2722) took 1809 ms. Trace:
I20260709 12:27:42.003758 13108 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:42.143215  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet bdf58974e308489d9ae9a07e0a775c71: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:27:42.139402  6302 rpcz_store.cc:276] 0709 12:27:40.248455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.248577 (+   122us) service_pool.cc:224] Handling call
0709 12:27:40.263824 (+ 15247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.482956 (+219132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.483003 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.483590 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.582227 (+ 98637us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:40.582288 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.040227 (+1457939us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b5400140d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:42.040281 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.040347 (+    66us) raft_consensus.cc:1685] finished
0709 12:27:42.040365 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.058062 (+ 17697us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1511,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"spinlock_wait_cycles":502888832,"thread_start_us":93182,"threads_started":1,"wal-append.queue_time_us":148791,"wal-append.run_cpu_time_us":3603,"wal-append.run_wall_time_us":1242570}
I20260709 12:27:41.997664 13104 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:27:41.993530 13028 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:27:42.155340  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:42.155929  6456 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.009871 13112 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:42.004058 13099 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:27:42.078459  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:42.157857  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79456.8us
I20260709 12:27:42.078702 13084 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:42.090689  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:42.092135  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.103809 13120 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:27:42.109889  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:42.110603  6382 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:42.096340  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:41.997542 12693 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:27:42.013605 13086 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:42.056313 13168 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:42.114202  6450 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:42.138032  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.140612  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.141165  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.202579 12606 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.141306  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:42.104259 12834 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.104982 13024 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:42.104082 13097 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.117365 13094 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:42.148263 13007 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:42.086092  7317 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:42.079722 13068 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:42.160468  6457 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:27:42.103564  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.162178  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.252185  6459 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:42.164124  6452 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:42.180428  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:42.113183 13154 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.112943 13121 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:42.220111 13165 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:27:42.184690 13145 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:27:42.186141  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.260274  6385 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.186340  6378 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:42.187656 13132 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:42.189386 13040 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:42.189666  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:42.189991 12974 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:42.190750  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:42.218884 13112 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:42.270735  6302 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:42.163658 13084 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.190759  6456 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:42.080369 13144 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.297235 13144 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:42.159412 13108 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:42.298018 13108 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:42.078966 13137 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.304605 13137 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.214874  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3150) took 1292 ms. Trace:
I20260709 12:27:42.305152  6377 rpcz_store.cc:276] 0709 12:27:40.922877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.139940 (+217063us) service_pool.cc:224] Handling call
0709 12:27:41.153319 (+ 13379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.375151 (+221832us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.214530 (+839379us) spinlock_profiling.cc:243] Waited 194 ms on lock 0x7b5400109ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:42.214792 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":445780224}
I20260709 12:27:42.210807  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:42.306872  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 104671us
I20260709 12:27:42.308245  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.272157  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:42.218801  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:42.219887  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54063.4us
I20260709 12:27:42.171355 13018 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:42.242156  6380 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:42.246376  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:42.310784  6309 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:42.210386  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:42.184787  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:42.312645  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 102369us
I20260709 12:27:42.313232  6385 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
W20260709 12:27:42.257156  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 246867us
I20260709 12:27:42.220819 12970 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:42.219688 13101 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:42.219340 13099 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:42.270037 13040 consensus_queue.cc:1243] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:42.221228 13001 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 12:27:42.224571 13168 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.211740 13032 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 12:27:42.280364  6307 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:42.283289 13112 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.271225 13121 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.267501 13154 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.290069 12834 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.259501 13094 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:42.294461 13135 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.315285  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:42.315796  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:42.365339  7316 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:42.366065  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 989) took 1349 ms. Trace:
I20260709 12:27:42.366693  7319 rpcz_store.cc:276] 0709 12:27:41.002658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.002811 (+   153us) service_pool.cc:224] Handling call
0709 12:27:42.351463 (+1348652us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:42.351688 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1713948544}
I20260709 12:27:42.318724  6309 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:27:42.317163  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:42.369091  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 52041.8us
I20260709 12:27:42.369210 13040 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:42.322129  6302 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:27:42.370002 13040 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:42.309366  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:42.370898  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61615.4us
I20260709 12:27:42.371201 13040 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.373414 13137 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.327230  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:42.374097 13137 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.374888  6303 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.277026  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 2972) took 1385 ms. Trace:
I20260709 12:27:42.376264  6300 rpcz_store.cc:276] 0709 12:27:40.891147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.903194 (+ 12047us) service_pool.cc:224] Handling call
0709 12:27:42.276799 (+1373605us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:42.276937 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29517568}
I20260709 12:27:42.331619  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:42.326949 13061 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:42.378082 13061 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:42.310901 13084 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:42.330798 13099 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:42.372186  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.389091 13137 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.398347  6308 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:42.399813  6303 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:42.427706  6308 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:42.403024  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3160) took 1436 ms. Trace:
I20260709 12:27:42.378508 13061 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:42.405195 12970 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:42.409530 13127 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.626s	user 0.012s	sys 0.024s
I20260709 12:27:42.352845  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:42.412909  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48290.8us
I20260709 12:27:42.328102 13101 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:42.428820  6308 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.440219  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.327916 13164 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:42.335239 13145 consensus_queue.cc:1243] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:27:42.417435  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 244222us
W20260709 12:27:42.418836  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 87313us
I20260709 12:27:42.430042  6378 rpcz_store.cc:276] 0709 12:27:40.966481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.157819 (+191338us) service_pool.cc:224] Handling call
0709 12:27:41.159759 (+  1940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.266837 (+1107078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.402735 (+135898us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003d3bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:42.402941 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2360476416}
I20260709 12:27:42.404624  7316 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:42.413769 13007 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:42.329896 13024 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.441037  6308 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:42.447408  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.443539  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.421785 13018 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:42.426991 12968 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:42.456514  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.457957  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:42.458884  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:42.443442 12974 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:27:42.461297 13096 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:42.436298 12970 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:42.442281 13164 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:42.474874  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:42.476569  6378 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:42.421104 13115 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:27:42.482281  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:42.470124  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.499612  6382 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:42.461839 12834 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:42.474063 13144 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
I20260709 12:27:42.468187  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:42.484838  6305 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.486959 13168 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:42.488339 13132 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:42.503328 13132 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.504626 13168 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:42.505208 13168 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:42.505422 13132 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.490001 13145 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:42.505900 12909 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:42.506305 13145 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:42.506829 13145 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:42.507081  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2977) took 1558 ms. Trace:
I20260709 12:27:42.507789 13145 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.509167 13145 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.507541  6301 rpcz_store.cc:276] 0709 12:27:40.948714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.951081 (+  2367us) service_pool.cc:224] Handling call
0709 12:27:42.041376 (+1090295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.070724 (+ 29348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.506867 (+436143us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54001b2bd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:42.507021 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2506633472}
I20260709 12:27:42.517334  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:42.519017 13173 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:27:42.520290  7318 pending_rounds.cc:85] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 13
I20260709 12:27:42.520413  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:42.478917 13090 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.474454 13031 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:42.454730 13020 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:27:42.464512 13018 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.556470 13018 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:42.442260 13175 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:42.557747 13175 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:42.504660 13108 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.501772 12834 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:42.559674 13096 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.465258 13094 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.560351 13096 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.560843 13108 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.499259  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1023) took 1345 ms. Trace:
I20260709 12:27:42.520656  7318 pending_rounds.cc:107] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:27:42.443595 13007 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:42.443882 12961 raft_consensus.cc:2764] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:42.563093 13007 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:42.563490 12961 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:42.534891  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.564152  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2881) took 2092 ms. Trace:
I20260709 12:27:42.558305 13175 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.565287  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:42.566522  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:42.567097  6380 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:42.444375 13099 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.568782 13099 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:42.570799  6376 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:42.571576  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89395.7us
I20260709 12:27:42.572269 13101 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.573134  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:42.573448 13101 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:42.574057 13135 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:42.535568  6385 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:27:42.574826 13135 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:42.575183  6385 pending_rounds.cc:85] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 7
I20260709 12:27:42.575802  6385 pending_rounds.cc:107] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:27:42.562374 13096 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.543098  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:42.544605 13169 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:27:42.576713  6378 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:42.579049 13061 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.580631  7320 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:42.583034 13061 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:42.583727 13121 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:42.585430 13024 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.575312 13135 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:42.586213 13135 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.479827 13164 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.506506 12909 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:42.553062 12970 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.559600 12834 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:42.576192  6385 raft_consensus.cc:2889] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.562835  7318 raft_consensus.cc:2889] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.595355 13090 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.597630 12834 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.562731  6304 rpcz_store.cc:276] 0709 12:27:41.154259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.552665 (+398406us) service_pool.cc:224] Handling call
0709 12:27:42.499020 (+946355us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b54002076d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:42.499189 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":692093824}
I20260709 12:27:42.544878  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:42.598922 12834 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:42.599057 13031 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:42.600121 13031 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.589442  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:42.602707  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.551247  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.564558  6383 rpcz_store.cc:276] 0709 12:27:40.472030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.473735 (+  1705us) service_pool.cc:224] Handling call
0709 12:27:40.474069 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.474474 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.474523 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.475035 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.610804 (+135769us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:40.610868 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.610971 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.408738 (+1797767us) spinlock_profiling.cc:243] Waited 3.17 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:42.563839 (+155101us) raft_consensus.cc:1685] finished
0709 12:27:42.563865 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.564090 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1236,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":7290240,"thread_start_us":135290,"threads_started":1,"wal-append.queue_time_us":231505}
I20260709 12:27:42.607229  6377 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:42.593578  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:42.594887  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:42.598325 13090 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.611083  6307 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:27:42.595959 12909 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:42.638689  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87519.7us
W20260709 12:27:42.609071 13063 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.562s	user 0.007s	sys 0.004s
I20260709 12:27:42.640558  6454 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:42.506090 13168 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:42.562639 13050 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:42.665470  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 68782.5us
I20260709 12:27:42.666404  6305 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:27:42.595965 13162 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.020s	sys 0.059s
W20260709 12:27:42.659765  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.647334  6382 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:42.463688  6308 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:27:42.692545  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:42.712896  6453 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.696790 13175 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.699633 13007 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:42.713990  6308 pending_rounds.cc:85] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 8
I20260709 12:27:42.723258  6308 pending_rounds.cc:107] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:27:42.723616  6308 raft_consensus.cc:2889] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.725894  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:42.662796 13063 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:42.650995 13145 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:42.747373  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:42.749203  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:42.749460  6451 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:42.722807 13162 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.020s	sys 0.060s
I20260709 12:27:42.713948  6377 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:42.754106  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.764084  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.780685  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.781637  6452 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:42.769109  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:42.770016  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:42.770875  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:42.771518  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.796103  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.797645  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:42.798290  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 426166us
I20260709 12:27:42.798348  6454 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:42.799521  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.800405  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:42.783746  6307 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:42.745906 12840 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
W20260709 12:27:42.817441  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48445.7us
I20260709 12:27:42.833797  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:42.822177  6454 raft_consensus.cc:2410] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:27:42.831305  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:42.836601  6455 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:42.838778  6454 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:42.840404  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.841861  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.861445  6454 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:42.861668  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.862717  6457 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:27:42.741008 13083 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:42.747140 13145 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.818270 13137 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:42.862825  6305 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:27:42.863435  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:42.912309  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48964.3us
I20260709 12:27:42.863724  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.865254  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.028864  6457 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:43.029762  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:42.905730  6453 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:42.815892 13136 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.118s
W20260709 12:27:43.059188 13176 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.213s	user 0.001s	sys 0.000s
I20260709 12:27:43.078830 13171 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:43.079792 13171 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.087857 13007 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:43.089562 13007 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:43.091857 13024 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.002s
I20260709 12:27:42.910753 13145 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.099666 13175 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:43.035658 13028 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:27:42.911269 13083 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.105342 13083 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:42.924903  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2997) took 1118 ms. Trace:
I20260709 12:27:43.106374  6305 rpcz_store.cc:276] 0709 12:27:41.806039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.912574 (+106535us) service_pool.cc:224] Handling call
0709 12:27:41.912887 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.484447 (+571560us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54002dc920. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa322506f 00007f6fa329df60 00007f6fa329f7de 00007f6fa32a9b49 00007f6fa32aa416 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62
0709 12:27:42.665824 (+181377us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b58005a9bd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa329dfe7 00007f6fa329f7de 00007f6fa32a9b49 00007f6fa32aa416 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4
0709 12:27:42.924531 (+258707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.924829 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426797184}
I20260709 12:27:43.108400  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 2998) took 1302 ms. Trace:
I20260709 12:27:43.108949  6310 rpcz_store.cc:276] 0709 12:27:41.806285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.997785 (+191500us) service_pool.cc:224] Handling call
0709 12:27:42.013490 (+ 15705us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.013832 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.013861 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.014299 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.399664 (+385365us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:42.399703 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.399770 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.108091 (+708321us) raft_consensus.cc:1685] finished
0709 12:27:43.108124 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.108322 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2118,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":100,"thread_start_us":385075,"threads_started":1,"wal-append.queue_time_us":712180}
I20260709 12:27:42.795571 13063 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:42.831996 12972 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.746270 13020 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:27:42.830082 12974 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.796942 13115 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.123s
W20260709 12:27:42.911105 13075 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.039s	sys 0.050s
I20260709 12:27:42.909826  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:43.170349  6454 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17.
I20260709 12:27:43.044945 13068 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:42.897852 13181 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:43.045867 13084 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.003s
I20260709 12:27:43.046314  6452 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:43.030985 13018 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:43.126642  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.040s	sys 0.042s
I20260709 12:27:43.156852  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:43.170377 13094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.014s	sys 0.017s
I20260709 12:27:43.154663 12970 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:43.173368  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.018s	sys 0.013s
W20260709 12:27:43.179090  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.040s	sys 0.043s
W20260709 12:27:43.180354  6378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.019s	sys 0.052s
I20260709 12:27:43.179774  6453 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:43.141594  6457 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:27:43.169874 13075 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.040s	sys 0.051s
W20260709 12:27:43.179881  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.005s	sys 0.028s
W20260709 12:27:43.168785  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.010s	sys 0.009s
W20260709 12:27:43.173513 13135 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
W20260709 12:27:43.183557  7318 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.010s	sys 0.009s
W20260709 12:27:43.185843  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 870794us
I20260709 12:27:43.175442 13181 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:43.199736  7318 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.010s	sys 0.009s
W20260709 12:27:43.184299 13040 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.785s	user 0.016s	sys 0.072s
I20260709 12:27:43.126935 13164 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.190116 12974 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:43.189903  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.554s	user 0.005s	sys 0.028s
I20260709 12:27:43.185174  6452 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:27:43.231364  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 595160us
W20260709 12:27:43.232833  6456 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.028s	sys 0.047s
W20260709 12:27:43.179083 13047 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.007s	sys 0.011s
W20260709 12:27:43.212244 13040 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.016s	sys 0.072s
I20260709 12:27:43.180593 13018 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:43.182466 13094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.014s	sys 0.017s
I20260709 12:27:43.154211 12972 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.246196  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.209498 13001 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:43.228638 13090 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:43.251772  6459 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.007s	sys 0.030s
W20260709 12:27:43.252192  6459 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.007s	sys 0.030s
W20260709 12:27:43.252602  6272 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.018s	sys 0.020s
W20260709 12:27:43.252959  6272 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.018s	sys 0.020s
W20260709 12:27:43.253280  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 683728us
W20260709 12:27:43.243009 13047 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.008s	sys 0.012s
I20260709 12:27:43.150821 13063 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:43.246912 12834 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.647s	user 0.012s	sys 0.001s
W20260709 12:27:43.185704  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.019s	sys 0.014s
I20260709 12:27:43.173823 13171 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:43.246184 13018 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:43.278219  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:43.233832  6456 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.028s	sys 0.047s
W20260709 12:27:43.205948 13009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.762s	user 0.034s	sys 0.055s
W20260709 12:27:43.190529  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.011s	sys 0.009s
I20260709 12:27:43.181005 13084 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.237744  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:43.285360  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 764777us
I20260709 12:27:43.203612 13181 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:43.254390  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 454046us
W20260709 12:27:43.206905  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.008s	sys 0.013s
I20260709 12:27:43.200153 13135 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.287361  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:43.193594  6455 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
W20260709 12:27:43.269572  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 954293us
W20260709 12:27:43.186116  6378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.019s	sys 0.052s
W20260709 12:27:43.289995  7286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.024s	sys 0.062s
W20260709 12:27:43.290356  7286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.867s	user 0.024s	sys 0.062s
I20260709 12:27:43.236383 12974 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.240397  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:43.286339  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s	user 0.008s	sys 0.014s
W20260709 12:27:43.298899  6450 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.013s	sys 0.006s
W20260709 12:27:43.299494  6450 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.013s	sys 0.006s
I20260709 12:27:43.306903  6303 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:43.317642 13180 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.288s	user 0.002s	sys 0.000s
I20260709 12:27:43.319944 13153 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:43.325760  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.328413  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3008) took 1008 ms. Trace:
I20260709 12:27:43.329942 13063 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:43.412634  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86994us
I20260709 12:27:43.340469 12972 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:43.340620  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.341648 12834 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:43.263895 13001 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.344836 12974 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:43.349896 13050 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:43.358942 13154 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:43.360481 13061 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:43.436442 13061 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:43.364455 13084 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:43.295688 13099 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.726s	user 0.013s	sys 0.000s
W20260709 12:27:43.366350  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 126070us
I20260709 12:27:43.439412  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.440843  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:43.441452  6456 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:27:43.366429 13199 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:43.442893 13199 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.445720 13199 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:43.368783 13192 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:43.369961  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.447062  6379 raft_consensus.cc:2410] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 16 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
W20260709 12:27:43.373968  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 84961us
I20260709 12:27:43.449641  6452 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:27:43.450547  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:43.285187 13009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.034s	sys 0.056s
W20260709 12:27:43.388038 13186 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:27:43.388971  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:43.397243 12909 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.733s	user 0.008s	sys 0.002s
I20260709 12:27:43.411619 13153 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:43.277105 13112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.037s
I20260709 12:27:43.413579  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.412124  6300 rpcz_store.cc:276] 0709 12:27:42.319560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.383631 (+ 64071us) service_pool.cc:224] Handling call
0709 12:27:42.384019 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:42.391787 (+  7768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.391833 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.392599 (+   766us) log.cc:844] Serialized 52 byte log entry
0709 12:27:42.818034 (+425435us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:42.818087 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.819721 (+  1634us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.328044 (+508323us) raft_consensus.cc:1685] finished
0709 12:27:43.328080 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.328309 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7058,"raft.queue_time_us":7173,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":424975,"threads_started":1,"wal-append.queue_time_us":867757}
W20260709 12:27:43.462442  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48983.9us
I20260709 12:27:43.463685  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.424361  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.424661 13205 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:43.426939 13028 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:43.435417 12809 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:43.446357 13199 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:43.372519 13188 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.145s
I20260709 12:27:43.446280  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 8) took 1190 ms. Trace:
I20260709 12:27:43.454942  6456 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:43.442472  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:43.426697 12840 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.468956  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 983) took 1374 ms. Trace:
I20260709 12:27:43.483523  6381 rpcz_store.cc:276] 0709 12:27:42.097014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.097231 (+   217us) service_pool.cc:224] Handling call
0709 12:27:42.097618 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.097898 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.097934 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.098548 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.319534 (+220986us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:42.319583 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.319672 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.316772 (+997100us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:43.468599 (+151827us) raft_consensus.cc:1685] finished
0709 12:27:43.468633 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.468883 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":45291392,"thread_start_us":220611,"threads_started":1,"wal-append.queue_time_us":696408}
I20260709 12:27:43.437883 13061 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:43.471531  6380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.883s	user 0.008s	sys 0.006s
W20260709 12:27:43.471604  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 184321us
I20260709 12:27:43.469002  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.435240  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:43.462393 13112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.006s	sys 0.038s
I20260709 12:27:43.469324 13195 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:43.488732  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46351.8us
I20260709 12:27:43.481917  6384 rpcz_store.cc:276] 0709 12:27:42.256086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.256226 (+   140us) service_pool.cc:224] Handling call
0709 12:27:43.445908 (+1189682us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400038754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ad735 00007f6fa826dd29 00007f6fa17ec1d4 00007f6fa17ec162 00007f6fa17ec091 00007f6fa17ec00c 00007f6fa17eb2c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:43.446170 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2727587072}
I20260709 12:27:43.496465  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:43.480209 13063 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:43.469997 13192 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:43.474321  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 561206us
I20260709 12:27:43.476769 12809 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:43.433513 12834 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.496289  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:43.499573 13199 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:43.459663 12909 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:43.520305  7331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.013s	sys 0.033s
W20260709 12:27:43.553877  7331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.014s	sys 0.034s
I20260709 12:27:43.544550  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:43.539577 13061 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:43.507103 13205 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:43.491181  6380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.902s	user 0.008s	sys 0.006s
W20260709 12:27:43.546936  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50722.3us
I20260709 12:27:43.505280 13063 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:43.547278  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44335.4us
I20260709 12:27:43.499253 13071 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:43.479597 13028 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:43.555948 13101 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.980s	user 0.012s	sys 0.004s
W20260709 12:27:43.476516 13162 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.006s	sys 0.006s
I20260709 12:27:43.573101 12961 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:43.575903 12834 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:43.559175 13192 raft_consensus.cc:2764] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:27:43.495409 13193 raft_consensus.cc:2764] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:27:43.576520 12809 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.577148 13210 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.577337 13193 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.577684 12809 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:43.577978 13193 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.577692 13210 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.580653 13193 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:43.567082 13199 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:43.573112 13162 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.006s	sys 0.006s
I20260709 12:27:43.583777 13199 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.502826  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:43.585938  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83190us
I20260709 12:27:43.567536 13121 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:43.586669 13071 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:43.587847  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.588570 13192 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:43.589211 13121 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:43.589529 13192 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:43.589879 13121 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:43.590776 13192 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:43.498211 13018 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.592684 13001 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:43.592901 13018 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:43.489454 12840 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.595367 12840 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:43.504561 13153 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.596299 13153 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:43.597183 13194 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:27:43.581171 13210 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:43.598927 13192 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:43.599802 13192 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.600996 13192 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:43.521183  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.601883  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:43.567904 13063 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:43.600281  6198 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 15 to 23. New cstate: current_term: 23 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:43.545866 13196 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:27:43.608160 13040 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:43.619117 13018 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:43.628796 12909 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:43.629894 13007 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:43.630765 13028 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:43.561070 13205 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:43.631542 13197 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:27:43.558197 13164 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:43.637143  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.030s	sys 0.032s
I20260709 12:27:43.637575  6204 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 267) took 7591 ms. Trace:
W20260709 12:27:43.641584  6458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.010s	sys 0.003s
W20260709 12:27:43.667510  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.030s	sys 0.032s
I20260709 12:27:43.650782 13028 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.651183  6198 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:43.647110 13007 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.645694 13162 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.667563  6204 rpcz_store.cc:276] 0709 12:27:36.018017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.018377 (+   360us) service_pool.cc:224] Handling call
0709 12:27:43.608780 (+7590403us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:38.823928 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:38.824767 (+   839us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:38.824816 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:27:38.824845 (+    29us) tablet.cc:662] Decoding operations
0709 12:27:39.030360 (+205515us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:39.030460 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:39.030510 (+    50us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:27:39.032320 (+  1810us) tablet.cc:701] Row locks acquired
0709 12:27:39.032342 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:27:39.032536 (+   194us) write_op.cc:270] Start()
0709 12:27:39.032669 (+   133us) write_op.cc:276] Timestamp: P: 1783600059032461 usec, L: 0
0709 12:27:39.032692 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:27:39.056138 (+ 23446us) log.cc:844] Serialized 8503 byte log entry
0709 12:27:39.619346 (+563208us) op_driver.cc:464] REPLICATION: finished
0709 12:27:40.198895 (+579549us) write_op.cc:301] APPLY: starting
0709 12:27:40.199024 (+   129us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:41.040672 (+841648us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:41.040783 (+   111us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:41.693157 (+652374us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:41.693590 (+   433us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:27:41.693639 (+    49us) write_op.cc:312] APPLY: finished
0709 12:27:41.694132 (+   493us) log.cc:844] Serialized 196 byte log entry
0709 12:27:41.837593 (+143461us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:41.893503 (+ 55910us) write_op.cc:454] Released schema lock
0709 12:27:41.998836 (+105333us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:42.054967 (+ 56131us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:42.055013 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":579181,"apply.run_cpu_time_us":42239,"apply.run_wall_time_us":1850089,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96131,"num_ops":22,"prepare.queue_time_us":116361,"prepare.run_cpu_time_us":11918,"prepare.run_wall_time_us":298014,"raft.queue_time_us":303131,"raft.run_cpu_time_us":79634,"raft.run_wall_time_us":360462,"replication_time_us":566215,"thread_start_us":712922,"threads_started":5,"wal-append.queue_time_us":354669,"wal-append.run_cpu_time_us":41708,"wal-append.run_wall_time_us":2268206}]]}
W20260709 12:27:43.679525  6458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.011s	sys 0.004s
I20260709 12:27:43.681716  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:43.609651 13094 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:43.691219  6307 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:43.646129 13018 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:43.648679 13135 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:43.694422  6451 raft_consensus.cc:606] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:43.702167  6451 raft_consensus.cc:614] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:43.708514  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:43.709872  6456 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:43.715992  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:43.717350  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:43.718832  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.720175  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.724226  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:43.649501 13205 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:43.727986  6457 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:43.729121  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:43.729472  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:43.733794  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet cdd6e8c2e3c642dc806079ce6070d975: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:43.730722  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:43.734858  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:43.633178 13145 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.730990  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:43.732199  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.738085  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:43.738719  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.739321  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:43.740270  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3076) took 1102 ms. Trace:
I20260709 12:27:43.740281  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.734208  6301 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:43.747308  6459 rpcz_store.cc:276] 0709 12:27:42.638052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.641107 (+  3055us) service_pool.cc:224] Handling call
0709 12:27:42.663459 (+ 22352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.663652 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:43.266458 (+602806us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.739953 (+473495us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.740004 (+    51us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.740053 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.740118 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.740217 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":504700,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":844,"thread_start_us":1049634,"threads_started":2}
W20260709 12:27:43.750831  6453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.010s	sys 0.015s
W20260709 12:27:43.751402  6453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.010s	sys 0.015s
I20260709 12:27:43.751672  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:43.693995 13020 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.754915  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3007) took 1498 ms. Trace:
I20260709 12:27:43.687848 13195 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:43.644446 13040 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.761322  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:43.762274  6384 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:43.762791  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:43.788506  6301 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:43.788877  6381 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:43.762858  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:43.763564  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:43.789779  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:43.789733  6382 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:43.790546  6301 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:43.798175  6382 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:43.727447 12834 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:43.768465  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.756350  6300 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:43.761047  6309 rpcz_store.cc:276] 0709 12:27:42.256784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.414086 (+157302us) service_pool.cc:224] Handling call
0709 12:27:42.414542 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.416276 (+  1734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.416323 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.416916 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.496262 (+ 79346us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b180037f7e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:43.295771 (+799509us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:27:43.295841 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.295951 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.754619 (+458668us) raft_consensus.cc:1685] finished
0709 12:27:43.754645 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.754850 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1201,"raft.queue_time_us":1336,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7580,"spinlock_wait_cycles":2669312,"thread_start_us":799246,"threads_started":1,"wal-append.queue_time_us":1034710}
I20260709 12:27:43.775079  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:43.696256 13094 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.763638  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:43.802386  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:43.803359  6379 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:43.693521 13216 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:43.767031  6376 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.010s	sys 0.019s
W20260709 12:27:43.809067  6376 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.010s	sys 0.019s
I20260709 12:27:43.801283  6309 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:43.809190  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:43.726691 13084 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.681121 13028 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:43.780267  6348 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.016s	sys 0.025s
W20260709 12:27:43.817483  6348 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.017s	sys 0.025s
W20260709 12:27:43.817862  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 902423us
W20260709 12:27:43.805342  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70162.1us
I20260709 12:27:43.820135  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.821610  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:43.816577  6309 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:43.818611  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:43.825026  6301 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:43.701442 13162 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.820227  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:43.860286  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:43.812866 13075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.012s	sys 0.010s
I20260709 12:27:43.819492 13214 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:27:43.846987 13215 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:27:43.816428 13216 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.819284 13084 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:43.808297 13094 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:43.813040 12909 raft_consensus.cc:2764] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:43.828159  6301 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:27:43.783602 13020 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:43.883662  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.861949 13219 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:43.863427 13068 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:43.820402 13171 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:43.887522  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 298371us
W20260709 12:27:43.903606  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67714.4us
I20260709 12:27:43.886369  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1006) took 1572 ms. Trace:
I20260709 12:27:43.888749  6308 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:43.868184  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:43.814738 12834 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:43.816311 13101 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 12:27:43.886727 13075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.013s	sys 0.010s
W20260709 12:27:43.912685  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.032s	sys 0.023s
I20260709 12:27:43.911000  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:43.994345  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 83470.7us
W20260709 12:27:43.905421 13209 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.336s	user 0.007s	sys 0.007s
W20260709 12:27:43.906096 13208 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.334s	user 0.000s	sys 0.001s
W20260709 12:27:44.065798  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:43.941550  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:43.956054  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.907229 13090 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:43.991101  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:44.076043  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:43.993980  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.032s	sys 0.023s
I20260709 12:27:44.078086  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.078660  7316 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:43.950300  7331 rpcz_store.cc:276] 0709 12:27:42.313959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.315934 (+  1975us) service_pool.cc:224] Handling call
0709 12:27:42.342106 (+ 26172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.342407 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:42.825418 (+483011us) log.cc:844] Serialized 14 byte log entry
0709 12:27:43.886015 (+1060597us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:43.886069 (+    54us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:43.886114 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.886192 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:43.886316 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13272,"prepare.queue_time_us":684197,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":6060,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":4294,"thread_start_us":1455179,"threads_started":3,"wal-append.queue_time_us":978133}
I20260709 12:27:43.926138  6378 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:43.835336 13018 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:44.080514  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89544.1us
I20260709 12:27:44.074139  6304 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:44.086390  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:44.090860  7332 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:43.909564 13063 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.093983  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.099383  7332 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:27:43.993947 13068 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:44.064091 13164 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:44.136826  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50548.8us
W20260709 12:27:44.118800  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.081241 12840 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.993911 13195 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 12:27:44.172277  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1087) took 1417 ms. Trace:
I20260709 12:27:44.178246  6450 rpcz_store.cc:276] 0709 12:27:42.754637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.756796 (+  2159us) service_pool.cc:224] Handling call
0709 12:27:42.757176 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.757730 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.757767 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.758326 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.300012 (+541686us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:43.300078 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.300186 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:43.814608 (+514422us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:44.171950 (+357342us) raft_consensus.cc:1685] finished
0709 12:27:44.171980 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.172206 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":331005,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":13589248,"thread_start_us":536731,"threads_started":1,"wal-append.queue_time_us":782188}
W20260709 12:27:44.100585 13153 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.004s
I20260709 12:27:44.176661  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:44.188822  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 30efe8bf06814d23bf20818fb23139be: Illegal state: Replica 4268023ecea149cd83cba77db53c3100 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:43.994175 13171 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.191425  6451 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:44.135124 13090 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:44.196138  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:43.885694  6198 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:27:44.192979 13223 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:27:44.220291  7314 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:27:44.222427  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 400894us
I20260709 12:27:44.224107  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:44.162106 12972 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:44.138680 13007 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:44.231707  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:44.239655  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:44.216390 13181 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.836s	user 0.010s	sys 0.004s
I20260709 12:27:44.250023  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:44.250808  6456 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:44.194378 13164 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:44.267621 13164 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:44.253777  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 117965us
W20260709 12:27:44.264479  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 269423us
I20260709 12:27:44.265717  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:44.196995 13221 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:44.216846 13009 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:44.286422  6456 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:44.227696 13137 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:44.172413 13068 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.307765  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:44.309559  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:44.310359  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:44.311177  7331 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.302567  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:44.312582  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:44.193543 12840 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.243201 13193 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:44.326547 13229 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
W20260709 12:27:44.327732 13233 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:27:44.328738 12809 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.751s	user 0.004s	sys 0.008s
I20260709 12:27:44.226938 13104 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 12:27:44.321420  6456 raft_consensus.cc:2368] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:27:44.241355 13028 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.328663 13162 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.320036  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:44.313522  6302 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.065s	sys 0.090s
I20260709 12:27:44.261461 12972 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:44.335353  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 111283us
W20260709 12:27:44.339810  6302 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.065s	sys 0.090s
W20260709 12:27:44.348474  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.304381 13195 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:44.355835 13162 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:44.323346 13115 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.273823 13181 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:44.340646 13222 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.455s	user 0.000s	sys 0.001s
I20260709 12:27:44.312252 13221 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.406579  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:44.407401 13171 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:44.407976 13221 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:44.408641 13231 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:27:44.409229  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.409925 12834 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:44.339043 13211 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.475s	user 0.000s	sys 0.001s
W20260709 12:27:44.371567 13220 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.484s	user 0.001s	sys 0.001s
I20260709 12:27:44.389685  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.417829  7332 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:44.418252  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.419178  7316 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:44.350224 13028 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.349534 13242 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:44.320606 13009 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.372123 13084 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.410282 13224 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.439384  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:44.395737 13007 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:44.478601 13240 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:44.478981 12972 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.416705 13020 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.485730  7331 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:44.356804 13241 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:27:44.439373 13216 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.497781  6382 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:27:44.499193  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.345996 13193 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.491206  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3074) took 1855 ms. Trace:
I20260709 12:27:44.501694  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.493249  7316 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:44.496313  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:44.501134  6458 rpcz_store.cc:276] 0709 12:27:42.636219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.636420 (+   201us) service_pool.cc:224] Handling call
0709 12:27:42.636915 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:42.658987 (+ 22072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.659017 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.846860 (+187843us) spinlock_profiling.cc:243] Waited 3.81 ms on lock 0x7b2c0012cca8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:27:42.912197 (+ 65337us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.313418 (+401221us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b48000cccd8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:27:43.694888 (+381470us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.694938 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.695014 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.490892 (+795878us) raft_consensus.cc:1685] finished
0709 12:27:44.490936 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.491123 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71544,"raft.queue_time_us":104616,"raft.run_cpu_time_us":3490,"raft.run_wall_time_us":567375,"spinlock_wait_cycles":65340032,"thread_start_us":575501,"threads_started":1,"wal-append.queue_time_us":766830}
I20260709 12:27:44.503266  6379 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:44.497052  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.505445  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.506906  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.508118  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.509009  6382 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 14: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:44.509846  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.403157 13115 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.504683  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.399396 13195 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.519176  6381 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:44.516736  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.454212 13224 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.525120  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.525928  6379 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
W20260709 12:27:44.514758  6275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.008s	sys 0.003s
W20260709 12:27:44.527477  6275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.009s	sys 0.003s
W20260709 12:27:44.528537  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.529086  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:44.529376  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.536301  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.449091 13162 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:27:44.537045  6385 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:44.523206  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3460) took 1684 ms. Trace:
I20260709 12:27:44.538920  6454 rpcz_store.cc:276] 0709 12:27:42.838702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.171203 (+332501us) service_pool.cc:224] Handling call
0709 12:27:43.171608 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.176470 (+  4862us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:43.176521 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.182144 (+  5623us) log.cc:844] Serialized 26 byte log entry
0709 12:27:43.680214 (+498070us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.680287 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.680403 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.522874 (+842471us) raft_consensus.cc:1685] finished
0709 12:27:44.522923 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.523117 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":694,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":3543,"thread_start_us":463623,"threads_started":1,"wal-append.queue_time_us":615897}
I20260709 12:27:44.530923  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:44.489104 13063 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.545667  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:44.546579  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:44.549569  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.502013 13086 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.551352  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:44.579399  6379 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.543962  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:44.550563  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:44.512917 13244 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.554131  6382 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:44.447554 13242 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.443166 13028 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.483593 13007 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.586714  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:44.588349  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.582376  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.592991  6385 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 11. (term mismatch)
I20260709 12:27:44.598953  6385 pending_rounds.cc:85] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 10
I20260709 12:27:44.599241  6385 pending_rounds.cc:107] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 14.11
I20260709 12:27:44.599524  6385 raft_consensus.cc:2889] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:44.601647  6382 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:44.604048  7332 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 23 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:44.612097  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.616670  6379 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:44.593243 13063 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:44.612795  7329 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:44.579348 13171 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:44.620409  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.626132  7317 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:27:44.543392 12834 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:44.626827  6384 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:27:44.627293  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 281238us
W20260709 12:27:44.625590  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 322448us
I20260709 12:27:44.629773  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.624367  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:44.632746  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:44.581913 13250 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:44.639194  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.634260 12840 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.592463 13084 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 12:27:44.625408 13171 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:44.643846  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.644872 13185 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:44.645565 13241 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:44.647001 13224 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.642213 13243 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:27:44.597090 13174 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:27:44.598591 13086 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.683076  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.639504 13068 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.629062 13063 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:44.596809 13244 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.679011  6384 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:44.710072  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.636719 13250 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:44.444641  6198 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 12 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 15 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:44.684060  6382 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
W20260709 12:27:44.684661  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.653332  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:44.702446  6454 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:44.658911 13171 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.633162 12834 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.658655 12840 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:44.715047  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:44.716683  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63497.2us
I20260709 12:27:44.747957  6384 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:44.747749  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:44.750924  6454 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:44.752882  6383 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:44.790941  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:44.791909  6382 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
W20260709 12:27:44.792572  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 168347us
I20260709 12:27:44.794245  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.794965  7318 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:27:44.724232 12840 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.719980 13068 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:44.804239  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:44.797400  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 49783us
I20260709 12:27:44.679260 13185 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.713971 13250 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.720640 13181 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:44.894562  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 90382.4us
I20260709 12:27:44.785521 13009 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.799374  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:44.800350 13084 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.825784  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:44.917483  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 91791.1us
I20260709 12:27:44.918854  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:44.805507 13213 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:44.875046  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.874223  6198 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 14 to 16, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 16 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:44.879977 13244 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.800635 13063 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.881704 12834 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.883481 13086 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.886807 13115 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.795102 13252 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:44.915632  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 116363us
I20260709 12:27:44.783839 12974 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:27:44.804586 12840 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.916980  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:44.934949  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:44.925705 13020 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:44.929183 13068 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:44.932168 13242 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:44.940003 13115 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:44.940214  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:44.940703 13121 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.330s	user 0.013s	sys 0.000s
I20260709 12:27:44.943878  6198 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:44.983413 13121 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.985621 13121 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.985301  6198 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 14 to 17, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 17 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:44.937382 13028 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:44.987236 13269 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.987462  6198 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 9 to 13, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 13 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:44.988067 13269 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.928972 13084 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.992143 13084 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:44.947211 13094 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.030s	user 0.012s	sys 0.000s
I20260709 12:27:44.949054 13171 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.994784 13171 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.928716 13181 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:44.995528 13181 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:44.996390 13181 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:44.997345 13274 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:44.925551 13213 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:44.896754 13221 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:27:44.998710 13213 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:44.999229 13068 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.999426 13221 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:44.999872 13221 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:44.999781 13068 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:44.929535 13232 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:45.000946 13221 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:44.969813 13192 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.368s	user 0.015s	sys 0.002s
I20260709 12:27:44.939783 13258 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:27:44.971690 13193 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.933997 13252 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:45.003793 13252 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.005210 13252 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:44.998242 13274 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:45.006768 13274 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:44.970659 13185 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:44.981796 13115 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.925958 13009 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:27:45.011679 13221 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:45.011772  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:44.950439 12834 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:45.013497 12834 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.013913 12834 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.016163 13068 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.017238 13145 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:44.996598  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56465.1us
I20260709 12:27:45.011806 13115 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:45.023133  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:45.026005 13266 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:27:45.001741 13232 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.026713 13232 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:45.002440  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.027915 13094 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.028946  6302 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:45.029985  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:45.032315  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:45.033324  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1098) took 1077 ms. Trace:
W20260709 12:27:44.981139 13205 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.195s	user 0.012s	sys 0.000s
I20260709 12:27:45.030109 13094 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.010990  7314 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:45.033699  6455 rpcz_store.cc:276] 0709 12:27:43.955860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.956489 (+   629us) service_pool.cc:224] Handling call
0709 12:27:44.068577 (+112088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.068782 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:44.425485 (+356703us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.654196 (+228711us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.032808 (+378612us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b180045b740. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:27:45.032986 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:45.033035 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:45.033079 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.033153 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.033257 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":326693,"prepare.queue_time_us":41044,"prepare.run_cpu_time_us":1849,"prepare.run_wall_time_us":6126,"raft.queue_time_us":1678,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":27647,"spinlock_wait_cycles":93786624,"thread_start_us":558341,"threads_started":2,"wal-append.queue_time_us":518645}
I20260709 12:27:45.047433 13115 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.050463 13115 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:45.051493  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 356799us
I20260709 12:27:45.054384  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:45.054934  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:45.055707  7332 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:45.057381  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:45.057987  7332 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:44.989713  6198 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:45.061775  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:45.064393  6302 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 12:27:45.074407  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:44.994431 12840 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.057384 13242 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.062855 13009 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:45.093123  7332 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:45.101577  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.058293 13205 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.118693 13242 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.061029 13249 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:45.122694 13269 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.147162  7320 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:45.127315 13009 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:45.127624 13192 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.160004 13205 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.162868  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:45.163803  6455 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.162899 13199 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:45.165241 13199 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:45.166121 13199 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.167131 13199 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:45.169909 13257 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.521s	user 0.004s	sys 0.000s
I20260709 12:27:45.172144 13040 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.172792 13158 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.173317 13158 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.175442 13158 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.177532  6455 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
W20260709 12:27:45.180595 13247 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.304s	user 0.003s	sys 0.005s
I20260709 12:27:45.181988 13009 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.210192 13213 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.213683 13250 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.216377 13250 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:45.217926 13241 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.539s	user 0.007s	sys 0.004s
I20260709 12:27:45.218643 13241 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:45.220613 12834 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.166844 13232 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.190107 13192 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:45.276980  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.288552 13242 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.276110 13213 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.293859 13192 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.299768  7318 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:45.299855 12834 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.304939 13274 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.308023 13274 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.310477 13232 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.315124 13009 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.342252 13009 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:45.386806 13018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.001s	sys 0.015s
W20260709 12:27:45.387495 13018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.002s	sys 0.015s
I20260709 12:27:45.485678  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:45.488708  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.493384  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.489176  6198 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 13 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:27:45.500633  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.493568  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3022) took 1803 ms. Trace:
I20260709 12:27:45.505661  6305 rpcz_store.cc:276] 0709 12:27:43.689697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.690574 (+   877us) service_pool.cc:224] Handling call
0709 12:27:43.690880 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.693485 (+  2605us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:27:44.640462 (+946977us) spinlock_profiling.cc:243] Waited 2.94 ms on lock 0x7b180047b480. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa6c9f3c7 00007f6fa32b19a7 00007f6fa32c10bf 00007f6fa32c1038
0709 12:27:44.715722 (+ 75260us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.039610 (+323888us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.040104 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.040246 (+   142us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:45.040281 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.040358 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.492740 (+452382us) raft_consensus.cc:1685] finished
0709 12:27:45.492970 (+   230us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.493338 (+   368us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57366,"prepare.queue_time_us":192192,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2767,"raft.queue_time_us":3459,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":31327,"spinlock_wait_cycles":6758272,"thread_start_us":398570,"threads_started":2,"wal-append.queue_time_us":421664}
I20260709 12:27:45.522797  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:45.523599  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.526145  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:45.530339  6385 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:45.555536  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.556955  6455 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:45.556842  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:45.565132  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1099) took 1608 ms. Trace:
I20260709 12:27:45.565735  6451 rpcz_store.cc:276] 0709 12:27:43.956312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.956600 (+   288us) service_pool.cc:224] Handling call
0709 12:27:43.957023 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.216441 (+259418us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.564528 (+1348087us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000c21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:45.564911 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538555264}
W20260709 12:27:45.578037  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.579252  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 208218us
I20260709 12:27:45.580533  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3093) took 1348 ms. Trace:
I20260709 12:27:45.581029  6453 rpcz_store.cc:276] 0709 12:27:44.231811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.277433 (+ 45622us) service_pool.cc:224] Handling call
0709 12:27:44.324884 (+ 47451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.544060 (+219176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.544207 (+   147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.544291 (+    84us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:44.544349 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.579900 (+1035551us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54001c1e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:45.579986 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.580296 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503305984}
W20260709 12:27:45.579689  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.597903  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:45.599587  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:45.600795  7329 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.672943 13205 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.680070 12972 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.680619 13205 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.681823 12972 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.683862 13028 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.684994 13028 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.685765  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.687433  6451 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 12:27:45.687844 13174 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.688778 13174 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.689257  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.690938 13213 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:45.693733  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:45.695070 13018 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:45.701786  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 98257.6us
I20260709 12:27:45.700682  6382 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:27:45.715929  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.724118  6453 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:45.727978 13188 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:45.728713 13213 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.736804 13249 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:45.737457 13249 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:45.738798  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:45.704049 13018 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:45.740212  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115274us
I20260709 12:27:45.738937  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:45.746512  6385 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:45.747189  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.753396 13221 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:45.751057  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:45.754743 13174 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.756086 13174 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:45.755180  6385 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:45.756898 13174 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.746811 13009 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.758090  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62263.6us
I20260709 12:27:45.762125  6450 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:45.769239 13205 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.762166  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.773360 13274 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:45.775172  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:45.775905 13028 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.778091  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:45.778842 13018 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.775528 13213 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.776051  6453 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:45.785640 13274 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.786418 13274 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.788767 12840 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.789175 13274 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.790622  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:45.791801  7316 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:45.791059 13009 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.790617  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:45.793537  7317 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:45.794567  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2755) took 1992 ms. Trace:
W20260709 12:27:45.775489  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50732.1us
I20260709 12:27:45.795073  6304 rpcz_store.cc:276] 0709 12:27:43.733077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.737825 (+  4748us) service_pool.cc:224] Handling call
0709 12:27:45.713088 (+1975263us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b5400140fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:45.725783 (+ 12695us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768396032}
I20260709 12:27:45.796198 13009 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.797755  6348 leader_election.cc:400] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:27:45.804376  6348 leader_election.cc:403] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:27:45.805668 13226 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:27:45.807242  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:45.811198  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.812827  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.813918 13020 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:45.815670  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.815872  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:45.816988  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.817873  6382 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:45.819007 13205 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:45.819970 13205 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:45.821287 13205 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.807534 12972 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.809221  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:45.810089 13221 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.810845 13009 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.837446  6453 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:45.853614  7329 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:27:45.849223  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:45.859220  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50344.5us
I20260709 12:27:45.813516 13174 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.863209  6382 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:45.864944  6451 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:45.864699  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:45.865703 13188 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:45.865952 13221 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.870417  6385 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:45.883736  6453 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:45.884560  6453 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:45.890349  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65290.5us
I20260709 12:27:45.859007 13009 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.885495 13205 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:45.894611 13299 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.897898  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:45.899073  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:45.902928 13232 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:45.902987  6453 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.906994 13297 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.909440  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.911118 13296 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.912791 13174 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.926548  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.893744  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.915516  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:45.932068  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:45.935164  6453 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.913993 13192 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:27:45.892277 13188 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:45.939958  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:45.955868  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53775.2us
I20260709 12:27:45.921592 13205 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:45.965960  6450 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:45.972561 13009 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:45.974818  6385 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:45.975164 13221 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:45.941042 13232 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:45.928268 13297 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:45.988478 13297 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:45.968609 13205 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:45.950979 13174 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:45.924546 13296 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.008217 13232 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:46.010560 13302 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:27:45.996474 13221 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:46.027984  6458 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:46.028024  6455 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:45.968107 13299 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.036911 13174 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:46.039141 13188 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.040225  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.059026  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:46.072425 13307 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:46.073411 13307 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.033385 13249 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.088505  6457 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:46.092048 13221 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.096711 13221 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.096782 13195 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:46.106451 13217 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 2.220s	user 0.003s	sys 0.020s
W20260709 12:27:46.113029  6383 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2901) took 1564 ms (client timeout 1990 ms). Trace:
I20260709 12:27:46.113373 13299 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:46.111506  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:46.117172  7314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1009) took 1814 ms (client timeout 1759 ms). Trace:
W20260709 12:27:46.117676  7314 rpcz_store.cc:269] 0709 12:27:44.302146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.302262 (+   116us) service_pool.cc:224] Handling call
0709 12:27:46.116751 (+1814489us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b5400565754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:46.116943 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1588726144}
I20260709 12:27:46.111937 13296 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:46.112260  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 989) took 3387 ms. Trace:
I20260709 12:27:46.119973  6376 rpcz_store.cc:276] 0709 12:27:42.725143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.725269 (+   126us) service_pool.cc:224] Handling call
0709 12:27:42.741302 (+ 16033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:43.111174 (+369872us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:43.809483 (+698309us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.138463 (+328980us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.138681 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.138738 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:44.138834 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.110293 (+1971459us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:46.110361 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.112063 (+  1702us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77102,"prepare.queue_time_us":1314069,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":102270,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":24414,"spinlock_wait_cycles":849803008,"thread_start_us":989635,"threads_started":2,"wal-append.queue_time_us":629481,"wal-append.run_cpu_time_us":2043,"wal-append.run_wall_time_us":1231762}
I20260709 12:27:46.113369  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.124281  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3190) took 3586 ms. Trace:
I20260709 12:27:46.124782  6380 rpcz_store.cc:276] 0709 12:27:42.537297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:42.567643 (+ 30346us) service_pool.cc:224] Handling call
0709 12:27:42.567983 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:42.576922 (+  8939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:42.576967 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:42.588076 (+ 11109us) spinlock_profiling.cc:243] Waited 2.82 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:27:42.588668 (+   592us) log.cc:844] Serialized 52 byte log entry
0709 12:27:43.559088 (+970420us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:27:43.559156 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:43.559266 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.105515 (+2546249us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:46.123875 (+ 18360us) raft_consensus.cc:1685] finished
0709 12:27:46.123903 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.124080 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5532,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"spinlock_wait_cycles":39406336,"thread_start_us":969906,"threads_started":1,"wal-append.queue_time_us":1196090}
I20260709 12:27:46.109797  6309 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:27:46.130822 13264 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.253s	user 0.014s	sys 0.022s
W20260709 12:27:46.130913  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.37621e+06us
I20260709 12:27:46.132057 13249 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:46.132956  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1019) took 1079 ms. Trace:
I20260709 12:27:46.133064  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:46.133344  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.133469  7318 rpcz_store.cc:276] 0709 12:27:45.053839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.054004 (+   165us) service_pool.cc:224] Handling call
0709 12:27:46.132540 (+1078536us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400566b54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:46.132748 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":562383232}
W20260709 12:27:46.113955  6383 rpcz_store.cc:269] 0709 12:27:44.548296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.549274 (+   978us) service_pool.cc:224] Handling call
0709 12:27:46.112533 (+1563259us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b540022f454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:46.112768 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":404861312}
W20260709 12:27:46.128162 13237 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.681s	user 0.008s	sys 0.010s
I20260709 12:27:46.151252  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:46.152168 12840 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:27:46.153803  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:46.154755 13213 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:46.156000  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.156358 13104 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:46.157245 13104 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.160374 12834 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:27:46.164274 13259 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.285s	user 0.017s	sys 0.003s
I20260709 12:27:46.167550 13304 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:46.168175 13304 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:46.171355  6455 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:46.168680  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3199) took 2406 ms. Trace:
I20260709 12:27:46.168960 13307 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:46.160080 13104 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.175515 13249 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:46.188935 13192 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:46.190266  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2748) took 2458 ms (client timeout 1589 ms). Trace:
W20260709 12:27:46.191103  6308 rpcz_store.cc:269] 0709 12:27:43.728835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.730655 (+  1820us) service_pool.cc:224] Handling call
0709 12:27:46.179230 (+2448575us) spinlock_profiling.cc:243] Waited 87.5 ms on lock 0x7b54002df5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:46.187092 (+  7862us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":201166720}
I20260709 12:27:46.195263  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.172502 13304 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.173810  6377 rpcz_store.cc:276] 0709 12:27:43.762198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.764663 (+  2465us) service_pool.cc:224] Handling call
0709 12:27:43.790557 (+ 25894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.790961 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.790994 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.791527 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.095589 (+304062us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:44.095646 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.095740 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.127190 (+2031450us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:46.168103 (+ 40913us) raft_consensus.cc:1685] finished
0709 12:27:46.168137 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.168440 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4796,"raft.queue_time_us":10785,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":5227,"spinlock_wait_cycles":15587584,"thread_start_us":303664,"threads_started":1,"wal-append.queue_time_us":583916}
I20260709 12:27:46.191058 13192 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.193109 13249 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:46.207587  6452 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:46.195601 13104 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:46.199055 13061 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.216951  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.224227  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:46.237967  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3198) took 2437 ms (client timeout 1918 ms). Trace:
W20260709 12:27:46.238502  6378 rpcz_store.cc:269] 0709 12:27:43.761926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.762569 (+   643us) service_pool.cc:224] Handling call
0709 12:27:46.191658 (+2429089us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54001946d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:46.199527 (+  7869us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315310336}
I20260709 12:27:46.229329 13248 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:27:46.236465 13104 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.243458  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1221) took 2166 ms. Trace:
I20260709 12:27:46.243957  7320 rpcz_store.cc:276] 0709 12:27:44.077400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.077535 (+   135us) service_pool.cc:224] Handling call
0709 12:27:44.101274 (+ 23739us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:45.152886 (+1051612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.242865 (+1089979us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54005da2d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:46.243251 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2404822272}
I20260709 12:27:46.245548 13104 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.248652  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.231515 13061 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.265359  6380 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:46.273715 13061 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.265878 13304 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:46.275641 13192 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.278687 13137 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.279585 13137 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.284318  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.285804  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.287164  6383 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:46.290105  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.291065  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:27:46.292390  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:46.299388  6378 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:46.300078 13249 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.302240  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.303366  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:46.299036  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100069us
I20260709 12:27:46.310983  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.311434 13018 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:46.305305 13192 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:46.314436 13213 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:46.315158  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:46.346858  6377 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:46.347471  6383 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:46.347795 13265 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.470s	user 0.010s	sys 0.025s
I20260709 12:27:46.348294  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:46.349090  6377 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.316664  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.346901 13192 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.351393  6456 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:46.354781 13018 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:46.355628 13213 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.356279 13018 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:46.358250 13225 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.708s	user 0.005s	sys 0.010s
I20260709 12:27:46.340495 13307 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:46.340796 13007 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:46.336404  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.363296  6385 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:27:46.339129 13317 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:46.369685  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.361227 13192 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:46.381078  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 998) took 1331 ms. Trace:
I20260709 12:27:46.382087 13137 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.385845  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.365271 13213 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.402357 13007 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:46.402968 13007 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:46.404178 13007 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.407033 13313 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:46.407583 13313 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:46.408010 13313 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:46.417754 13007 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:46.413440 12840 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
W20260709 12:27:46.435773  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.387301  6384 rpcz_store.cc:276] 0709 12:27:45.034545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.035029 (+   484us) service_pool.cc:224] Handling call
0709 12:27:45.035448 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:45.035958 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:45.036000 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:46.240450 (+1204450us) spinlock_profiling.cc:243] Waited 940 us on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:27:46.241419 (+   969us) log.cc:844] Serialized 52 byte log entry
0709 12:27:46.291931 (+ 50512us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:46.291981 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.292094 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.359331 (+ 67237us) raft_consensus.cc:1685] finished
0709 12:27:46.359351 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.365906 (+  6555us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1422,"raft.queue_time_us":669,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":22010,"spinlock_wait_cycles":2162048,"thread_start_us":22795,"threads_started":1,"wal-append.queue_time_us":111370}
W20260709 12:27:46.438642  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.361891 13205 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:46.442587 13228 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 2.122s	user 0.019s	sys 0.008s
W20260709 12:27:46.444557  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 74340.4us
I20260709 12:27:46.446578  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:46.448055  6378 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
W20260709 12:27:46.453110 13319 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:27:46.455295 13047 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:46.455677 13047 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:46.456650 13047 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.458108 13047 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:46.458595 13061 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.459287 13061 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.461812 13061 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:46.465916 13263 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.682s	user 0.005s	sys 0.025s
W20260709 12:27:46.468528 13316 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.156s	user 0.003s	sys 0.001s
I20260709 12:27:46.472446  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.480659  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.483181  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.492164  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
W20260709 12:27:46.494905  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.496263  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2908) took 1875 ms. Trace:
I20260709 12:27:46.499042  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.496789  6379 rpcz_store.cc:276] 0709 12:27:44.620767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.622833 (+  2066us) service_pool.cc:224] Handling call
0709 12:27:46.207503 (+1584670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.233122 (+ 25619us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:46.262143 (+ 29021us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.263594 (+  1451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:46.263657 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:46.263724 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.263813 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.495917 (+232104us) spinlock_profiling.cc:243] Waited 1.57 s on lock 0x7b54001946d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:46.496063 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1823,"prepare.queue_time_us":51809,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":150,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":170,"spinlock_wait_cycles":3612909824,"thread_start_us":10792,"threads_started":2,"wal-append.queue_time_us":139432}
I20260709 12:27:46.507710  6455 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:27:46.511849  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.514150  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.515089  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.516490  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.517469  6453 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:46.521595  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.523984  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.500319 13061 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:46.526583  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.529527  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.527335  6459 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:46.526584  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.531486  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.533106  7329 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:46.530472 13061 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.538944 13061 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.539126 13210 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:46.542894  6458 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22.
I20260709 12:27:46.543941  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.545090  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.545674  6458 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:46.546063  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2900) took 1998 ms. Trace:
W20260709 12:27:46.546590 13164 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.834s	user 0.028s	sys 0.044s
W20260709 12:27:46.547053 13164 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.899s	user 0.028s	sys 0.044s
I20260709 12:27:46.532933  6384 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:46.553045  6457 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:46.554544  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.554567  6376 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:46.551719  6383 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:46.558631  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:46.558913  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 112496us
I20260709 12:27:46.559420  6454 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17.
I20260709 12:27:46.559969  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:46.560931  6454 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [15.9-17.10]
I20260709 12:27:46.556504  6379 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:46.562354  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.568302  6456 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:46.570007  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:46.571488 13239 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.966s	user 0.005s	sys 0.012s
I20260709 12:27:46.571599  6382 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:46.568856  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88298us
I20260709 12:27:46.561105 13286 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:46.546452  6381 rpcz_store.cc:276] 0709 12:27:44.547453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.548597 (+  1144us) service_pool.cc:224] Handling call
0709 12:27:44.549025 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.601565 (+ 52540us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.601610 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.602195 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.722619 (+120424us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:44.722998 (+   379us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.936403 (+213405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.545431 (+1609028us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b54003d40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:46.545498 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.545560 (+    62us) raft_consensus.cc:1685] finished
0709 12:27:46.545656 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.545884 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37867,"prepare.queue_time_us":257,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":103693,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":13486,"spinlock_wait_cycles":71690624,"thread_start_us":274787,"threads_started":2,"wal-append.queue_time_us":306160,"wal-append.run_cpu_time_us":2610,"wal-append.run_wall_time_us":1026336}
I20260709 12:27:46.588795  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2753) took 2856 ms. Trace:
I20260709 12:27:46.589171 13188 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:46.590284 13188 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.589248  6310 rpcz_store.cc:276] 0709 12:27:43.732548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.740181 (+  7633us) service_pool.cc:224] Handling call
0709 12:27:43.740586 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.740832 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:27:43.740857 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.755654 (+ 14797us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.092142 (+336488us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:44.092204 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.092311 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.479290 (+2386979us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:46.577953 (+ 98663us) raft_consensus.cc:1685] finished
0709 12:27:46.577968 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.588587 (+ 10619us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612333824,"thread_start_us":275633,"threads_started":1,"wal-append.queue_time_us":717259}
I20260709 12:27:46.594427 12809 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:46.595141 12809 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:46.596524 12809 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.607734  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.608979  6453 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:27:46.586542  6377 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:46.606765  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:46.603192 12809 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:46.631214  6378 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:27:46.578933 13018 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:46.640851  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.611268 13188 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.650131  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.602110 13164 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:27:46.650909 13248 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:46.683212  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.649817 12840 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:46.685817  6377 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:46.676025  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:46.689147  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:46.692444  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.664053 13188 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:46.696144  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.685171 13205 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.674085 13213 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:46.699623 12974 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.720146  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.720865  6457 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:46.720952  6378 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:46.722618 12840 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.725226 12840 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:46.725854 13153 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:46.726454 13153 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:46.727530 13336 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:46.727488 13153 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.730883  6456 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:46.730711 12974 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.732983 12974 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.734058 13188 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.734637 13213 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.736759 13213 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.737330 13164 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:46.740689  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.741587  6459 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:46.742585  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:46.742856  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:46.743760  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.744338  6458 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:27:46.690056 13018 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.746587  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.747478  6310 raft_consensus.cc:2410] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:46.748263  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.748416  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.749009 13018 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.749512  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:46.750392 13331 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:27:46.750777  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:46.751113  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 181711us
I20260709 12:27:46.752051  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:46.752454  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.753504  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:46.750327  6377 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:27:46.748982  6309 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:46.742209 13210 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:46.755307  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:46.749150  6304 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:27:46.760762  6456 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:46.763381  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.764056  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:46.765268  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:46.765764  6378 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:46.768543  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.769469  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.757880 13094 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:46.764832  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.818s (SENT) [suppressed 26 similar messages]
I20260709 12:27:46.761822 13307 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:46.776748  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:46.777884  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:46.776108  6451 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:27:46.760452 13061 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.781113  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:46.784955  6455 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
W20260709 12:27:46.785110  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.789749  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.790397  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.790700  6309 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:46.787297  6455 pending_rounds.cc:85] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 9
I20260709 12:27:46.779476 13213 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:46.779888 12974 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:46.798102  6304 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:46.792532  6455 pending_rounds.cc:107] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:27:46.800688  6455 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:46.801514  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:46.801816  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48489.2us
I20260709 12:27:46.802381  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:46.805720  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:46.810098  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.810995  6303 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:46.814529 13216 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:46.808902 12974 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.820307  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:46.820969  7332 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:46.821823 12974 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:46.815021 13094 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:27:46.824358 13094 consensus_queue.cc:1243] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:46.824883 13094 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:46.825187 13094 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:46.825205  6310 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:46.827442  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:46.829061  6275 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 521c92260b1b4a6abba1ce9d3c7c5ad2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:27:46.829927 13028 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:46.830463 13028 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:46.830938 13028 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:46.831682 13061 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:46.817214  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.834688  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:46.835613  7320 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:46.815124 13216 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:46.837297  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:46.838140  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:46.839116  6304 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:27:46.839808  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.842119  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:46.836588  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:46.847071 13115 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:46.847733 13115 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.848222 13115 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.850291  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:46.851423  6308 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:46.833264 13213 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.853809 13213 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.854305 13192 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:46.854789 13192 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.855141 13192 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:46.826807 13094 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.856534 13094 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:46.857847  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:46.858906 13007 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:46.839941 13188 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:46.859359 13216 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:46.860996  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.861238  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.861961  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:46.804675 13313 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:46.863021  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.872691  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:46.873435  6304 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:46.844667 13205 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:46.879050  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:46.881218  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.880053  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:46.881876  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45390.3us
I20260709 12:27:46.862571 13250 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:46.887140 13164 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:46.881426 13205 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.889552  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.890584  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2918) took 1186 ms. Trace:
I20260709 12:27:46.921670  6382 rpcz_store.cc:276] 0709 12:27:45.697114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.931698 (+234584us) service_pool.cc:224] Handling call
0709 12:27:45.932090 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.572181 (+640091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.883687 (+311506us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b540023ebd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:46.883912 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1467904384}
I20260709 12:27:46.896739  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:46.876227 13221 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:27:46.923030 13205 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:46.903633 13250 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:46.942312  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.944434  6310 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:46.936594  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:46.944819  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:46.977952  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:46.937574 13007 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:46.979363 13193 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:46.989363 13193 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:46.989920 13250 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:46.987463  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:46.990641  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68013.6us
I20260709 12:27:46.991186  7332 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:46.984740  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:47.039687  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.026s (SENT)
I20260709 12:27:47.042764  6305 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:46.983999  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.027468  6301 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
W20260709 12:27:47.049046  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 152383us
I20260709 12:27:46.989354  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.054045  6310 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:46.991122 13250 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.056871  6302 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:46.988580 13090 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.943284 13192 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.052594  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.057659  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:47.063920  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50205.3us
I20260709 12:27:46.949247 13028 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.027650 13018 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.071700  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.988543 12909 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:47.011911 13137 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:47.062307  6300 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:47.069439  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:46.989142 13115 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.086951  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.087570  6301 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.088194  6305 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:47.092830  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:47.105836 13344 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:27:47.109632  7317 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:47.110713  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.109645  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.114701  6308 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.080727 13137 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.110335 12974 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:47.128819  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:46.990459 13193 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:47.140501  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92816.3us
W20260709 12:27:47.147898  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61459.8us
I20260709 12:27:47.147921 13248 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:47.148537  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.149184 13248 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:47.149533 13007 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:47.084259 13018 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.147996  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.127454 13137 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.126183 13216 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:47.156561  6310 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:47.087247 13090 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.150125 13307 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.008s
W20260709 12:27:47.166344  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 206091us
I20260709 12:27:47.173046  6305 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:47.141882 13164 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:47.182111  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.162950 13313 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.162786 13216 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.124018 12909 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.158543 13248 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:47.185101  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:47.185876  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.170374  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:47.186602  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.187332  6310 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:47.192830  6300 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:47.194794  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.190460  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:47.197429  6301 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
W20260709 12:27:47.186657 13349 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.138s	user 0.004s	sys 0.000s
I20260709 12:27:47.190682  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:47.196056  6309 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:47.174261 13153 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:47.200332  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:47.201694  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.202728 13090 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.202808  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.216938  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.208700 13192 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.209367  7332 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:47.216877  6301 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:47.217628  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.222754  6310 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.224381  6309 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.224586  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
W20260709 12:27:47.225046 13350 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:47.225303  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.225633  6303 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:47.224932 13115 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.225656  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.227239  6300 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:47.227767 12909 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.228387  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:47.229928 13313 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:47.224385  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.229928  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:47.230850 13164 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.231647  6310 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:47.232224 13047 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:47.232685 13047 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:47.233165 13047 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.230685 13313 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:47.239324 13313 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.241308  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:47.242218  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71943.3us
W20260709 12:27:47.244416  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.246824  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:47.247596 13342 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.248466  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:47.251680  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:47.252045 13221 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:47.252319  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.252544 13221 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.252943 13221 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:47.253129  6300 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.254709 13304 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.254824 13356 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:47.255537 13356 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:47.256052 13356 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:47.196870 13216 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.259356  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.260221  6304 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:47.255290 13304 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.195794 13248 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:47.254859  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.267891  6310 raft_consensus.cc:2376] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:27:47.275033  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:47.274000  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:47.266803 13344 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:47.286553  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:47.283396  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:47.283768  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96286.5us
I20260709 12:27:47.289105  6383 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:47.297276  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:47.297866  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:47.298235  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.304920  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:47.305917  6451 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.307052  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:47.311419  6382 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:47.314150  6380 raft_consensus.cc:2368] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18.
I20260709 12:27:47.315356  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:47.321746  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:47.287309 13249 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:47.287593 13028 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.335287  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.337858  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.340181  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:47.331430  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:47.343565  6310 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.337327 13357 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.277277 13153 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:47.353876  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56509.8us
I20260709 12:27:47.360595  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.361410  6303 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
W20260709 12:27:47.366132 13360 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:27:47.345585 13249 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:47.368072  6376 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: [14.9-16.10]
I20260709 12:27:47.368448  6378 raft_consensus.cc:2410] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:27:47.369323  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.370461  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.372141  6303 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:47.358376 13028 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:47.377102  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:47.385486  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 103803us
I20260709 12:27:47.391548  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.397957  6304 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:47.365129 13258 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:47.391043  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:47.401693  6303 raft_consensus.cc:2410] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:27:47.347925 13248 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.413892 13248 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:47.414625  6309 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:47.415603  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:47.416145  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.416831 13090 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.419867  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.431838  6304 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:47.420279  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.435108  6303 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.411816  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.441397  6310 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.421092 13358 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.438791  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:47.408469 13353 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
W20260709 12:27:47.405224 13346 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:27:47.408740 13018 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.363600 13357 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.449391  6309 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:47.418813 13028 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:47.464617  6301 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:47.448462 12840 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:47.446575 13358 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.478740  6380 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:47.480489 13358 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.487775 13342 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:47.488435 13205 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.002s
W20260709 12:27:47.488610 13335 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.039s	sys 0.060s
I20260709 12:27:47.488915 13205 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.491590 13028 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.444602 13164 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
W20260709 12:27:47.488921 13335 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.039s	sys 0.060s
I20260709 12:27:47.498677 13137 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.500445 13047 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.523473 13047 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:47.475390 13249 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.477687 13304 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:47.530210 13348 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.333s	user 0.004s	sys 0.000s
W20260709 12:27:47.545851  6425 leader_election.cc:336] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.589s (SENT)
W20260709 12:27:47.551172  6425 leader_election.cc:336] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.640s (SENT)
I20260709 12:27:47.551586  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:47.551987 13357 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.552668 13307 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:47.552675  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 161722us
I20260709 12:27:47.556751  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:47.559013 12840 raft_consensus.cc:2764] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:47.558403  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:47.563220 13018 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:47.563313 13153 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.563987 13018 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:47.565907 13361 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:27:47.565394 13018 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.508713 13205 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.509027 12974 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.570396 12974 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.570263  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:47.572211 13342 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:47.572817 13367 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:47.573376  6424 leader_election.cc:400] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18.
I20260709 12:27:47.573688  6424 leader_election.cc:403] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:47.515610  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.516199  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:47.507524 13362 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:47.589033  6310 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.568271  6379 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:47.568568  6378 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:47.566708 13258 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:47.590961  6453 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:27:47.595716  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:47.595729  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.598843  6451 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:47.605113  6451 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:47.601608  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:47.606596  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:47.608626  6451 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:47.606719  6456 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:47.612742  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:47.604521  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:47.614841  6380 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.615518  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.616737  6450 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:47.448765 13090 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.625141  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.614073  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:47.518211 13221 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:47.630888  6425 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.536s (SENT)
I20260709 12:27:47.647022  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:47.651029  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:47.602103 13210 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18.
W20260709 12:27:47.621726 12809 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.976s	user 0.014s	sys 0.002s
I20260709 12:27:47.574402 13367 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
W20260709 12:27:47.660089 13363 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 12:27:47.660846  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.666656  7314 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:47.673254  6451 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:47.664665  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.647262 13205 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:47.681614  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50778.3us
W20260709 12:27:47.683110  6425 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.759s (SENT)
I20260709 12:27:47.683638  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:47.654865 12840 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:47.677801  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:47.629678 13007 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:27:47.691296  6304 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:47.665736 12834 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:47.664716 13137 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:47.693751  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.701287  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.702015  6300 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 12.
W20260709 12:27:47.699859  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 187188us
I20260709 12:27:47.677503 13205 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.671067 13342 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:27:47.699021 13313 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:47.697674 12834 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:47.701388 13061 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:27:47.734849  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 122189us
W20260709 12:27:47.736301  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.734177  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:47.718324 13205 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:47.697640 12840 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.723433 13047 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:47.735414 12834 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:47.710388 13196 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:27:47.753142 13313 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:47.766824 13313 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:47.769613 13356 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.002s
I20260709 12:27:47.770251 13356 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.771220 13090 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.772861 13090 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:47.773051 13356 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:47.739682 13007 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:47.745916 13061 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:47.762339  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:47.793982  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.799813 12909 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.799973 13258 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.800560 12909 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.800707 13258 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.779697  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.806975  6310 raft_consensus.cc:2376] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:27:47.809381  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3512) took 1040 ms. Trace:
I20260709 12:27:47.815042  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:47.819129  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136062us
I20260709 12:27:47.819622  6305 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:47.778529 13216 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.502s	user 0.011s	sys 0.000s
I20260709 12:27:47.761029 13385 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.825909  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:47.825937  6310 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:47.828042  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:47.807986 12840 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:47.834331 13373 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:27:47.834952  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:47.843118 13378 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:27:47.832612 13061 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.849494  6380 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:47.853888  6308 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.857059  6301 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:47.815274  6454 rpcz_store.cc:276] 0709 12:27:46.769173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.769285 (+   112us) service_pool.cc:224] Handling call
0709 12:27:46.991854 (+222569us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:27:46.992090 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.040598 (+ 48508us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:47.175655 (+135057us) log.cc:844] Serialized 14 byte log entry
0709 12:27:47.679092 (+503437us) log.cc:844] Serialized 14 byte log entry
0709 12:27:47.808900 (+129808us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:47.808963 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:47.809038 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.809137 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.809297 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":37827,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":103,"raft.queue_time_us":127391,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":29271,"spinlock_wait_cycles":2949888,"thread_start_us":705218,"threads_started":4,"wal-append.queue_time_us":194343}
I20260709 12:27:47.873904  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.767804 13313 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.882412  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:47.865386 13377 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:27:47.890224  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:47.907372  6451 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:47.865688 13192 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.912111  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.914459  6450 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
W20260709 12:27:47.921132 13376 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:27:47.856909 13385 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.948576  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:47.852314 13216 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:47.975394  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.976518  6452 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:47.977732  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:47.980556  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.981933  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.929818 12909 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.983091  6453 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:47.992172  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:47.995941 12974 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:47.998719 13258 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:47.909451 13164 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:27:47.966038 13355 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:48.013159 13355 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:48.014283 13355 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:48.016326 13385 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:47.987990 13388 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:27:48.018409 13380 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:27:48.018738 13205 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.018923  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:48.019874  6452 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:48.021695 13250 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.958s	user 0.012s	sys 0.001s
I20260709 12:27:47.976704 13313 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:48.024333  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:48.025120  6454 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:48.025774 13342 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:47.965543 13192 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.029125 13192 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:48.005774  6379 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:48.031670  6307 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:27:47.970216 13090 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:27:48.035578  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:48.042553  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:48.046123  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:48.046744  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:48.003937 12974 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:48.051432  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:48.052196  6459 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:48.055089  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 327306us
W20260709 12:27:48.059377  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 457801us
I20260709 12:27:48.008921  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:48.043995 13342 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:48.095551  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 241460us
W20260709 12:27:48.103295  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94493.1us
I20260709 12:27:48.112807  6300 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:48.112943  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:48.203439  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 158788us
I20260709 12:27:48.204634  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:48.206326  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103779us
W20260709 12:27:48.219365  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.207322  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:48.240368  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 128490us
I20260709 12:27:48.243088  6272 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22.
I20260709 12:27:48.243532  6272 leader_election.cc:403] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:48.250715  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1053) took 1267 ms. Trace:
I20260709 12:27:48.251041  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.259240  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:48.098790 13388 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:48.260134  6453 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:48.251351  6307 rpcz_store.cc:276] 0709 12:27:46.983633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.060696 (+ 77063us) service_pool.cc:224] Handling call
0709 12:27:47.119977 (+ 59281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.032066 (+912089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.242225 (+210159us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54002de6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:48.250635 (+  8410us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095515008}
W20260709 12:27:48.267787  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60583.4us
I20260709 12:27:48.271847  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.235248 13384 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:48.267267 13094 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.410s	user 0.018s	sys 0.001s
W20260709 12:27:48.267567 13249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.019s	sys 0.018s
I20260709 12:27:48.259271 13205 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:48.266817 13018 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22.
W20260709 12:27:48.291908  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87406.1us
I20260709 12:27:48.269918 13248 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:48.265285 13388 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:48.292873  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46412.1us
I20260709 12:27:48.294104  6457 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:48.242316 12840 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 12:27:48.301641  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:48.279024 13384 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.318637  6453 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:27:48.289659 13250 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.321102  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:48.330369  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 352744us
I20260709 12:27:48.330996  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:48.341159  6451 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.298388 13388 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:48.331414 13393 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:48.336654  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109713us
W20260709 12:27:48.353782 13371 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.356s	user 0.001s	sys 0.018s
I20260709 12:27:48.360579  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1128) took 1109 ms. Trace:
I20260709 12:27:48.324388 13400 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.363780 13400 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.367372 13221 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:48.371342  6379 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:48.361047  6457 rpcz_store.cc:276] 0709 12:27:47.250768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.252501 (+  1733us) service_pool.cc:224] Handling call
0709 12:27:47.252879 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.296253 (+1043374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.360285 (+ 64032us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:48.360503 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2393580928}
I20260709 12:27:48.372681 13221 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.373822  6382 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:48.373026  6424 leader_election.cc:400] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:48.374372  6424 leader_election.cc:403] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:27:48.377279 13216 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:48.307346 13342 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.373620 13221 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.372934  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:48.389261  6381 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
W20260709 12:27:48.395817  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95917.2us
W20260709 12:27:48.404978  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53952.9us
W20260709 12:27:48.407405  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.407819  6451 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:48.319991 13384 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:48.412269  6452 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
W20260709 12:27:48.416018  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 312459us
W20260709 12:27:48.420154  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 99144.1us
I20260709 12:27:48.469883  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:48.438886  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.426654  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:48.408428 13216 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.417889 13094 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:48.404861 13258 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:48.440459  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:48.476811 13395 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:27:48.526999  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57213.2us
I20260709 12:27:48.468822  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.430665  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:48.418201 13249 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:48.465622 13399 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:27:48.476964 12834 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.684s	user 0.009s	sys 0.005s
I20260709 12:27:48.439505 13391 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:48.519554 13248 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.614939 13188 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:48.651894  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 221333us
W20260709 12:27:48.658929  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 232435us
I20260709 12:27:48.425058 13250 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.441322 13367 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.210s
I20260709 12:27:48.661907  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:48.631150 13397 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:48.422955 12840 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.234s
I20260709 12:27:48.633729 13258 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:48.683991 13258 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:48.675604  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 258447us
I20260709 12:27:48.687059 13357 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:48.687810 13357 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:48.687935  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.689009 13357 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:48.689999  6272 leader_election.cc:400] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17.
I20260709 12:27:48.657512 12974 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.636973 13216 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.693259 13216 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:48.676563 13196 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:48.694864 13196 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.697575 13397 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:48.681797  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:48.665096 13250 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.701740 13250 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:48.702557  6272 leader_election.cc:403] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:48.704612 13397 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17.
I20260709 12:27:48.705569 13397 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:48.706915 13018 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:48.707432 13018 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:48.526585  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:48.715945  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.662781 13393 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:48.718786  6274 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.553s (SENT)
I20260709 12:27:48.680979  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:48.651563 13304 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:27:48.665904 13386 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:48.674782 13188 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:48.722177  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 253454us
W20260709 12:27:48.709920 13313 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.655s	user 0.013s	sys 0.000s
I20260709 12:27:48.448681  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
W20260709 12:27:48.729529  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 272129us
I20260709 12:27:48.659746 12834 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:48.752753  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.756062  6382 raft_consensus.cc:606] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:48.759296  6382 raft_consensus.cc:614] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:48.665643 13248 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.708598 13018 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:48.774478  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112652us
W20260709 12:27:48.784555  6459 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.730s	user 0.013s	sys 0.003s
I20260709 12:27:48.785166  6459 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:48.791112 13383 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:48.792865  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.795542  6304 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:48.758744 13391 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:48.757494 13386 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:48.775992 13258 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:48.807964 13375 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:48.813707 13375 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:48.814330 13375 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.819000 13196 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:48.820477  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 103058us
I20260709 12:27:48.820943 13248 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:48.822031 13386 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:48.800920 12809 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:48.809126 13411 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:27:48.793107 13018 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:48.822237  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:48.834789  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.835932 13416 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:48.836301  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.803563 13391 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:48.850008 13383 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.839517 13188 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:48.855172 13408 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.500s	user 0.005s	sys 0.008s
I20260709 12:27:48.855044  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.856513 13383 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.858423  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.859318 13383 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:48.861255 13383 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:48.864329  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:48.867849  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.871871  7332 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:48.872205  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 152220us
I20260709 12:27:48.839908 13250 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:48.840161 13391 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:48.873929  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:48.875125  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:48.875824  7332 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:48.876799  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:48.879525 13412 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:27:48.873221  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:48.884433  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44632us
W20260709 12:27:48.885399  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57212.5us
I20260709 12:27:48.846700 13400 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:48.889518 13403 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.542s	user 0.000s	sys 0.012s
I20260709 12:27:48.884212 13250 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.890385  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:48.890944 13250 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:48.891430 13367 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:27:48.892086 13388 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.548s	user 0.007s	sys 0.005s
I20260709 12:27:48.892642  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:48.893714 13367 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:27:48.894068 13367 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:48.894425 13367 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:48.899739  6274 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.818s (SENT)
I20260709 12:27:48.907809 13425 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:48.908309 13425 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:48.908649 13425 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:48.910012 13342 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:48.910929 13342 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:48.911268 13342 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:27:48.911150  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47002.9us
I20260709 12:27:48.912681 13342 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:48.913177  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.915439  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.918157 13400 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.918670 13400 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:48.919049 13400 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:48.925678  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.929584  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:48.931689 13426 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:48.930914  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:48.939973  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.892719 13388 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:48.942926  6274 leader_election.cc:336] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.229s (SENT)
I20260709 12:27:48.939965  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:48.944649  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.945351  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:48.942303  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:48.947131  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:48.947355  6310 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:48.925487 13366 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:48.948108  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:48.929893 13356 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:48.946188  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:48.952790  6302 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:48.952654  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:48.951161  7314 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:27:48.955839  6308 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:48.943435 13061 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:48.926929 13392 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:27:48.963689  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:48.943706 13248 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:48.965499  7318 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:48.956225  6308 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:48.949498  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:48.945731  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:48.942251 13094 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:27:48.951907  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:49.003783  7332 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:48.960579 13387 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:27:49.003824 13094 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:49.005429 13094 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:49.005810 13248 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.006305 13248 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.006714 13094 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:48.968631  6307 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:49.008200 13094 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.011312  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:49.013331  6301 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:48.990068  6308 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.014410 13416 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:49.014905  6308 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:49.016750 13388 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:48.989013 13384 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:27:48.968938 13383 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:48.943593 13313 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:27:48.995860 13423 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:27:48.996691 13375 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:48.950086 13421 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:49.036577 13421 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.036975 13421 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.000018 13397 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:49.002795 12834 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:48.961679 13061 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.046934 13061 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:49.048043 12974 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:49.048609 12974 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.051698  6458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34bae875d7104f298673c223da6caf2e"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:49.052364  6458 raft_consensus.cc:606] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:49.052676  6458 raft_consensus.cc:614] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:48.989076 13391 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.055441 13391 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.027264 13192 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.056615  6381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:27:49.057251  6381 raft_consensus.cc:606] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:27:49.025322  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75921.1us
I20260709 12:27:49.030928  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.063339  6302 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:48.960172 13342 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.034965 13394 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.035223  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:49.067320  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:49.064886  6308 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:27:49.076378  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 136456us
W20260709 12:27:49.078693  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 43559.8us
I20260709 12:27:49.005533 13387 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:49.090656  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:49.091526  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:49.091853 13391 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.092201  6458 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:49.092414 13391 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.095202  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.097288 13304 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:49.097681 13383 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:27:49.100579 13405 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.644s	user 0.002s	sys 0.000s
I20260709 12:27:49.081461 13387 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:49.104153 12974 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.061298 13384 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.106904 12974 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.109422 13400 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:49.111223 13401 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.755s	user 0.002s	sys 0.011s
I20260709 12:27:49.111779 13367 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.119468  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2797) took 1332 ms. Trace:
I20260709 12:27:49.119916  6304 rpcz_store.cc:276] 0709 12:27:47.786876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.787190 (+   314us) service_pool.cc:224] Handling call
0709 12:27:47.802939 (+ 15749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.796115 (+993176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.119220 (+323105us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b54002d28d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:49.119391 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2282332544}
I20260709 12:27:49.149497  6302 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:49.152616  6310 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
W20260709 12:27:49.163174  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46153.2us
I20260709 12:27:49.086194 13407 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:49.165807  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 98622.9us
I20260709 12:27:49.169035  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.163928  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:49.113224 13115 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.098052 13394 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.171783  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "55b778a085744d27a1adc35598438230"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:49.178925  6451 raft_consensus.cc:606] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.106978 13384 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.112928 13210 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.172945  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.167630 13391 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.174719  6302 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:49.088661 13250 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.187022  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.193660  6381 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:27:49.187232  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:49.176591  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 233666us
W20260709 12:27:49.196547  6274 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.140s (SENT)
I20260709 12:27:49.185298  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.186417  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.172343 13407 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.198168 13407 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.206751 13196 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:49.207252 13196 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:49.207638 13196 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:49.167088 13392 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:27:49.210820  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.212890 13375 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.214834  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.216233  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.217820  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:49.218619  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.222283  6456 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:49.223809  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:49.225675 13394 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.227274 13342 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:49.227757 13342 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.192509 13210 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.228128 13342 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:49.229387 13425 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:49.241411  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54276.4us
I20260709 12:27:49.242245  6452 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:49.245245  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.247594  6457 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:49.245611  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:49.251387  6300 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:49.251997  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66806.1us
W20260709 12:27:49.252918  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 83975.7us
I20260709 12:27:49.254470  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:49.254856  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.257122  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.267903  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.268903  6378 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:27:49.277125  6457 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:27:49.282531  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.193207 13398 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:49.291278  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 127430us
I20260709 12:27:49.292754 13434 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:49.294600 13434 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:49.295439 13434 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:49.230355 13210 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.224296 13400 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:49.298784  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44512.2us
I20260709 12:27:49.290395  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:49.250412 13393 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:49.239987 13367 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.304826  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:49.305358  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59825.5us
I20260709 12:27:49.304991  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:49.311229  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.270750 13397 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:49.281612 12974 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.312950  6458 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.310243  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.312584  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.289038 13398 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:49.328670  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 187113us
I20260709 12:27:49.331053  6459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:49.331516  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:49.305913 13258 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:49.333210  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:49.329557  6425 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.555s (SENT)
I20260709 12:27:49.335013  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.336166  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.246269 13428 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 12:27:49.310762 13313 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:49.339099  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.341173  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.345983  6310 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:49.346577  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:49.347293  6302 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:49.310864 13090 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:49.338925  6307 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:49.331630  6459 raft_consensus.cc:606] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:49.342818  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.311362 13210 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:49.349783  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.343902  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.344173  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:49.350728  6457 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:49.348886 13392 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:27:49.351621 13425 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.352291 13392 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:27:49.352705 13392 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:49.353152 13435 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:49.353644 13435 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.353546 13392 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.341845  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:49.354558 13090 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.354781  6304 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:49.355024 13090 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:49.355768 12840 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:27:49.349524  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.355794  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.354200 13435 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:49.344552  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87532.3us
I20260709 12:27:49.358441  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:49.359212  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.359998  6385 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
W20260709 12:27:49.361567  6425 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.988s (SENT)
I20260709 12:27:49.359047  6305 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:49.318562 13429 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:49.340420 13393 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:49.331137 13398 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.363175  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.386124  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.386819  6378 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:49.364434  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.341524 13258 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.362915  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.372656  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:49.373332  6458 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:27:49.389750  6458 pending_rounds.cc:85] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 8
I20260709 12:27:49.390089  6458 pending_rounds.cc:107] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 14.9
W20260709 12:27:49.379552  6425 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.996s (SENT)
I20260709 12:27:49.387667  6302 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:49.390319  6458 raft_consensus.cc:2889] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.390710  6458 pending_rounds.cc:107] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:27:49.391196  6458 raft_consensus.cc:2889] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.345557 13313 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.389460  6459 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:49.391851  6310 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:49.390668  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.367455 13007 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.322688 13397 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.405057 13007 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.406795 13397 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.406623 13196 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.407240 13007 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.408823 13196 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.356979 13094 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:49.410722 13061 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.411468 13372 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:27:49.412864 13061 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.341037 13248 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.414824  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:49.415544  6304 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:49.396397 13429 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:49.416692  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.417014 13429 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.418535 13433 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:49.419602  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:49.322093 13342 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.422279 13342 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.394655 13258 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:49.415575 13248 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.427325  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.395637  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:49.427754  6451 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:49.418731  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:49.426574 13313 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.442729  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.410954 13164 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:49.431738  6302 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:27:49.433159  6378 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:27:49.435986  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.437870  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:49.428609  6305 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:49.438611  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:49.448385  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.440783  6457 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:49.340754 12974 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.455868  6450 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:27:49.449716  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.456511  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:49.451649 13381 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:49.454452  6459 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:49.383481 13393 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:49.410364 13250 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.464136  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.359786 13210 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.460675  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.468880  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:49.450449 13431 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:27:49.468966  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 112254us
I20260709 12:27:49.464515  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:49.461651 13411 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:27:49.476243  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.481770  6304 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:27:49.482299  6302 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:49.446338 13392 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:27:49.475630  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 96134.3us
I20260709 12:27:49.487514  6457 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:49.479460  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.483872  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.484288  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:49.477128  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118470us
I20260709 12:27:49.469873 13393 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:49.496219  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.409701 13421 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.499768  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:49.473875 12809 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:49.502503  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.503728  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.453496 12840 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:27:49.509186  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:49.509994  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.510869  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:49.545933  6379 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:49.504801 13250 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:49.563313 13381 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:49.564577  6385 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:49.565066  6379 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:27:49.564885  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54944.5us
I20260709 12:27:49.566195  6385 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.511348 12809 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:49.567118  6379 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:49.568274  6304 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 13.
I20260709 12:27:49.566581  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:49.591684  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:49.594947  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.499151 13007 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:27:49.561290 13397 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.601552  6385 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:49.601343  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.513700 13383 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:27:49.611649  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.517097 13391 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:27:49.608635  6385 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:49.624986  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:49.605208  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 267599us
I20260709 12:27:49.618918  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:49.630995  6307 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
W20260709 12:27:49.631631  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132780us
I20260709 12:27:49.632530  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.638921  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:49.634243 13451 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:27:49.645197  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.609026 13248 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:49.639657  6305 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:49.650440  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.617744 13382 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:49.609148 13446 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:49.620622 13357 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:49.620793 13388 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:49.662902  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.627947 13375 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.663853  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.665241  6307 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.609650 13381 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:27:49.667616  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:49.646965 13342 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:49.670040  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:49.674736  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 352710us
I20260709 12:27:49.660414 13094 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:49.654657 13248 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.614822 13397 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.692487  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.695958  6309 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:49.670400 13446 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:49.706101  6302 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:49.682211 13304 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:49.684137 13382 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:49.685141 13248 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:49.713557  6425 leader_election.cc:400] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19.
I20260709 12:27:49.690979 12974 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.695765 13094 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:49.677102 13375 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.714875 13304 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.684398 13090 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.716857 13382 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.713487 13460 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:49.731319 13304 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.708577 13444 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:27:49.733217  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 220549us
I20260709 12:27:49.703307 13007 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:27:49.683996 12809 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:49.728840 13115 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.003s
I20260709 12:27:49.746747 13455 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:49.747251 13468 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:27:49.762428  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84006us
I20260709 12:27:49.738718 13382 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:49.759007  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:49.767233  7329 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:49.751929 13421 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.717993 13457 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:27:49.780305  6459 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:49.781364 13470 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:27:49.782159  6456 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:49.759564 13455 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.767089 13115 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:49.765242 13444 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:49.791095 13455 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:49.797542  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:49.766830 13094 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.787241 13460 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:49.808498  6377 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.809783  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:49.810613  7329 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
W20260709 12:27:49.797394 13192 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.000s
W20260709 12:27:49.817588  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.818564  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:49.825938 13090 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.827286 13398 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.828480  6380 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:49.829860 13439 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:49.830310 13439 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:49.830681 13221 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.831153 13439 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.832476  6304 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:49.833647  6304 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:49.833851 13375 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:49.834690 13471 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:27:49.821561 13453 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.837100 13453 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.847187  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:49.855103  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:49.856941  7329 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:49.859242  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.864948  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:49.822232 13192 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:49.872279  7317 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:27:49.872368  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.872578  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.776942 13446 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.806767 13258 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.876979  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.879817  6304 raft_consensus.cc:2368] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26.
W20260709 12:27:49.880548  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.854725 13375 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:49.864953 13407 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.666s	user 0.006s	sys 0.007s
I20260709 12:27:49.890368 13407 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.889290  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.895485  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.896391  6383 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:27:49.898465  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.899310  6304 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:49.896304 13397 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.904729  7314 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:49.869899 13391 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:27:49.865559 13248 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.867020 13421 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:27:49.863278 13313 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.902225 13407 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:49.945117  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.948015  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:49.900835 13375 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.950160  6304 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:49.951115  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.951368  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.953888  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.955207 13421 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:49.955785 13421 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.957806 13421 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:49.958606 13463 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:27:49.959455  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:49.959956 13476 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:49.944208 13477 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:27:49.952111 13460 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:49.961877 13477 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:49.963101 13090 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:49.964094 13090 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:49.965216 13090 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.963065 13460 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.967470 13477 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:27:49.966910 13342 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:49.966846 13090 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:49.975281 13258 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:49.976981 13313 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.911198 13444 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:49.978981 13473 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:49.979918 13473 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:49.989709  7288 leader_election.cc:400] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26.
I20260709 12:27:49.990971  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.991569  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.993342  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:49.994567  6304 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:49.997608  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:49.953399 13391 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:50.000774  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:50.002847  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:49.958477 13421 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:50.003726  7318 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:50.005401  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:50.008453  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:49.997900 13393 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.013558  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.014384  6307 raft_consensus.cc:2376] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18. Re-sending same reply.
I20260709 12:27:50.014979 13478 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 12:27:50.016136 13487 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:50.005724 13404 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:50.024343  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.024834  6459 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:50.023301  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3059) took 1085 ms. Trace:
I20260709 12:27:50.027612  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.028026  6458 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:50.025681  6301 rpcz_store.cc:276] 0709 12:27:48.937709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.937829 (+   120us) service_pool.cc:224] Handling call
0709 12:27:48.938133 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.013707 (+ 75574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.023067 (+1009360us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b5400206cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:50.023241 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172152192}
I20260709 12:27:50.034289  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:50.035265  6383 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:50.016516 13487 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:50.040632 13488 ts_tablet_manager.cc:933] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:27:50.043431 13404 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:50.044584 13474 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:50.044430 13404 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.045795 13474 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:50.046168 13474 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:50.013602 13393 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:49.996536 13484 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:50.047665  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.049170  6303 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:27:50.053996  6303 pending_rounds.cc:85] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:27:50.054270  6303 pending_rounds.cc:107] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:27:50.054570  6303 raft_consensus.cc:2889] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:50.016929 13421 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:50.035207 13342 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:49.996183 13455 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.005445 13387 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.066082 13387 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:50.053224 13407 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:50.067397  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.068706  6385 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:50.069425 13313 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:50.069702 13421 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.070089 13313 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.071135 13421 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:50.074914  6303 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:50.075327 13391 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.080184 12834 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:50.081341  6308 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 12. Current term is 13. Ops: []
I20260709 12:27:50.082878 13398 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:50.083034 13487 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.098129 13453 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:50.047068 13474 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.149459  6378 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:50.149753 13474 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:50.152479  6305 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:50.129038 13445 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.456s	user 0.002s	sys 0.000s
I20260709 12:27:50.160138  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:50.160837  7316 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:50.130211 13455 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:50.138513 13435 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.748s	user 0.008s	sys 0.010s
I20260709 12:27:50.166363  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:50.131424  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:50.131850 13210 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.132009 13007 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:50.131862 13366 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:50.133028  6381 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:50.133565  7314 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:50.133749  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3057) took 1196 ms. Trace:
I20260709 12:27:50.087579 13444 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:50.137148  6304 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
W20260709 12:27:50.137595  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 272997us
I20260709 12:27:50.140038  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:50.142185  7318 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:50.134837  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:50.183121  6310 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: [20.15-22.16]
I20260709 12:27:50.147519  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:50.148689  7320 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:50.153353  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:50.183422  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:50.162112  7329 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:50.135077 13381 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:50.172276  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.173771  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:50.131472 13375 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:50.179152  6308 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:27:50.179803  6382 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:50.139495 13367 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:50.183923  6383 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:50.187613  6385 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:50.158036 13416 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.158349 13425 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:50.227303  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.219612  6304 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:50.181630  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:50.233830  6450 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:50.182008  6300 rpcz_store.cc:276] 0709 12:27:48.937073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.937262 (+   189us) service_pool.cc:224] Handling call
0709 12:27:48.937728 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:49.251782 (+314054us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.133486 (+881704us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54002df0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:50.133685 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720934656}
I20260709 12:27:50.235929  6452 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:50.237708  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:50.238612  6308 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:50.237712  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:50.229866 13381 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:50.216535 12809 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:50.181357 13210 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:27:50.219702 13411 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:50.188695  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:50.236171 13450 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:27:50.242103  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60562.7us
I20260709 12:27:50.240305  6385 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:50.164652 13397 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.069s
W20260709 12:27:50.243628  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66203us
I20260709 12:27:50.233788  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:50.182015 13221 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
W20260709 12:27:50.231673 13413 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:50.217069 13433 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:50.218631 13367 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:50.226105 13428 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.264894  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:50.266957  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 78340.9us
I20260709 12:27:50.268411  6382 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:50.151834 13304 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.181627 13366 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:50.217631 13388 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.069s
I20260709 12:27:50.244676 13473 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.186762 13007 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.179270 13435 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:50.305279  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78882.1us
I20260709 12:27:50.261509 12834 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:50.304849  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:50.262127 13313 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.299571  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:50.239799 13425 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:50.315632  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50867.1us
I20260709 12:27:50.290578 13210 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.237950 13416 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:50.298564 13437 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.890s	user 0.001s	sys 0.004s
I20260709 12:27:50.295028 13490 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:50.325012  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:50.254592 13484 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.089s
I20260709 12:27:50.297384 13394 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
W20260709 12:27:50.357071  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52252.3us
I20260709 12:27:50.357730  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:50.285094 13453 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:27:50.342821  7332 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:50.267445 13358 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:27:50.304919 13357 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:27:50.309530 13433 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:50.355105  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:50.420333  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.424983  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 81286us
I20260709 12:27:50.371816  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
W20260709 12:27:50.383100  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 209371us
W20260709 12:27:50.415030  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57381.4us
I20260709 12:27:50.428846  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:50.429126  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:50.361369  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:50.341936 13196 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.333114 13488 tablet_copy_client.cc:323] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:27:50.308961 13428 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.359243 13455 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 12:27:50.461041  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.478991  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54661.2us
I20260709 12:27:50.361003 12974 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:50.483120  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.439695  7331 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:50.417141 13397 consensus_queue.cc:1243] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:50.439013  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2811) took 1096 ms. Trace:
I20260709 12:27:50.362465 13210 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.363670 13216 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:50.520416  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:50.517913  6307 rpcz_store.cc:276] 0709 12:27:49.342402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.015234 (+672832us) service_pool.cc:224] Handling call
0709 12:27:50.015599 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.015800 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.015821 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.016356 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.340525 (+324169us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:50.340678 (+   153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.340770 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.438621 (+ 97851us) raft_consensus.cc:1685] finished
0709 12:27:50.438670 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.438913 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":323701,"threads_started":1,"wal-append.queue_time_us":417040}
I20260709 12:27:50.375499 13429 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.111s
W20260709 12:27:50.521818  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.526090  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.434726 13357 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:50.422348 13313 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.091s
I20260709 12:27:50.373193 13367 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.422206 13508 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:50.319494 13381 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:50.439188 13443 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.034s	user 0.000s	sys 0.004s
I20260709 12:27:50.533654  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.431411 13384 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.101s
I20260709 12:27:50.541036  6308 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:50.542245  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.542639  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:50.543813  6308 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:50.529148  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 164642us
I20260709 12:27:50.417634 13452 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.354s
I20260709 12:27:50.529109 13216 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:50.533385 13358 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:50.559561  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.479363 13196 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.552778  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1021) took 1883 ms. Trace:
I20260709 12:27:50.561080  6307 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:50.561823  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.564393  7316 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:27:50.532377 13357 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:50.421247 13366 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.561100  6384 rpcz_store.cc:276] 0709 12:27:48.669849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.669992 (+   143us) service_pool.cc:224] Handling call
0709 12:27:48.682861 (+ 12869us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:48.710893 (+ 28032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.710939 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:49.053854 (+342915us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b1800250f60. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a
0709 12:27:49.058959 (+  5105us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:27:49.059688 (+   729us) log.cc:844] Serialized 52 byte log entry
0709 12:27:49.328293 (+268605us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:49.617102 (+288809us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.617989 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.618079 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.101576 (+483497us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:50.552557 (+450981us) raft_consensus.cc:1685] finished
0709 12:27:50.552588 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.552719 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90680,"prepare.queue_time_us":810387,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":76838,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":10872,"spinlock_wait_cycles":84217216,"thread_start_us":458344,"threads_started":2,"wal-append.queue_time_us":348764}
I20260709 12:27:50.554715 13473 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:27:50.576541  6307 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:50.551851  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:50.559919 13216 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:50.566201 13384 consensus_queue.cc:1243] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:27:50.564608 13388 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.575860  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:50.604408  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.573952 13358 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:50.555433 13433 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.610576 13358 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:50.611780 13497 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:27:50.608480 13435 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.616511  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.620951  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.616765  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:50.621838  6451 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:50.582072 13304 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:50.627197  6458 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:50.628906  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.629698  6454 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:50.616349 13499 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:27:50.622677 13493 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:27:50.613312 13366 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:50.635895  6451 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:50.614987 13007 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:50.645763  6307 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:50.602370 13388 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.650686  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:50.647367  6307 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:50.616707 13452 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:27:50.634415 13499 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:50.633842 13425 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:50.667161  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:50.667773  6451 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:50.674080 13474 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.004s
I20260709 12:27:50.656898 13007 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.646335 13416 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:50.684099  6451 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:50.654407  6307 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:27:50.676756 13007 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.691783  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1037) took 1005 ms. Trace:
I20260709 12:27:50.664985 13486 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:50.698634  6454 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:50.630438 13357 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.705622  6307 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:27:50.694769 13509 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:27:50.713320 13421 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.642s	user 0.007s	sys 0.001s
I20260709 12:27:50.706679 13486 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.717754 13357 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:50.724438 13505 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:27:50.726404  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3078) took 1253 ms. Trace:
I20260709 12:27:50.678597 13428 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:50.704475  7332 rpcz_store.cc:276] 0709 12:27:49.686760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.686880 (+   120us) service_pool.cc:224] Handling call
0709 12:27:49.700122 (+ 13242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.401065 (+700943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.691531 (+290466us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b540016dd54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:50.691722 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1477696384}
I20260709 12:27:50.733554 13486 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:50.734777 13499 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.736629 13499 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:50.739508 13007 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.705631 13210 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.726832  6303 rpcz_store.cc:276] 0709 12:27:49.472658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.078937 (+606279us) service_pool.cc:224] Handling call
0709 12:27:50.079304 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.166382 (+ 87078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.166416 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.231202 (+ 64786us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.342877 (+111675us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:50.367904 (+ 25027us) log.cc:844] Serialized 14 byte log entry
0709 12:27:50.368143 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.368209 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.726029 (+357820us) raft_consensus.cc:1685] finished
0709 12:27:50.726052 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.726330 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87155,"prepare.queue_time_us":74438,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":68,"raft.queue_time_us":86798,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":48641,"spinlock_wait_cycles":8620032,"thread_start_us":55883,"threads_started":1,"wal-append.queue_time_us":181518}
I20260709 12:27:50.751261 13196 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:50.752864  6302 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:27:50.754359  6302 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:50.733985 13388 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:50.758872  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 330122us
I20260709 12:27:50.759819  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:50.760015 13221 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:50.760421 13221 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:50.760504  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:50.760906  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.761525  7314 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:27:50.762789 13397 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:50.763505 13397 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.764202 13397 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.764554 13504 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:50.766966  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:50.763576 13382 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.985s	user 0.015s	sys 0.000s
W20260709 12:27:50.772763  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 345121us
I20260709 12:27:50.775832  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:50.780154  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:50.781955  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:50.796146  7332 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:50.796226  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:50.784497  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:50.785844  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:50.817574  6301 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:50.818137  6301 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:50.819348  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:50.821666  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:50.822031  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:50.786681  6307 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:27:50.825196  6307 pending_rounds.cc:85] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 8
W20260709 12:27:50.832435  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 46731.2us
I20260709 12:27:50.822479  7320 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:50.761608 13221 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:50.821120  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:50.783788 12809 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:50.819198  6301 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.850976  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:50.783550 13388 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:50.795274 13474 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.859131  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:50.867719  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.795151 13210 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:50.814867 13477 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:50.825551  6307 pending_rounds.cc:107] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.9
W20260709 12:27:50.905542  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54677.1us
I20260709 12:27:50.906024  6307 raft_consensus.cc:2917] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Aborting config change with OpId 14.9: Aborted: Op aborted by new leader
I20260709 12:27:50.906677  6307 raft_consensus.cc:2889] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:50.906827  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:50.834529 13397 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:50.820349 13486 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:27:50.902940  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.880375  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.768780 13196 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:27:50.784253 13452 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 12:27:50.931586  6301 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:50.936456  7335 raft_consensus.cc:1217] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Deduplicated request from leader. Original: 13.10->[14.11-18.13]   Dedup: 14.11->[16.12-18.13]
I20260709 12:27:50.856231 13164 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:50.939611  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.953215 13221 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:50.857441 13455 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:50.783876 13440 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 12:27:50.792896 13382 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.874723 13416 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:50.957955  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:50.907429 13521 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:50.849215 13515 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:50.849704 13394 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:50.959394  7335 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: [14.11-18.13]
W20260709 12:27:50.959517  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 199788us
I20260709 12:27:50.985280  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:50.965577  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:36232 (request call id 2) took 6723 ms. Trace:
I20260709 12:27:50.871207 13384 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 12:27:50.963198 13486 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:50.863760 13258 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:50.982616  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:50.901703 13474 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.907785 13388 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.015650  6457 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:27:51.024966  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.015969  6455 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:27:50.986209  6307 rpcz_store.cc:276] 0709 12:27:44.234436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.234654 (+   218us) service_pool.cc:224] Handling call
0709 12:27:46.108233 (+1873579us) spinlock_profiling.cc:243] Waited 46.1 ms on lock 0x7b580017ffd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa32a270a 00007f6fa32a376a 00007f6fa32a291b 00007f6fa32ae03c 00007f6fa32ad6b3 00007f6fa826dd29 00007f6fa17ec1d4 00007f6fa17ec162
0709 12:27:46.213769 (+105536us) log.cc:844] Serialized 485 byte log entry
0709 12:27:46.446813 (+233044us) spinlock_profiling.cc:243] Waited 1.39 ms on lock 0x7b180019c060. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa3200b0b 00007f6fa3285b63 00007f6fa32ad752 00007f6fa826dd29
0709 12:27:46.462418 (+ 15605us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b180019c060. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:27:46.463670 (+  1252us) spinlock_profiling.cc:243] Waited 22.4 ms on lock 0x7b180019c060. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3220142 00007f6fa3200eaa 00007f6fa3207d53
0709 12:27:46.464139 (+   469us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b54003159a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3220142 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634 0000000000388479 00007f6fa01ba475
0709 12:27:46.547159 (+ 83020us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b180019c060. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:27:50.957448 (+4410289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2343,"raft.queue_time_us":49566,"raft.run_cpu_time_us":4584,"raft.run_wall_time_us":57638,"spinlock_wait_cycles":224967040,"thread_start_us":127269,"threads_started":1,"wal-append.queue_time_us":195008,"wal-append.run_cpu_time_us":2747,"wal-append.run_wall_time_us":1027818}
I20260709 12:27:50.954161 13484 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:50.930948 13210 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:51.021438  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 77850.6us
I20260709 12:27:51.029780  6301 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:27:51.030336  6301 pending_rounds.cc:85] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 9
I20260709 12:27:51.032186  6301 pending_rounds.cc:107] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:27:51.032544  6301 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:51.043282 13525 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:51.043826 13525 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.043943  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.044219 13525 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:51.015538 13394 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.045233  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:50.986994 13452 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:50.945413 13477 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.061475  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:50.989099 13007 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:27:50.980520 13416 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:50.981598 13455 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.063701  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:51.051069 13504 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:51.079097 13523 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:51.080005  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2: Aborted: Op aborted by new leader
I20260709 12:27:51.080430 13534 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:51.081236 13534 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:51.082796 13532 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:51.083369 13532 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:51.083702 13455 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.084299 13532 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.085142 13455 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.085606 13455 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.088225 13477 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.092614  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.093262  6308 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:51.094095  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:51.095851 13416 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.074980  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.108816 13455 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.110342 13525 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.112224 13382 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.113536 13525 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.117537 13388 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.121366 13210 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.069033 13452 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.124404 13452 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.125739  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:51.126570  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:51.128993 13394 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.129333 13474 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:51.131259  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56368.9us
I20260709 12:27:51.062845 13216 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.132083  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.133015 13258 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:51.133692 13258 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:51.136878 13460 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:51.138698 13460 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:51.138882 13534 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:51.139837 13534 consensus_queue.cc:1243] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:27:51.078321 13487 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:51.148435  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:51.152110  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:51.152248  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:27:51.153473  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet 521c92260b1b4a6abba1ce9d3c7c5ad2 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-521c92260b1b4a6abba1ce9d3c7c5ad2
I20260709 12:27:51.154986  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:51.155658 12974 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:51.156455 13428 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:51.139962 13460 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.157267 12974 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:51.158649  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.159263 13367 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:51.152693 13511 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.136202 13416 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.176234  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.134281 13258 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 12:27:51.183595  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.134392 13216 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.148216  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:51.175925  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.181509  6301 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:51.187479  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:51.188953  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:51.194059  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:51.207144  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.177327 13534 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:51.223629  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:51.228597 13535 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:51.230203 13540 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:51.234129 13366 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.590s	user 0.012s	sys 0.000s
I20260709 12:27:51.235704 13540 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:51.236157  6320 tablet_copy_source_session.cc:215] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:51.237267 13525 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:51.237752 13525 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:51.238828 13525 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.240127 13488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 521c92260b1b4a6abba1ce9d3c7c5ad2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:27:51.201735 13504 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:51.243567 13504 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.244026 13504 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:51.222466 13388 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:51.248912 13388 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.249390 13388 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:51.192967 13428 raft_consensus.cc:2764] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
W20260709 12:27:51.259733  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.261404  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.262337  6301 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:51.263638  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.333058  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.405880  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.412001  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.219125 13521 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:27:51.448953  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148070us
I20260709 12:27:51.450299  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2810) took 2108 ms. Trace:
I20260709 12:27:51.460358  6310 rpcz_store.cc:276] 0709 12:27:49.342057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.015060 (+673003us) service_pool.cc:224] Handling call
0709 12:27:50.182882 (+167822us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:50.183466 (+   584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.449965 (+1266499us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54000b4d50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:51.450212 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385092992}
I20260709 12:27:51.220306 13397 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:51.558373  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.445195 13216 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.219195 13210 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.301s
I20260709 12:27:51.191103 13416 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.445132 13382 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:51.517941 13313 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.578313  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.003s	sys 0.023s
W20260709 12:27:51.614213  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.004s	sys 0.023s
W20260709 12:27:51.614979  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 818793us
I20260709 12:27:51.615221  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.453204 13521 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.547159 13007 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.585311  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.025s	sys 0.002s
I20260709 12:27:51.444849 13164 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:51.616360  6301 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:51.181612 13511 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.527596 13196 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.557911 13094 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.625530  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.026s	sys 0.003s
I20260709 12:27:51.223434 13533 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.366s
I20260709 12:27:51.638207  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:51.682173 13522 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:51.571696 13535 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:51.683089 13522 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.683610 13535 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.686034 13522 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.693363  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:51.654210  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:51.656503 13511 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.657631 13470 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:51.658587  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.658324 13196 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.661034  6458 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:51.668702 13164 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.682299  6377 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.564303 13428 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:51.615005 13499 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.878s	user 0.013s	sys 0.003s
I20260709 12:27:51.687088 12834 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:27:51.646783  6308 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.004s	sys 0.025s
I20260709 12:27:51.686643 13535 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.699035 13094 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.621315 13313 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.688295  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:51.689879 13541 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:51.696399 13367 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.519s	user 0.010s	sys 0.000s
I20260709 12:27:51.710708  6305 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:51.585278 13486 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:27:51.716131  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.001s	sys 0.033s
W20260709 12:27:51.785699  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.002s	sys 0.033s
I20260709 12:27:51.618817 13358 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:51.728639 13470 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.791697 13541 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.791936 13358 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.792119  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.793627 13547 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:51.794534 13358 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:51.737820  6302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.020s	sys 0.013s
W20260709 12:27:51.795308  6302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.020s	sys 0.013s
I20260709 12:27:51.795449 13547 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:51.798309 13313 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.800050 13546 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:51.800729 13546 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.806634 13094 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.739732 13210 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.807713 13210 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.814968  6452 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:27:51.820195 13546 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:51.822664 13221 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.859s	user 0.010s	sys 0.003s
I20260709 12:27:51.635033 13382 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.748757  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.838404  6385 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
W20260709 12:27:51.751724  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:51.755697  6308 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.005s	sys 0.025s
W20260709 12:27:51.715633 12809 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.016s	sys 0.016s
I20260709 12:27:51.675585 13383 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.755893  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:51.769927  6301 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:51.587110 13216 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.765878  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:51.778414  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.785003  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:51.855798  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.738737 13542 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:51.641480 13521 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:51.738510  6303 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.877s	user 0.013s	sys 0.018s
W20260709 12:27:51.859391  6303 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.014s	sys 0.018s
I20260709 12:27:51.738931 13387 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:51.794122 13470 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.754557 13428 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.754885 12834 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:51.646060 13007 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.851639  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.851917  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.857733  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:51.858429  6300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.013s	sys 0.025s
W20260709 12:27:51.847432 12809 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.017s	sys 0.017s
I20260709 12:27:51.828612 13210 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.845407  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:27:51.868362  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:51.865513 13388 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.615s	user 0.013s	sys 0.001s
I20260709 12:27:51.875360  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:51.872905  6300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.013s	sys 0.026s
I20260709 12:27:51.882827  6383 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:51.835669 13486 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.886104  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:51.887359  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:51.888492  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:51.889883 13210 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:51.890847 13007 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:51.891994 13544 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 12:27:51.894290  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2823) took 1573 ms. Trace:
I20260709 12:27:51.894842  6303 rpcz_store.cc:276] 0709 12:27:50.321417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.748962 (+427545us) service_pool.cc:224] Handling call
0709 12:27:50.749400 (+   438us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.773839 (+ 24439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.773871 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.784818 (+ 10947us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.859763 (+1074945us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:51.859804 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.859868 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.894010 (+ 34142us) raft_consensus.cc:1685] finished
0709 12:27:51.894039 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.894222 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24004,"raft.queue_time_us":24529,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":18904,"thread_start_us":997975,"threads_started":1,"wal-append.queue_time_us":1060101}
W20260709 12:27:51.897228 13487 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.756s	user 0.009s	sys 0.000s
I20260709 12:27:51.858659 13542 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:51.871307 13387 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:51.861020 13383 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.884454 13313 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:51.897442  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.924281  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.924706 13090 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:51.925112  6303 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:51.925396 13090 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:51.916436 13428 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.930668 13196 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.934124 13382 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:52.004738  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 329083us
I20260709 12:27:51.935617 13523 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:51.935926 13216 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.937255 13416 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.876410 13535 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:51.901433 12834 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.943317  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:51.943012  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.945022  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.912593  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.003337 13540 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.077227  6383 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:51.913816  6381 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:27:51.916092  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:52.077946  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.912153  6305 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:52.078385  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.079010  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.079185  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.949708  6379 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:52.079679  6376 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:51.955574  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.042181 13523 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:51.904927 13504 consensus_meta.cc:287] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.660s	user 0.010s	sys 0.002s
I20260709 12:27:51.880247 13553 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.091420  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:51.977908  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.983296  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.092392  6380 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
W20260709 12:27:51.919672 13502 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.283s	user 0.006s	sys 0.021s
I20260709 12:27:51.927047 13515 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:51.997581  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.905543 13541 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:51.992158  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:52.111619  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 119574us
I20260709 12:27:52.112607  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:51.914183 13486 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.939085 13381 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:52.000705  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:51.903875 13387 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:51.884945 13534 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:51.905197 13383 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:51.944067 13388 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:51.982728 13487 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.042977  6456 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:27:52.079288  6383 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:52.073891  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:52.131433  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57673.5us
I20260709 12:27:52.080562  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.081030  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.133936  6301 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:52.079679  6305 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:52.092345  6308 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
W20260709 12:27:52.099052  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 155868us
I20260709 12:27:51.898522 13542 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.103380  6377 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:52.121349 13494 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.119922 13470 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.125321 13516 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:52.123183  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.101130 13375 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:52.083073 13416 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.132835  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.132562  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:52.110277 13515 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:52.135281  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:52.121711 13382 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:52.115864  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.143505  6450 raft_consensus.cc:2368] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:27:52.143592  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3090) took 1645 ms. Trace:
I20260709 12:27:52.073982 13535 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.131660 13094 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.103017 13504 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.187362  6381 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:52.154958  6303 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:52.188818  6302 rpcz_store.cc:276] 0709 12:27:50.502469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.756795 (+254326us) service_pool.cc:224] Handling call
0709 12:27:50.757113 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.779567 (+ 22454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.779600 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.780089 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.795690 (+1015601us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:52.142775 (+347085us) log.cc:844] Serialized 14 byte log entry
0709 12:27:52.143158 (+   383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.143260 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.143331 (+    71us) raft_consensus.cc:1685] finished
0709 12:27:52.143350 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.143513 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21993,"raft.queue_time_us":26786,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6186,"spinlock_wait_cycles":44065152,"thread_start_us":1362064,"threads_started":2,"wal-append.queue_time_us":1022764}
I20260709 12:27:52.200665  6303 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:52.207360  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1084) took 1062 ms. Trace:
I20260709 12:27:52.207811  6303 rpcz_store.cc:276] 0709 12:27:51.144733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.200059 (+1055326us) service_pool.cc:224] Handling call
0709 12:27:52.200431 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.207031 (+  6600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.207289 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:52.081185 13090 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.210968 13090 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:52.211869 13500 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:52.212347 13500 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.214306 13500 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.136302 13487 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.143391 13196 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:52.185818  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:52.187227  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52031us
I20260709 12:27:52.185959  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:52.191918  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.007s	sys 0.056s
I20260709 12:27:52.153684  6301 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:52.221613  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1083) took 1079 ms. Trace:
I20260709 12:27:52.221504  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:52.156260  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:52.201581  6302 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:27:52.222096  6301 rpcz_store.cc:276] 0709 12:27:51.144323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.141329 (+997006us) service_pool.cc:224] Handling call
0709 12:27:52.153390 (+ 12061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.221394 (+ 68004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.221564 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:52.159917  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.224928  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.012s	sys 0.061s
W20260709 12:27:52.157428 13550 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:27:52.159029 13542 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:52.128266 13534 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.110929 13553 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.222968  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1085) took 1078 ms. Trace:
I20260709 12:27:52.219648  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3179) took 2191 ms. Trace:
W20260709 12:27:52.225327  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 606557us
I20260709 12:27:52.262629  6302 rpcz_store.cc:276] 0709 12:27:51.145076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.200974 (+1055898us) service_pool.cc:224] Handling call
0709 12:27:52.201353 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:52.222710 (+ 21357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.222917 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:52.245893 13375 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:52.216491 13094 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.229956  6308 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:52.230228  7318 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:52.167083 13516 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:52.189220 13382 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.152094 13470 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.217398 13511 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:52.151038 13540 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.160000 13416 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.225647 13196 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.257544  6381 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:27:52.258873  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 126449us
I20260709 12:27:52.248106 13563 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.262831  6459 rpcz_store.cc:276] 0709 12:27:50.026240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.028535 (+  2295us) service_pool.cc:224] Handling call
0709 12:27:50.028834 (+   299us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:50.039828 (+ 10994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.039861 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:50.157060 (+117199us) log.cc:844] Serialized 52 byte log entry
0709 12:27:50.414109 (+257049us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:50.414242 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.414424 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.918832 (+1504408us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:52.205804 (+286972us) raft_consensus.cc:1685] finished
0709 12:27:52.205829 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.218048 (+ 12219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10539,"raft.queue_time_us":11353,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":157,"spinlock_wait_cycles":5414784,"thread_start_us":175574,"threads_started":1,"wal-append.queue_time_us":357371}
I20260709 12:27:52.151564 13494 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.272713  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.228071  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:52.270864 13555 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:27:52.279623  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:52.262269 13164 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:52.278613 13467 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:52.311058  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83079.5us
W20260709 12:27:52.319649 13557 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:27:52.319541 13164 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.313158 13375 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:52.320633 13375 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.321031 13375 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:52.322028 13164 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.287346  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2821) took 2593 ms. Trace:
I20260709 12:27:52.318691 13573 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:52.319989  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:52.325085  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:52.302923  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.313843  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1040) took 2007 ms. Trace:
I20260709 12:27:52.326129  6382 rpcz_store.cc:276] 0709 12:27:50.306976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.307151 (+   175us) service_pool.cc:224] Handling call
0709 12:27:50.307535 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.337376 (+ 29841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.337405 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.090375 (+752970us) spinlock_profiling.cc:243] Waited 1.32 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:27:51.097321 (+  6946us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.548418 (+451097us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:51.548465 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.548530 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.313527 (+764997us) raft_consensus.cc:1685] finished
0709 12:27:52.313550 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.313782 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7380,"raft.queue_time_us":29794,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":16128,"spinlock_wait_cycles":3028736,"thread_start_us":450550,"threads_started":1,"wal-append.queue_time_us":880814}
I20260709 12:27:52.323505  6309 rpcz_store.cc:276] 0709 12:27:49.694393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.130785 (+436392us) service_pool.cc:224] Handling call
0709 12:27:50.982332 (+851547us) spinlock_profiling.cc:243] Waited 6.96 ms on lock 0x7b5800134198. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:27:50.982668 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.988073 (+  5405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.988122 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.018750 (+ 30628us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.786113 (+767363us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:51.786153 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.786227 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.281412 (+495185us) raft_consensus.cc:1685] finished
0709 12:27:52.281447 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.287263 (+  5816us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17253,"raft.queue_time_us":121,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":5538,"spinlock_wait_cycles":16007552,"thread_start_us":726288,"threads_started":1,"wal-append.queue_time_us":941567}
I20260709 12:27:52.328310  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3082) took 2185 ms. Trace:
I20260709 12:27:52.282428 13382 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:52.333696  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2838) took 1301 ms. Trace:
I20260709 12:27:52.283898 13342 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:52.328805  6300 rpcz_store.cc:276] 0709 12:27:50.143005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.238727 (+ 95722us) service_pool.cc:224] Handling call
0709 12:27:50.239005 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:50.244524 (+  5519us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:50.244576 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.268207 (+ 23631us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.161629 (+893422us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b18003aa080. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:27:51.882841 (+721212us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:51.882897 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.882989 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.323286 (+440297us) raft_consensus.cc:1685] finished
0709 12:27:52.323308 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.328240 (+  4932us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24473,"raft.queue_time_us":5256,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":11430,"spinlock_wait_cycles":38704512,"thread_start_us":720980,"threads_started":1,"wal-append.queue_time_us":1882279}
I20260709 12:27:52.286926 13500 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:52.286387 13535 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.287272 13196 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:52.330435 13486 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:52.335826 13007 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:52.380554  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.338716 13210 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:52.387517  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.338639 13342 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:52.339569 13532 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.280597 13516 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.398162  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:52.322598 13563 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.400676  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3238) took 3062 ms. Trace:
I20260709 12:27:52.402359  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:52.402745  6378 rpcz_store.cc:276] 0709 12:27:49.337928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.508529 (+170601us) service_pool.cc:224] Handling call
0709 12:27:49.558743 (+ 50214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.149949 (+591206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.400422 (+2250473us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54003e7e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.400616 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358095744}
W20260709 12:27:52.393117 13521 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.525s	user 0.005s	sys 0.007s
I20260709 12:27:52.376482 13500 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.379830 13388 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.380146 13164 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.405232 13486 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:52.422295 13500 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:52.422644 13568 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:27:52.423301 13470 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.423458 13416 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.379628 13562 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:27:52.331146 13573 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.412326 13504 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.395598 13007 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.423888 13342 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.404508 13532 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.407207 13521 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.427937 13563 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.431161 13248 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.417655 13522 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:52.429232 13486 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.417868 13164 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.449326 13467 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:52.442754 13577 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
W20260709 12:27:52.484660 13578 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:52.484689 13563 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:52.485625 13563 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:52.486845 13563 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:52.477506 13313 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.003s
I20260709 12:27:52.492105 13313 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:52.491223 13553 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.483330  6457 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:27:50.341305  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:52.499469  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83991.7us
I20260709 12:27:52.490971 13164 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:52.509249  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.470605 13522 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:52.524008  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.475104 13486 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:52.496245 13534 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.511831  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.534960  6385 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:52.512168  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.536027  6376 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:52.509742  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:52.513432 13248 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.513715 13490 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.548570 13382 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:52.520089  7285 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 2.200s (SENT) [suppressed 17 similar messages]
I20260709 12:27:52.521171 13467 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.522867  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:52.524819  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.552850  6378 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:52.525349 13582 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:27:52.508319  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 173481us
W20260709 12:27:52.555765 13574 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:27:52.555963  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:52.556896 13553 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:52.557327 13553 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.557650 13553 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:52.528029 13488 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43752 (request call id 1257) took 2583 ms. Trace:
I20260709 12:27:52.562850 13488 rpcz_store.cc:276] 0709 12:27:49.947623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.952321 (+  4698us) service_pool.cc:224] Handling call
0709 12:27:50.041228 (+ 88907us) ts_tablet_manager.cc:934] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
0709 12:27:51.096279 (+1055051us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4c002a5780. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1573a20 00007f6fa1584a29 00007f6fa158513f 00007f6fa15492ab 00007f6fa1576f3c 00007f6fa1577461 00007f6fa1577768 00007f6fa72fd2eb 00007f6fa8222d3b 00007f6fa821d008 00007f6fa821c405
0709 12:27:51.238702 (+142423us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:27:52.082896 (+844194us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:52.100704 (+ 17808us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:27:52.477117 (+376413us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:27:52.527329 (+ 50212us) tablet_replica.cc:206] Creating consensus instance
0709 12:27:52.527768 (+   439us) ts_tablet_manager.cc:989] Replica registered.
0709 12:27:52.527920 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":37490,"spinlock_wait_cycles":33111424,"tablet-copy.queue_time_us":81278,"thread_start_us":301105,"threads_started":2}
I20260709 12:27:52.564872 13488 tablet_copy_client.cc:806] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:52.565529 13488 tablet_copy_client.cc:670] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:27:52.527845 13522 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.566042 13553 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.567148 13522 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:52.569227 13553 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.510005 13313 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:27:52.529029 13540 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.532406  6380 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:52.537201  6382 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:27:52.574436  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.575006  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.577801  6382 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:52.595194 13007 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.599104  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:52.600028  6385 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:52.602020  6304 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:52.602578  6304 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:52.605870 13461 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.606719 13461 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:52.622627  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.622854 13573 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.623517 13392 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:52.624017 13392 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.625528  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1082) took 1483 ms. Trace:
I20260709 12:27:52.626073 13392 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.628115 13094 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.629211 13440 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:52.596735 13499 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:52.633859  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.634181 13499 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.630268 13366 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:27:52.631443 13571 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:27:52.625942  6308 rpcz_store.cc:276] 0709 12:27:51.142108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.140388 (+998280us) service_pool.cc:224] Handling call
0709 12:27:52.154274 (+ 13886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.272157 (+117883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.625274 (+353117us) spinlock_profiling.cc:243] Waited 75.4 ms on lock 0x7b54001b3fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.625460 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173448960}
I20260709 12:27:52.632052 13452 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:52.618755 13490 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.639781  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.622049  6304 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.590683 13248 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.651491  6304 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:52.648893 13452 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.642330 13584 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:52.650211  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.658370  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.648423 13583 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:52.660595  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.662168  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.666929  6300 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:52.663056  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.667572  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.663132  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.635893 13382 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.639213 13397 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:52.636931 13534 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.676493  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:52.633882 13440 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:52.679237  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.669790  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.677533  6302 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:52.647183 13573 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.680524  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:52.677987 13400 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:52.694976  6308 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:52.678301 13494 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.696377  6385 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:52.710456  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.711323  6385 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:52.687481 13532 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.708971  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:52.690459 13382 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:52.713559  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.714373  6302 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:52.701128 13400 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:52.700959  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.696492  6304 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:27:52.716781  6304 pending_rounds.cc:85] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 11
I20260709 12:27:52.717199  6304 pending_rounds.cc:107] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:27:52.718639  6304 raft_consensus.cc:2889] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.719120  6304 pending_rounds.cc:107] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:27:52.719631  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.725301  6304 raft_consensus.cc:2889] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.725538  6302 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:52.719966  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.726105  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1076) took 2774 ms. Trace:
I20260709 12:27:52.726465  6304 rpcz_store.cc:276] 0709 12:27:49.951451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.228979 (+277528us) service_pool.cc:224] Handling call
0709 12:27:50.229309 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.601881 (+2372572us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54002feca0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa321edc0 00007f6fa32a91c0 00007f6fa32aa231 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c
0709 12:27:52.725836 (+123955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.726054 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":663758080}
I20260709 12:27:52.686908 13534 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.729238  6459 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:52.687563 13583 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.732064 13248 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.687728 13397 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.675243 13584 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:52.735292  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 215220us
I20260709 12:27:52.707966 13461 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:52.729242 12840 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:27:52.721043 13571 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:52.735520  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.737577  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.757761  6302 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:52.745649  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.758706  6309 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 13.
I20260709 12:27:52.730990 13384 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.719597 13440 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.764595  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:52.764875 13440 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:52.762436 13490 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.766005 13461 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:52.763736  6304 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:52.705305 13573 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.719698 13382 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.822917  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3196) took 1183 ms. Trace:
I20260709 12:27:52.823316  6458 rpcz_store.cc:276] 0709 12:27:51.640234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.889403 (+249169us) service_pool.cc:224] Handling call
0709 12:27:51.889697 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.902656 (+ 12959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.902688 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.902739 (+    51us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:51.902772 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.822683 (+919911us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.822731 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.822861 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29136128}
I20260709 12:27:52.824553 13382 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.824860  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.825536  6458 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:52.755976 13400 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.828524 12834 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.773757 13583 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.836323  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.837713  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.768599 13384 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.839670  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:52.778954 13587 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:27:52.782472 13573 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.783952 13534 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.757107 13571 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.785717 13452 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.787199 13397 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.860505  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.861755  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:52.787596 13515 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.601s	user 0.012s	sys 0.000s
I20260709 12:27:52.789811 13196 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.792263 13375 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.795671  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3190) took 1247 ms. Trace:
I20260709 12:27:52.859086  6309 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:27:52.796599  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1042) took 1213 ms. Trace:
W20260709 12:27:52.797580  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.334204  6305 rpcz_store.cc:276] 0709 12:27:51.032135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.136182 (+1104047us) service_pool.cc:224] Handling call
0709 12:27:52.136559 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.136855 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:52.276295 (+139440us) log.cc:844] Serialized 14 byte log entry
0709 12:27:52.330333 (+ 54038us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:52.331648 (+  1315us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:52.331733 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.333449 (+  1716us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.333582 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":125961,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":3336,"raft.queue_time_us":1850,"thread_start_us":181489,"threads_started":2}
I20260709 12:27:52.800314 13460 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:52.801803  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.861063  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.872170  6376 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:27:52.803480  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.872957  6302 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:52.805064  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.806283  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.874204  6454 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:52.810046  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3169) took 3033 ms. Trace:
I20260709 12:27:52.811043  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3187) took 1288 ms. Trace:
I20260709 12:27:52.768879 13439 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:52.882603  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:52.884446  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:52.875315  6455 rpcz_store.cc:276] 0709 12:27:49.776548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.776680 (+   132us) service_pool.cc:224] Handling call
0709 12:27:49.781602 (+  4922us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.026050 (+1244448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.809700 (+1783650us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54002c8654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.809972 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2838122240}
I20260709 12:27:52.817333  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3191) took 1258 ms. Trace:
I20260709 12:27:52.885860  6451 rpcz_store.cc:276] 0709 12:27:51.558772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.558955 (+   183us) service_pool.cc:224] Handling call
0709 12:27:51.559380 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.684565 (+125185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.684600 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.684648 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:51.684682 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.817065 (+1132383us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54001626d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.817117 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.817250 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287344512}
I20260709 12:27:52.818882  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3197) took 1139 ms. Trace:
I20260709 12:27:52.820811  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3192) took 1251 ms. Trace:
I20260709 12:27:52.860726 13571 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:52.779721 13589 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:27:52.842041  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.850545  6459 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
W20260709 12:27:52.852540  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 104906us
I20260709 12:27:52.858636  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.868736  6384 rpcz_store.cc:276] 0709 12:27:51.583498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.583631 (+   133us) service_pool.cc:224] Handling call
0709 12:27:51.623128 (+ 39497us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.793463 (+170335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.793494 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.793537 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:51.793564 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.796348 (+1002784us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.796403 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.796536 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":391284608}
I20260709 12:27:52.867908  6452 rpcz_store.cc:276] 0709 12:27:51.548505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.551498 (+  2993us) service_pool.cc:224] Handling call
0709 12:27:51.551875 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.815504 (+263629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.795378 (+979874us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54000c12d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.795600 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603972352}
I20260709 12:27:52.900806  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:52.901696  7329 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
W20260709 12:27:52.902513  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2833) took 1931 ms (client timeout 1968 ms). Trace:
I20260709 12:27:52.866914 13416 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.876717  6453 rpcz_store.cc:276] 0709 12:27:51.522686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.522910 (+   224us) service_pool.cc:224] Handling call
0709 12:27:51.572316 (+ 49406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.862788 (+290472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:51.862816 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.862859 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:51.862914 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.810758 (+947844us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54001e23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.810816 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.810974 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667547392}
W20260709 12:27:52.902989  6307 rpcz_store.cc:269] 0709 12:27:50.971565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.045005 (+ 73440us) service_pool.cc:224] Handling call
0709 12:27:52.902322 (+1857317us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54002e7f54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:52.902439 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189247872}
I20260709 12:27:52.880609  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.904171  6458 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:52.814826 13499 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.886968 13461 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:52.907083  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.828809 13584 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.888818  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.888306  6457 rpcz_store.cc:276] 0709 12:27:51.569249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.569442 (+   193us) service_pool.cc:224] Handling call
0709 12:27:51.572170 (+  2728us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:52.495938 (+923768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.820539 (+324601us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54002c9554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:52.820746 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2094865152}
I20260709 12:27:52.908202  6454 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:52.909123  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.909750  6454 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 12:27:52.910228  6454 pending_rounds.cc:85] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 14
I20260709 12:27:52.910516  6454 pending_rounds.cc:107] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 12:27:52.910888  6454 raft_consensus.cc:2889] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.912830  6454 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 12:27:52.912856  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.859614 13504 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.887511  6450 rpcz_store.cc:276] 0709 12:27:51.679531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.688004 (+  8473us) service_pool.cc:224] Handling call
0709 12:27:52.818639 (+1130635us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400343854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:52.818798 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850616192}
I20260709 12:27:52.883591  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:52.915354  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:52.903455  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.879294  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:52.918164  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:52.918478  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:52.918944  7331 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:52.919256  6455 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:27:52.865548 13397 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.870437 13473 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:52.905197  7318 raft_consensus.cc:1217] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Deduplicated request from leader. Original: 13.10->[14.11-18.13]   Dedup: 16.12->[18.13-18.13]
W20260709 12:27:52.903223  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 119498us
I20260709 12:27:52.814412 13486 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.854295 13425 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.892239  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:52.892335 13386 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:52.917995  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:52.924278  7318 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: [14.11-18.13]
I20260709 12:27:52.924525  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:52.914999 13523 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.000s
I20260709 12:27:52.938469  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.930768  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:52.925109  6457 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:27:52.945046  6457 pending_rounds.cc:85] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:27:52.945343  6457 pending_rounds.cc:107] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:27:52.946254  6457 raft_consensus.cc:2889] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:52.946463  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54371.7us
I20260709 12:27:52.946863  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.936961  6453 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:52.947561  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:52.948740  6385 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:52.947867  7335 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:52.949317 13523 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.949832  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:52.950659  7314 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:52.908783 12974 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:52.910378 13461 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:52.949331  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:52.872013 13515 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.955133  6458 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:52.953155  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:52.922379 13441 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:52.958179  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:52.960392  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:52.929351 13375 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.960713  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:52.951122 13452 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:52.951309 13565 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:52.966862  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:52.928277 13384 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:52.910813 13416 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:52.970359  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.013940  6376 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:27:52.971685  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.015129  7335 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:52.964986 13481 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:52.959023 13461 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:53.010913  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44137.3us
I20260709 12:27:53.011409  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:53.028738  6378 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:52.923463 13504 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.032040 13504 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:52.968218 13258 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.032882 13258 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.033478 13258 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:52.968847 13547 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:52.932600 13473 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.014542  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:53.045614  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:52.961916 13441 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:52.944123 13397 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.048537 13441 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.048995 13386 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:53.049405 13386 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.049758 13386 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:53.050062 13397 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.051870 13547 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:53.052428  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.048723 13216 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:53.052973  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2968) took 1303 ms (client timeout 1653 ms). Trace:
I20260709 12:27:53.053262 13216 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:53.026777  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.053783 13216 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:52.967391 13584 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:53.013727 13221 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:52.952090 13425 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.055689 13221 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.056180 13221 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:53.057451 13425 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:53.058666  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.061008  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.061808  6378 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:53.047663  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:53.062719 13473 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.063472 13474 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:27:53.063915 13474 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.064242 13474 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:53.068202 13487 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.069181 13386 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.070007  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.070662  6382 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:53.070729 13565 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:53.071630  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.072754  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:53.073735  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.074792  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:53.075471 13596 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:53.074743 13216 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:52.969563 13452 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.076503 13397 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:53.077066  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.077246 13452 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.077843 13584 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:53.078310 13584 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:53.079501 13471 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:53.079885 13471 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.080782  6385 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:27:53.080744 13471 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:53.053443  6379 rpcz_store.cc:269] 0709 12:27:51.749906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.750036 (+   130us) service_pool.cc:224] Handling call
0709 12:27:53.052838 (+1302802us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b540022ea54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:53.052923 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52879616}
I20260709 12:27:53.082472  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:53.083323  6301 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:53.083607 13563 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:53.063493  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.087563 13535 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.087880 13386 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.088141 13535 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.091391 13486 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.092150  6378 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:53.093470 13521 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:52.959535 12974 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.095568 13452 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.096088 12974 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.097779 13452 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.099953  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.100601  6453 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:53.101632  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:53.103191  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.104008  6458 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:53.107577 13515 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.110773  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.110983  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.111460  6377 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:53.111960  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.112525  7331 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:53.113386  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.113907 13216 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.117913  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.052872 13547 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.119313  6459 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:53.120008  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.120584  7317 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:53.123222  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:53.123571  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.125200  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.126391  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:53.127012  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.127345  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.128047  7320 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:53.111516  7318 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:53.136857  6378 raft_consensus.cc:1217] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 14.11->[16.12-18.13]   Dedup: 16.12->[18.13-18.13]
I20260709 12:27:53.138069  6378 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: [16.12-18.13]
I20260709 12:27:53.138630  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.139304  6453 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:53.136121  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:53.142109  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.142175 12834 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:53.143227  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:53.144878  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.122660  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.146364  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.147104  7332 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:53.082273 13471 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.150223  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:53.157261  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:53.158044  6300 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.159005 12834 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:53.154036  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:53.156412  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:52.971487 13164 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:53.161003  6301 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:53.161180  6457 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:53.171284  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.090343 13535 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.160552 13534 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.172473 13532 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.172380 13534 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.172955 13532 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.173398 13532 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.174762 12974 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:27:53.175742 13441 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.176856  6376 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:53.162189  7331 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:53.148890 13425 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:53.167438 13593 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:27:53.167397  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.167212  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:53.192176  6308 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:53.205415  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.211643  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:53.212252  6451 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.077132 13221 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.216238  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:53.160984 13521 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:53.168081 13515 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.077116 13397 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.226523  6301 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:53.179232 13523 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.228092  6451 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:53.230001  6453 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:53.148514 13571 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.228690  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.182437 12834 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:53.189589 12974 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:27:53.238072  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:53.240456  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.241156  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:53.230729 13521 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.255401  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.256081  6384 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:53.227628 13196 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.236649 13515 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.278429  7314 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:53.280058  6348 leader_election.cc:400] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:27:53.280565  6348 leader_election.cc:403] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:53.261988 13521 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:53.261029 13164 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:53.248939 13452 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:27:53.297132  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:53.309739  6456 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1150) took 2695 ms (client timeout 2200 ms). Trace:
W20260709 12:27:53.310220  6456 rpcz_store.cc:269] 0709 12:27:50.614299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.614464 (+   165us) service_pool.cc:224] Handling call
0709 12:27:53.309471 (+2695007us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:53.309617 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944872192}
I20260709 12:27:53.311703  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 2989) took 1054 ms. Trace:
W20260709 12:27:53.312306 13597 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:53.258667 13548 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:27:53.273097 13515 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.318717  7329 raft_consensus.cc:2410] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:27:53.292325 13392 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:27:53.324275  6452 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:53.312175  6380 rpcz_store.cc:276] 0709 12:27:52.257023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.257924 (+   901us) service_pool.cc:224] Handling call
0709 12:27:53.311526 (+1053602us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54003d3bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:53.311643 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510842112}
I20260709 12:27:53.293222 13164 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:53.334045  6300 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:53.284492 13384 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:27:53.339000  6308 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:53.319702 13394 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.352782  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.298643 13440 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:27:53.323314 13210 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.342358  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.303841 13090 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:27:53.360514  6380 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:53.365450  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.329272 13439 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:53.371833  7331 raft_consensus.cc:2368] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:27:53.378746  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.374158  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:53.269013 13258 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.332170 13164 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.391184  7314 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:53.337069 13441 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.395876  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.352541 13534 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.375401 13439 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.418727  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.419973  6380 raft_consensus.cc:2410] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:53.370615 13210 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.391567 13366 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:53.386067 13196 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:53.373931 13394 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.401438 13090 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:53.411161 13583 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:27:53.406857 13452 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:53.437528  7335 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:53.426626 13221 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.452971  6382 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:53.443336 13397 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.446408 13547 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
W20260709 12:27:53.481065  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.424742 13439 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.488657  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3213) took 1012 ms. Trace:
I20260709 12:27:53.489252  6455 rpcz_store.cc:276] 0709 12:27:52.476535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.943454 (+466919us) service_pool.cc:224] Handling call
0709 12:27:52.957370 (+ 13916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.957580 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:53.044309 (+ 86729us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.488143 (+443834us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.488223 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:53.488300 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.488412 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.488583 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":320429,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"thread_start_us":514898,"threads_started":2}
W20260709 12:27:53.527505  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 236467us
I20260709 12:27:53.531172  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.532234  6456 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19.
I20260709 12:27:53.532284  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.533149  6451 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:53.534457  6450 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:27:53.539008  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.539878  6453 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:53.540750  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.541808  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.544531  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.525373 13547 consensus_queue.cc:1243] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:53.543799  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:53.551576 13603 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:53.552043 13584 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.552778 13583 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:27:53.553179  6198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 145) took 31015 ms (client timeout 15000 ms). Trace:
I20260709 12:27:53.554981  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.556068  6455 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:27:53.556639  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.557104  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.543125 13581 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:27:53.559084 13435 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:27:53.561671 13604 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:27:53.546701 13216 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:53.563093 13216 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.565667  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:53.568109 13602 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:27:53.570040 13386 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:53.570538 13386 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.571390 13386 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.572342 13386 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:53.546626 13433 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:27:53.574055 13210 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.564922  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.578545  7317 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:53.579636  6453 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:53.566447  7331 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:53.557539  6457 raft_consensus.cc:2410] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:27:53.583400  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.588518 13416 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.487521 13474 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.593061  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.593868  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:53.597244  7317 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:27:53.592396  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:53.581453 13439 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.612035  7314 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:53.614758  7314 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:53.609218  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:53.608747  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.580840 13397 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:53.554303 13584 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:53.601065  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.564002 13216 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.617039  6450 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:53.598222 13416 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:53.598825 12974 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:27:53.612666 13210 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:53.631502 13342 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:53.638878  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95220.6us
I20260709 12:27:53.640357  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:53.641074 13090 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:53.640892 13521 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.641336 12974 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:53.629887  6378 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:53.644754  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:53.645280  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 240065us
I20260709 12:27:53.601244  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:53.570405 13164 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:53.653910  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52741.4us
W20260709 12:27:53.633994 13601 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:27:53.656281  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:53.648042  7314 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.657023  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:53.658258  7314 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.639039 13210 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:53.626603 13534 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:53.664463  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:53.630803 13258 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.642365 13452 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:27:53.644469 13487 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:53.671219  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:27:53.669409 13460 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.792s	user 0.004s	sys 0.008s
I20260709 12:27:53.689230  6383 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:27:53.659354  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:53.649591 12974 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:53.691833  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 172383us
I20260709 12:27:53.676664 13583 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:27:53.678581 13164 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.670471 13391 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:53.712198 13391 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.713025 13391 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:53.713168 13504 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:53.713896 13504 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:53.553903  6198 rpcz_store.cc:269] 0709 12:27:22.538059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.541450 (+  3391us) service_pool.cc:224] Handling call
0709 12:27:53.553014 (+31011564us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:45.987104 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:45.988084 (+   980us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:45.988159 (+    75us) write_op.cc:435] Acquired schema lock
0709 12:27:45.988204 (+    45us) tablet.cc:662] Decoding operations
0709 12:27:46.055351 (+ 67147us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:46.068275 (+ 12924us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:46.068316 (+    41us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:27:46.088516 (+ 20200us) tablet.cc:701] Row locks acquired
0709 12:27:46.088585 (+    69us) write_op.cc:260] PREPARE: finished
0709 12:27:46.143831 (+ 55246us) write_op.cc:270] Start()
0709 12:27:46.175340 (+ 31509us) write_op.cc:276] Timestamp: P: 1783600066136243 usec, L: 0
0709 12:27:46.179423 (+  4083us) op_driver.cc:348] REPLICATION: starting
0709 12:27:46.223466 (+ 44043us) log.cc:844] Serialized 3894 byte log entry
0709 12:27:46.620226 (+396760us) op_driver.cc:464] REPLICATION: finished
0709 12:27:47.172403 (+552177us) write_op.cc:301] APPLY: starting
0709 12:27:47.184746 (+ 12343us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:47.207101 (+ 22355us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:47.207221 (+   120us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:47.788076 (+580855us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:47.835580 (+ 47504us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:27:47.835650 (+    70us) write_op.cc:312] APPLY: finished
0709 12:27:47.836071 (+   421us) log.cc:844] Serialized 97 byte log entry
0709 12:27:48.330001 (+493930us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:48.340197 (+ 10196us) write_op.cc:454] Released schema lock
0709 12:27:48.409055 (+ 68858us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:48.439456 (+ 30401us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:48.439494 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":58893,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":518839,"apply.run_cpu_time_us":199784,"apply.run_wall_time_us":1253636,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":81574,"prepare.run_cpu_time_us":42139,"prepare.run_wall_time_us":393349,"raft.queue_time_us":98496,"raft.run_cpu_time_us":101132,"raft.run_wall_time_us":929437,"replication_time_us":402569,"thread_start_us":1461515,"threads_started":5,"wal-append.queue_time_us":756580,"wal-append.run_cpu_time_us":11844,"wal-append.run_wall_time_us":2132710}]]}
I20260709 12:27:53.684365 13515 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.664153 13216 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.683979 13521 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.643247 13397 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.717365  7314 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:27:53.717890  7314 pending_rounds.cc:85] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 12
I20260709 12:27:53.646814 13090 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:53.631948 13532 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.662933 13471 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:53.705945  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.712255 13571 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.728681  6351 leader_election.cc:400] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:27:53.728999  6351 leader_election.cc:403] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:53.732805  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:53.734143  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:53.716126 13504 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.688866 13577 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.656374 13440 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:53.718163  7314 pending_rounds.cc:107] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:27:53.689379 13523 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:53.691402 13542 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:53.736795  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.735224  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:53.739239  7314 raft_consensus.cc:2889] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.743278  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.699891 13433 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:27:53.699621 13565 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:53.749130 13433 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:53.749539 13433 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:27:53.724076 13461 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.678s	user 0.006s	sys 0.006s
I20260709 12:27:53.775627  6305 raft_consensus.cc:2368] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:27:53.759977 12974 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.760620 13515 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:53.724373 13584 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:53.763056  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:53.765954 13394 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.766549 13474 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.723719 13090 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:53.766620 13210 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:53.767448 13612 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:53.744536 13258 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:53.752030 13416 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:53.770973  6272 connection.cc:256] RPC call timeout handler was delayed by -1.3705455890003577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 12:27:53.773628 13563 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.791412  6379 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:53.744372 13577 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.745007 13523 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.783229 13440 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.791141 13571 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:53.747273 13471 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.747262 13542 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:53.776257 13565 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:53.793094  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:27:53.798420  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45274.1us
W20260709 12:27:53.875821  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82834.6us
I20260709 12:27:53.785802 13461 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:53.826573  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 151644us
I20260709 12:27:53.876961  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:53.881647  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:53.881852  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.793872 13560 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.818457 13416 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:53.883693  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:53.887910 13471 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.890563 13515 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:53.890987 13604 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:27:53.891343  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.891999 13416 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:53.892839 13461 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.894781 13478 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:53.895190 13478 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.895555 13478 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:53.897004 13521 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:53.891014 13515 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.900211 13515 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.902166  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3122) took 1069 ms. Trace:
I20260709 12:27:53.903359  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:53.902571  6309 rpcz_store.cc:276] 0709 12:27:52.832245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.868465 (+ 36220us) service_pool.cc:224] Handling call
0709 12:27:52.868769 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.058222 (+189453us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:53.058529 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 12:27:53.327251 (+268722us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.331553 (+  4302us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:53.332211 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.633469 (+301258us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:27:53.901934 (+268465us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54000942d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:53.901979 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:53.902111 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":2076,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":183,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":6224,"spinlock_wait_cycles":458565248,"thread_start_us":268009,"threads_started":1,"wal-append.queue_time_us":409366}
I20260709 12:27:53.903921  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1049) took 1329 ms. Trace:
I20260709 12:27:53.903366 13525 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.906769 13560 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.827581 13523 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:53.791772 13435 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.077s
I20260709 12:27:53.918744 13617 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:53.823662 13577 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.804235 13397 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.821074 13258 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:53.917841 13515 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.827770 12974 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:53.904620  6383 rpcz_store.cc:276] 0709 12:27:52.574378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.574628 (+   250us) service_pool.cc:224] Handling call
0709 12:27:53.903638 (+1329010us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400195ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:53.903820 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2560532224}
I20260709 12:27:53.803932 13392 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:27:53.795563 13584 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.944417  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:53.944022  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:53.948628  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.913731 13440 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:53.959620  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:53.961841  6304 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:27:53.975867  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 326090us
I20260709 12:27:53.920122 13532 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:53.919406 13617 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:53.921949 13521 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.939105 13258 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:53.981276  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.044217  6383 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:53.939421 13573 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.829380 13563 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:53.937440 13619 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:53.958146 13542 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:53.978679 13616 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.030325 13382 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.036195 13515 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:54.039728 13584 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:54.048731 13597 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:54.047804  6351 leader_election.cc:400] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:27:53.983258 13474 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.048585  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:54.066555 13521 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.018265  6304 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 14.
I20260709 12:27:54.009040 13617 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:53.918984 13565 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.034085 13391 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:54.088434  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 213986us
W20260709 12:27:54.046008  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 68516.6us
I20260709 12:27:54.080701  6351 leader_election.cc:403] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:54.064473 13619 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:54.093451  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45822.5us
I20260709 12:27:54.093904 13471 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:54.094736  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:54.093786  6456 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:54.047691 13090 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.107268  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:54.112758  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:54.113528  6452 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:54.107909  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.114975  6381 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:54.108381  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:54.116791  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:54.109133  6377 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:27:54.109714  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:54.103291  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:54.118719  6304 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:54.079429 13626 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.122632  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:54.127203  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:54.129101  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:54.129258  6455 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:27:54.129266  6452 raft_consensus.cc:2368] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:27:54.130008  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:54.132022 12840 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.132314  6304 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:54.062592 13573 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.131042  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:54.100008  6383 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:27:54.142254  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:54.142614  6450 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 12:27:54.133673  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.144045  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:54.145548  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.145753  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:54.145993  7335 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:54.147756  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.148468  6381 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:54.141170  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:54.090077 13560 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:54.149943  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.153980  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.157537  6376 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:54.152386  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.074412 13382 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.097600 13521 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:54.101215 13619 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.162101  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.164983  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.134248 13470 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:27:54.171396  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.133098 13533 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:54.140193 13625 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.156433 13221 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:27:54.086917 13474 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.168148  6381 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:54.170818  6385 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:54.133005 13515 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:54.190654 13515 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:54.191119 13515 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:54.192004  6452 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:54.193158 13433 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.194178  7317 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:54.195583  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:54.074468 13616 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.170115  6202 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 12, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 12 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.203265  6202 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.204757 13616 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:54.205538 13616 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:54.205001  6202 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 18, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 18 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.205901 13616 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.206522  6202 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 17, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 17 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:54.180114  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.209121 13573 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:54.180968  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.213510  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.181077  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:54.214545  6458 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:54.155217 13392 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:54.186730  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.187747 13391 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:54.154989 13584 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:54.184306 13618 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 12:27:54.208026  6202 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 17, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 17 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:54.178918  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.208052  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.254225  6385 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:54.254633  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.206270 13616 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.159889 13342 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:54.166138 13394 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.227591  7317 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:54.235255  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.226356  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:54.151466 13547 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:27:54.254925  6202 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 21, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 21 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.278556  6202 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 20, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.262588  6384 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:54.266355  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:54.132617 12840 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.178135 13630 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.275226  6381 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:54.147272 13626 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.215096 13470 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:54.264647 13607 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:27:54.280481  6202 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.284750  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.252158 13622 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:54.287375  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:54.276401  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50148.2us
I20260709 12:27:54.288384  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:54.253901 13584 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.290966  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:54.295953  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 352013us
I20260709 12:27:54.297037  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.262460 13392 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:54.300652  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:54.273000 13342 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.302166  6202 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 18, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 18 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.277659 13504 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:54.303946  6381 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:54.315634  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:54.252262 13613 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:27:54.283679 13221 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:54.279142 13629 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:54.315940  6384 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:54.311980  6304 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:27:54.320926  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:54.340785  6202 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.344660  6202 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.346083  6385 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:54.346016  6202 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 22, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 22 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.353430  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:54.353519  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.354398  6385 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:54.321344  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:54.339666 13461 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:54.321254  6348 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:54.356220  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:54.303699 13622 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:54.357497 13563 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:54.374230 13622 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:54.374554  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83667.2us
I20260709 12:27:54.375404  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:54.376818  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:54.312943 13584 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:54.378742 13525 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:54.379302 13521 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:54.379859 13521 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.380246 13521 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:54.380517  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.316161 13481 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:54.382568  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:54.383590 13525 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:54.362820 13461 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.384184 13461 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:54.293524 13391 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:27:54.360201  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:54.284217 13394 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.324590 13504 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:54.359813  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:54.389290  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:54.434608  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 74843.8us
I20260709 12:27:54.361779  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:54.397771  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76659.7us
I20260709 12:27:54.277410 13478 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.353734  6202 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 6 to 18, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 18 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.362146 13397 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:27:54.400138  6450 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:27:54.447232  6381 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:54.325551 13342 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.316330 13632 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:54.343014 13629 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:54.415370  6385 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:54.361999 13627 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:54.398147 13481 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:54.425048 13210 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:27:54.403468  6376 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:54.438905  6348 leader_election.cc:400] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:27:54.500006  6382 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
W20260709 12:27:54.500499  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61934.5us
I20260709 12:27:54.436129  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:54.503237  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67250.4us
I20260709 12:27:54.504241  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:54.441792  6202 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:54.505970  6202 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 20, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:54.508081  6202 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 16, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 16 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.324400 13221 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:54.428731 13090 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:54.430074 13164 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.479877  6380 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
W20260709 12:27:54.482370  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68435.9us
I20260709 12:27:54.482631  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:54.521030  6450 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17.
W20260709 12:27:54.449654 13633 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:27:54.483616  6456 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:27:54.441807 13639 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:54.443738 13504 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:54.486956  6459 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:27:54.428691 13525 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:54.427335 13258 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.468791 13433 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:54.495692  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:54.216837 13625 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.419425 13470 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.431769 13392 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.511107  6381 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:54.493376 13624 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:27:54.512802  6202 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:27:54.541061 13525 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.295874 13630 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.515249  6272 leader_election.cc:400] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19.
I20260709 12:27:54.684368  6272 leader_election.cc:403] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:27:54.685532  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 170482us
I20260709 12:27:54.525094 13164 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:54.526780 13645 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:27:54.498530 13210 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:54.519140 13221 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:54.518723 13640 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:54.518908 13627 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:54.580583  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 88280.3us
W20260709 12:27:54.627008 13636 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
W20260709 12:27:54.626127 13642 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:27:54.531466 13639 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:54.678639 13474 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:54.581180 13631 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:27:54.644026  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:54.628057 12840 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:54.542480 13619 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.657821 13196 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:54.659801 13342 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.522238 13481 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:54.663226 13471 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.568s	user 0.007s	sys 0.003s
W20260709 12:27:54.666916  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 129458us
I20260709 12:27:54.679142  6202 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 13 to 18, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 18 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:54.688779 13638 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19.
I20260709 12:27:54.703514 13639 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:54.682129 12974 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.734s	user 0.004s	sys 0.007s
I20260709 12:27:54.701872  7285 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17.
W20260709 12:27:54.702654  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 200677us
I20260709 12:27:54.751780  7285 leader_election.cc:403] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:27:54.714771  6376 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:54.719187 13196 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.724897  6202 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.756892  6202 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 16, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 16 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.494755 13629 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:54.768488 13617 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.667s	user 0.003s	sys 0.012s
I20260709 12:27:54.686839  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:54.696012 13394 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:54.754864  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59014.9us
W20260709 12:27:54.770370  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83898.9us
I20260709 12:27:54.771807  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:54.772884 13638 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.772977 12974 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:27:54.773727  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:54.774121 13581 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:54.777235  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:54.781929  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:54.758201  6202 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 10 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 15 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.723404 13090 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:54.760468 13435 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17.
I20260709 12:27:54.912438  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:54.718863 13210 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:54.764539 13515 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.573s	user 0.005s	sys 0.004s
I20260709 12:27:54.751338 13650 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:54.797210  6202 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 12 to 18, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 18 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:54.730785 13460 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:54.913254  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 160958us
I20260709 12:27:54.726076 13471 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:27:54.768978 13617 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:54.921583  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.810302  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:54.920485  6202 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 20, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 20 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:55.140784  6202 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:54.924700  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:54.793668 13196 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:54.928324  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:55.012861 13617 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:55.160598  6425 leader_election.cc:336] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.968s (SENT)
I20260709 12:27:55.168963  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:55.102176 13504 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.560s	user 0.002s	sys 0.010s
I20260709 12:27:55.054634  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:54.999236 13471 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:55.114406 13416 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.222s	user 0.011s	sys 0.003s
I20260709 12:27:55.065003  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:54.977926 13577 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.057s	user 0.012s	sys 0.000s
I20260709 12:27:54.925530 13478 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:54.933365 13460 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.996244 13210 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:27:55.010776  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.196547  6459 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:55.107337 13391 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.930761 13650 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.207918 13391 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.209755 13650 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:55.066119 13653 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:27:54.726310 13481 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.212334 13629 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.009086 13342 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.052845 13640 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.922333 13515 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:54.915076 13532 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:55.026131  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
W20260709 12:27:55.023597 13534 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.351s	user 0.011s	sys 0.000s
W20260709 12:27:55.021734 13523 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.111s	user 0.009s	sys 0.003s
I20260709 12:27:55.024091 13625 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.017109 13221 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:54.911751 13090 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.087977  6378 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:54.913990 13627 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:55.024778 13622 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.650s	user 0.005s	sys 0.003s
W20260709 12:27:54.992455  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65841us
I20260709 12:27:55.111123 13619 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:55.119141 13584 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.738s	user 0.007s	sys 0.003s
I20260709 12:27:55.195824 13577 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.100198 13397 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:55.143836  6202 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:55.143496  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:55.136427 13616 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.858s	user 0.011s	sys 0.000s
W20260709 12:27:55.184088  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 373869us
I20260709 12:27:55.185422  7316 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:55.186681  6305 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:55.239507  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:55.198539 13657 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 12:27:55.241811  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.193140 13416 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.244994  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:55.237020  6202 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:55.191380  6455 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:27:55.239470  6459 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:55.152115 13164 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.243677  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:55.245739  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:55.183208 13525 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.247346  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:55.216991 13521 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.836s	user 0.010s	sys 0.002s
I20260709 12:27:55.256179  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:55.255474  6202 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:55.257143  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 88938.9us
I20260709 12:27:55.255172  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.300247  6459 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:55.259423  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.301211  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.300590  6202 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:27:55.301898  6301 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:27:55.192703 13623 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:55.225216 12974 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.298350  7314 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:55.196353 13617 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:55.312219  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.228600 13547 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:55.299084  6450 raft_consensus.cc:606] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:55.319394  6450 raft_consensus.cc:614] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:55.189636 13504 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:55.295794  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50912.9us
I20260709 12:27:55.304646  6455 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:55.218895 13460 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.306063  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.324736  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:55.246912 13571 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.376s	user 0.011s	sys 0.002s
W20260709 12:27:55.247072 13542 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.167s	user 0.010s	sys 0.003s
W20260709 12:27:55.308406  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52319.3us
I20260709 12:27:55.239584 13534 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:55.213016 13481 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.309300  6308 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:55.312860 13627 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:55.313638  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.192957 13471 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.242347 13625 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.252447 13391 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.319933  6459 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:55.210414 13650 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:55.306447  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60853.8us
W20260709 12:27:55.371371 13461 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.987s	user 0.018s	sys 0.001s
I20260709 12:27:55.311854 13573 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:55.310622  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.391266  6457 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:55.247359 13397 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.239389 13622 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:55.367802  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 124198us
I20260709 12:27:55.393638  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:55.375949 13433 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:55.311157 13521 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.403998  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:55.413507 13521 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.319101 13547 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:55.313277 13665 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:55.361557 13542 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:55.423266  7314 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1073) took 1541 ms (client timeout 1776 ms). Trace:
I20260709 12:27:55.426396  6309 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:55.256660 13616 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.343295 13563 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:55.247251 13584 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.318114 13532 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:55.348959  6455 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:27:55.484277  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.243150 13640 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:55.368294 13636 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:55.362939 13571 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:27:55.321105 12974 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.241501 13523 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.397676 13622 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.399260 13382 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.400374 13629 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.507820  6452 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:55.403357 13626 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.370229 13627 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:55.410905 13534 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:55.411535 13416 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:55.492115  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.514892  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.493692  6455 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:55.387256 13625 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.491170  6301 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:55.506064 13628 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.296s	user 0.014s	sys 0.011s
I20260709 12:27:55.524938  6377 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:55.357764 13613 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:27:55.497054  6305 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:55.497663 13681 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:27:55.499568 13433 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:55.505319  6453 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:27:55.505497  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3147) took 1273 ms. Trace:
I20260709 12:27:55.393893 13461 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.533097  6302 rpcz_store.cc:276] 0709 12:27:54.231897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.232135 (+   238us) service_pool.cc:224] Handling call
0709 12:27:54.270206 (+ 38071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.503886 (+1233680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.503918 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:55.503957 (+    39us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:55.503985 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.505282 (+  1297us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400206cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.505324 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.505443 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836807936}
I20260709 12:27:55.415499 13515 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:27:55.424571 13638 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.651s	user 0.007s	sys 0.003s
I20260709 12:27:55.525295  6455 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.542176  6455 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:55.427924  7314 rpcz_store.cc:269] 0709 12:27:53.881472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.882560 (+  1088us) service_pool.cc:224] Handling call
0709 12:27:55.423006 (+1540446us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b54007a0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.423181 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3234105600}
I20260709 12:27:55.400895 13560 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:27:55.425531 13632 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.926s	user 0.010s	sys 0.002s
I20260709 12:27:55.425350 13577 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.418098 13623 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:55.415604 13665 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:55.560076 13665 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:55.560989  6272 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:53970 dropped due to backpressure: service queue is full with 50 items
I20260709 12:27:55.423137 13471 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.545939 13560 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:55.563352 12974 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.474445 12840 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:55.563634 13560 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.564898 13560 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.564648 12840 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.566036 13461 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.566661  6379 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:27:55.567200  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.567891  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:55.532454 13626 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:55.570149 13626 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.572292 13626 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.545722 13523 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.528146 13571 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.573952 13571 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.513561  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:55.417724 13481 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:55.579276 13481 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.566795 13632 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:55.581267 13481 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.581583  6451 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:55.536257 13629 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:55.491343 13673 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:55.582319 13629 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.582873 13673 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.583732  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.546326 13638 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:55.585204 13673 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.584384  6451 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:55.586957  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.587796  6451 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:55.589175  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3237) took 1476 ms. Trace:
I20260709 12:27:55.589617  6456 rpcz_store.cc:276] 0709 12:27:54.112280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.211791 (+ 99511us) service_pool.cc:224] Handling call
0709 12:27:54.212220 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.523941 (+311721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.588886 (+1064945us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b5400345654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.589111 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":623575808}
I20260709 12:27:55.591178 13571 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:55.593279  7317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1072) took 1712 ms (client timeout 1834 ms). Trace:
W20260709 12:27:55.593746  7317 rpcz_store.cc:269] 0709 12:27:53.881109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.881607 (+   498us) service_pool.cc:224] Handling call
0709 12:27:55.593108 (+1711501us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54006595d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.593216 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716843776}
I20260709 12:27:55.594130  6451 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:27:55.595124 13504 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.393357 13573 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.504673 13397 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.593312 12840 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:55.521440 13621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.334s	user 0.008s	sys 0.024s
I20260709 12:27:55.595803 13573 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:55.596539 13621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.420s	user 0.008s	sys 0.025s
I20260709 12:27:55.541138  6385 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:55.597962  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.598255  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.526124  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.599083  6385 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 16. Current term is 17.
I20260709 12:27:55.599326  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:55.500730 13640 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.521787 13627 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:55.561362  6272 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:53970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2877)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1096)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2880)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1107)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1100)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2871)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2882)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2878)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2886)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1106)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2883)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2885)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2876)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2884)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2873)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1109)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2881)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1098)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1099)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1101)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1087)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2859)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2868)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2861)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2865)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2870)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2872)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2879)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1088)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1090)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1091)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1102)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1103)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1105)
I20260709 12:27:55.604776 13573 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.526093  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:55.535859 13639 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 12:27:55.606875 13639 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.607363  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.608006  6377 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.609115 13639 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.554818 13542 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.609119 13258 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.599910  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.514355  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.611131  6383 raft_consensus.cc:2376] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17. Re-sending same reply.
I20260709 12:27:55.611461  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.494392  6309 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:55.618667 13573 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.570298 13433 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.523909 13563 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:55.584388 13629 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.390357 13650 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.624780 13563 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.626799 13650 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:55.626942  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3218) took 2503 ms (client timeout 1935 ms). Trace:
W20260709 12:27:55.627462  6458 rpcz_store.cc:269] 0709 12:27:53.123937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.124876 (+   939us) service_pool.cc:224] Handling call
0709 12:27:55.626737 (+2501861us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400344c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.626853 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2503195904}
I20260709 12:27:55.629285  6458 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:55.554363 13625 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.634886  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1067) took 1354 ms. Trace:
I20260709 12:27:55.635488  6376 rpcz_store.cc:276] 0709 12:27:54.280514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.499194 (+218680us) service_pool.cc:224] Handling call
0709 12:27:54.499641 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.752485 (+252844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.634411 (+881926us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.634791 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":469827456}
I20260709 12:27:55.637271  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.639240  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.640305  6309 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:55.642755  6455 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:55.645409  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1064) took 1372 ms. Trace:
I20260709 12:27:55.645826  6381 rpcz_store.cc:276] 0709 12:27:54.272621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.447965 (+175344us) service_pool.cc:224] Handling call
0709 12:27:54.448208 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.678154 (+229946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.645156 (+967002us) spinlock_profiling.cc:243] Waited 62.7 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.645344 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144126208}
I20260709 12:27:55.647218  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.647950  6381 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:55.605114  6272 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:53970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2877)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1096)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2880)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1107)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1100)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2871)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2882)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2878)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2886)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1106)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2883)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2885)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2876)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2884)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2873)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2881)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1098)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1099)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1101)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1087)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2859)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2868)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2861)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2865)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2870)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2872)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2879)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1088)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1090)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1091)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1102)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1103)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1105)
I20260709 12:27:55.657603  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.638150  6385 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:55.612243  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.504662 13196 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.674474 13258 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:27:55.675053  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3142) took 1796 ms (client timeout 1768 ms). Trace:
I20260709 12:27:55.675624 13258 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:55.675949 13221 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.613324  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 2998) took 1503 ms. Trace:
I20260709 12:27:55.676410 13196 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.676823  6382 rpcz_store.cc:276] 0709 12:27:54.110007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.110135 (+   128us) service_pool.cc:224] Handling call
0709 12:27:54.110398 (+   263us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.500460 (+390062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.613094 (+1112634us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400037354. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.613265 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895150848}
I20260709 12:27:55.677594  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.677935  6454 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:55.678113  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.678826  6382 raft_consensus.cc:2376] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:27:55.679355  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.616894  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3301) took 1519 ms. Trace:
I20260709 12:27:55.679978  6378 rpcz_store.cc:276] 0709 12:27:54.097552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.098384 (+   832us) service_pool.cc:224] Handling call
0709 12:27:55.616745 (+1518361us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b54003d40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.616844 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1917015424}
I20260709 12:27:55.680543  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.681263  6307 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:55.682035  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:55.617502  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 104033us
I20260709 12:27:55.683847 13452 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:55.684419  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:55.660386 13650 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:55.686887  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.687045 13650 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.687639  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3231) took 1582 ms. Trace:
I20260709 12:27:55.688066  6452 rpcz_store.cc:276] 0709 12:27:54.105332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.193953 (+ 88621us) service_pool.cc:224] Handling call
0709 12:27:54.194223 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.510048 (+1315825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.687389 (+177341us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b5400161cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.687577 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3020451712}
I20260709 12:27:55.689097 13650 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.689190  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.690101  6383 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:55.587028  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.691071  6310 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:55.691893  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:55.675472  6308 rpcz_store.cc:269] 0709 12:27:53.878350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.878610 (+   260us) service_pool.cc:224] Handling call
0709 12:27:55.674866 (+1796256us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001b21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.674975 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3130014976}
I20260709 12:27:55.687685  6451 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:55.693337  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:55.612413  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3149) took 1324 ms (client timeout 1586 ms). Trace:
I20260709 12:27:55.665284  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.613945 13616 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.667768 13625 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.665838  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.673568 13629 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:55.694049  6304 rpcz_store.cc:269] 0709 12:27:54.287651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.288134 (+   483us) service_pool.cc:224] Handling call
0709 12:27:55.612225 (+1324091us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b54002d19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.612355 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12949376}
I20260709 12:27:55.701059  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.694092  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.701849  6455 raft_consensus.cc:2368] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19.
I20260709 12:27:55.701987 13630 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.702590  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.679685  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.658519  6305 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:55.676030 13258 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:55.702718 13625 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.678390  6456 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:55.681114  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.618001 13460 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:55.692469  6310 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:55.695474  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.704214  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.719746  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.720607  6301 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:27:55.721720  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.704475  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.612475 13584 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.704879  7285 leader_election.cc:400] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19.
I20260709 12:27:55.722849  7285 leader_election.cc:403] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:27:55.611217 13547 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.702732  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.723388  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.723891  6455 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:55.724915  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.704236  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.713737  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:55.725961  6305 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:55.694144  6308 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:27:55.697302  6380 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3006) took 1532 ms (client timeout 1862 ms). Trace:
I20260709 12:27:55.701252  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.709353  6454 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:55.663306  6377 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
W20260709 12:27:55.726914  6380 rpcz_store.cc:269] 0709 12:27:54.164927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.165094 (+   167us) service_pool.cc:224] Handling call
0709 12:27:55.697114 (+1532020us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54003d36d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.697236 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465020160}
I20260709 12:27:55.670104  6272 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:53970 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2871)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2882)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2878)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2890)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2858)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2886)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2860)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1106)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2888)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2883)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2864)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2885)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2866)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2876)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2884)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2892)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2873)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2881)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2862)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2887)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2863)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2867)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1098)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2889)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2869)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1099)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1093)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1101)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1087)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2893)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1089)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2859)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1086)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1095)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2868)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2891)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2861)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2865)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2870)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2872)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2879)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1088)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1090)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1091)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1103)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1105)
I20260709 12:27:55.677376  7335 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:55.696722  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:55.729120  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:55.730101  7317 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:55.419436 13090 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.739805  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:55.740285  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.740823  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.741556  6455 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20.
I20260709 12:27:55.712133  6382 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:55.748409  6382 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:55.715828  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:55.733952  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:55.735742  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.750073  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.750382  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.736347  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.737185  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.737746  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.739223  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.748593  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.752831  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:27:55.753407  6305 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:27:55.753875  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.755232  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.755918  6377 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:27:55.751942  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.757872  6376 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20.
I20260709 12:27:55.758852  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.759071  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.759230  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:55.759588  6376 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:55.760012  6455 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:55.756853  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.760916  6377 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:55.751363  6300 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:27:55.754172  6456 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:55.763137  6309 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:27:55.713433 13386 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:55.757835  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.778739  6308 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:55.779490  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.768476  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.780627  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.781180  6302 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:55.781455  6305 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:55.782375  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.782721  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.769778  6304 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
W20260709 12:27:55.770319  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.783488  6302 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:55.785944  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:55.788414  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84177.5us
I20260709 12:27:55.789083  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.789405  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:55.711052 13542 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:27:55.717206 13677 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.792167 13677 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:55.794214 13677 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:55.775107  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.775542  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.800237  6454 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:55.777168  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:55.800796  7314 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:55.779702  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.802278  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.802544  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:55.803022  6309 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:55.804730  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.805866  6309 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:55.807363  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:55.808821  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.717094 13617 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:27:55.717403 13460 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.757583 13523 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:55.743643 13630 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:55.738082 13616 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:55.825222 13460 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.116495 13654 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:56.117494  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.816490  6310 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:56.120200  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.121395  6310 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:55.766456  6272 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20.
I20260709 12:27:56.123049  6272 leader_election.cc:403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:27:55.784154  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:55.813498  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.824440  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.125509  6305 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.126600  6302 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:27:56.126981 13665 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.127833 13665 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.825474  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:55.898377  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.132990  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.134001  6310 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:56.134218  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.135236  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.135761  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.136108  6310 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:55.827286  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.053680  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.061666  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:55.905963  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:55.910014  6351 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d53ab19d424b4a85af2153ef24520678. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:27:55.900974  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 361154us
I20260709 12:27:56.157246  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:55.769433  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:55.811304  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
W20260709 12:27:56.080538  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1061) took 1844 ms (client timeout 1667 ms). Trace:
I20260709 12:27:55.826208  6381 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
W20260709 12:27:56.160481  6379 rpcz_store.cc:269] 0709 12:27:54.236013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.255671 (+ 19658us) service_pool.cc:224] Handling call
0709 12:27:56.079861 (+1824190us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b540010a4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:56.080196 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2779703040}
I20260709 12:27:55.809908  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:56.161793  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:55.827831  6385 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:55.906912  6384 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:55.976941  6424 leader_election.cc:336] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.533s (SENT)
W20260709 12:27:56.166352  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 189388us
I20260709 12:27:55.899230  6455 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:55.768435  6456 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:55.972633  6457 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:56.172961  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:55.966033  6450 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:27:55.969058  6452 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:55.972106  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3227) took 1867 ms. Trace:
I20260709 12:27:56.197127  6453 rpcz_store.cc:276] 0709 12:27:54.104229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.106585 (+  2356us) service_pool.cc:224] Handling call
0709 12:27:54.106858 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:55.537841 (+1430983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.971799 (+433958us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000c12d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:55.972033 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3215580928}
I20260709 12:27:55.920254  6454 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.7-15.7]
W20260709 12:27:55.922899  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.029s	sys 0.013s
W20260709 12:27:56.200757  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.855s	user 0.029s	sys 0.013s
W20260709 12:27:56.201722  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 877016us
W20260709 12:27:56.100458  7288 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:53970 dropped due to backpressure: service queue is full with 50 items
I20260709 12:27:56.203329  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:56.205835  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 205830us
I20260709 12:27:55.995589  7314 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:55.796559  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 987) took 1023 ms. Trace:
I20260709 12:27:55.822827 13210 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:55.738741 13221 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:56.216691  7335 rpcz_store.cc:276] 0709 12:27:54.773481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.776931 (+  3450us) service_pool.cc:224] Handling call
0709 12:27:55.796369 (+1019438us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b54003bc4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:55.796502 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2068198784}
I20260709 12:27:56.217692 13210 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:55.976889 13382 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:56.224460 13382 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:56.224924 13655 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.225792 13655 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:55.907986  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.235869  7317 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:55.991422  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:55.988173  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.238471 13210 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.240294  7331 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:55.768841 12840 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:56.242718 12840 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:56.243122  6309 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:56.244984 13221 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19.
I20260709 12:27:55.737835 13258 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.248950 13258 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:55.987593 13515 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:55.899227 13523 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:56.250917  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.250955 13515 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:56.254079 13523 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.767498 13632 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:55.983515 13617 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:56.262709  6459 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:56.263116 13617 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.009380 13625 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.267329 13625 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:56.268754  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 96085.9us
I20260709 12:27:55.903635  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.269990  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:55.910276 13630 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:56.274420  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 370760us
I20260709 12:27:56.275821  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:56.274212  7320 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
W20260709 12:27:56.010406 13675 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.486s	user 0.005s	sys 0.006s
I20260709 12:27:55.973121 13672 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:56.285841 13672 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.286851 13342 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:56.289160 13672 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.007455 13488 tablet_copy_client.cc:538] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:56.128453  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.289989  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:56.291025  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71208.5us
I20260709 12:27:56.293107 13683 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:56.293737 13683 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:56.142000  6304 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:56.117646 13654 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.135512 13665 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:55.802394  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:56.173149  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
W20260709 12:27:56.146667  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 236603us
I20260709 12:27:56.174137 13090 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:56.186566  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3015) took 1973 ms. Trace:
I20260709 12:27:55.983263 13386 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.304730  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.305526 13386 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:56.307296  6455 raft_consensus.cc:606] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:56.307989  6455 raft_consensus.cc:614] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:56.311079  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:56.012940 13547 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:56.253618 13515 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.301221  6384 rpcz_store.cc:276] 0709 12:27:54.212924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.436626 (+223702us) service_pool.cc:224] Handling call
0709 12:27:54.437035 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.164521 (+1727486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.180781 (+ 16260us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:56.186338 (+  5557us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3358258944}
I20260709 12:27:56.226254 13382 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.313599 13382 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:56.313800 13515 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:56.268605  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.261281 13650 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:55.713402 13629 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:56.319311  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 516768us
I20260709 12:27:55.776592 13627 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.322947  6300 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:56.275583 13630 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.324743 13665 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:56.325167 13665 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:56.325722 13665 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:56.275741  6459 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:56.107774 13616 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.328186 13655 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:55.936374 13639 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.330480 13616 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:56.284394 13677 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.331310 13639 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.284301 13625 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:56.334692 13625 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.322201 13629 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.335155 13625 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.272951 13617 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:56.334774 13632 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.304611  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:56.311317 13654 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.325817  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:56.344772  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.300870 13090 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.334226 13639 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.334827  6451 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
W20260709 12:27:56.353444  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.350203  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:53970 dropped due to backpressure: service queue is full with 50 items
W20260709 12:27:56.357321  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55375.4us
I20260709 12:27:56.324009 13627 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.335676 13629 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:56.339545 13632 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:27:56.360702  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.362790 13672 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:56.353223 13090 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:56.364228  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.370211  6459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:27:56.371371  6459 raft_consensus.cc:606] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:27:56.372042  6459 raft_consensus.cc:614] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:56.375998  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.377593  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.378909  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:56.380450  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:56.356740  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 590449us
I20260709 12:27:56.381562  6300 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:56.390913  6305 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:56.363274 13672 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.401922  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:27:56.396171 13672 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:56.409808  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:56.417806  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148015us
W20260709 12:27:56.417339  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 120202us
I20260709 12:27:56.426048  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.360000 13683 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.432986  6303 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:56.438791 13627 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:56.445894  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.448601  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:56.449764 13655 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:56.459529  6310 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
W20260709 12:27:56.454888  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132640us
W20260709 12:27:56.470166  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82883us
I20260709 12:27:56.474428  6274 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20.
I20260709 12:27:56.450212  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.450277 13655 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.478957  6300 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:56.472807  6309 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:56.478613  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:27:56.485621  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1089) took 1052 ms. Trace:
I20260709 12:27:56.486937  6305 rpcz_store.cc:276] 0709 12:27:55.433164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.425548 (+992384us) service_pool.cc:224] Handling call
0709 12:27:56.485538 (+ 59990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.482815  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:56.495374  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:56.499694  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.501799  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:56.501619 13650 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.486080 13655 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:56.488886 13639 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:56.511652  6310 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:56.477720 13703 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.516158  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.517127  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2859) took 1100 ms. Trace:
I20260709 12:27:56.524616  6310 rpcz_store.cc:276] 0709 12:27:55.416686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.472910 (+1056224us) service_pool.cc:224] Handling call
0709 12:27:56.517037 (+ 44127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.530283  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1086) took 1097 ms. Trace:
I20260709 12:27:56.531109  6309 rpcz_store.cc:276] 0709 12:27:55.432239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.496290 (+1064051us) service_pool.cc:224] Handling call
0709 12:27:56.529993 (+ 33703us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.510658 13386 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.547668  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.548875  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.551523  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.554649  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.559540  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2868) took 1130 ms. Trace:
I20260709 12:27:56.560155  6310 rpcz_store.cc:276] 0709 12:27:55.425757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.526562 (+1100805us) service_pool.cc:224] Handling call
0709 12:27:56.556461 (+ 29899us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:56.561167  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 508154us
I20260709 12:27:56.546305 13617 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:56.564419  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 114436us
I20260709 12:27:56.565605  6300 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:56.568619  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1095) took 1094 ms. Trace:
I20260709 12:27:56.568882  6300 rpcz_store.cc:276] 0709 12:27:55.474714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.509100 (+1034386us) service_pool.cc:224] Handling call
0709 12:27:56.568541 (+ 59441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.562615 13392 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:56.573931  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.570405  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.576301  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:56.579134 13705 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:27:56.580488 13705 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:56.580909 13705 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:56.540099 13703 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.583196  6310 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:27:56.550877 13650 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:27:56.587510  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:56.587401  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2865) took 1165 ms. Trace:
I20260709 12:27:56.589058  6310 rpcz_store.cc:276] 0709 12:27:55.418217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.565888 (+1147671us) service_pool.cc:224] Handling call
0709 12:27:56.583018 (+ 17130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.583780 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.583918 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.589977  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.573534 13392 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.591316  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.595615  7314 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:56.572872 13665 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.599593 13627 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:56.600386 13654 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:56.600953 13639 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.598453  6305 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:56.602317  6308 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:56.582226 13705 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.615164  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:56.615931  6300 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:56.618119  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.618880  7331 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:56.585680 13617 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.624270 13386 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.594048 13392 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:56.595220 13630 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:56.631494  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1093) took 1197 ms. Trace:
I20260709 12:27:56.641053  7317 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:56.642010  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2872) took 1208 ms. Trace:
I20260709 12:27:56.630856  6309 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 13. Current term is 14. Ops: []
I20260709 12:27:56.633891  6304 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
W20260709 12:27:56.635483  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 58873.6us
W20260709 12:27:56.637223  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.637121  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2870) took 1205 ms. Trace:
I20260709 12:27:56.643894  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2867) took 1218 ms. Trace:
I20260709 12:27:56.640538  6308 rpcz_store.cc:276] 0709 12:27:55.434324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.135423 (+701099us) service_pool.cc:224] Handling call
0709 12:27:56.631414 (+495991us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.644188  6305 rpcz_store.cc:276] 0709 12:27:55.426318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.560882 (+1134564us) service_pool.cc:224] Handling call
0709 12:27:56.586435 (+ 25553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.631182 (+ 44747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.631405 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.646306  6308 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:56.647251  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1088) took 1214 ms. Trace:
I20260709 12:27:56.647571  6308 rpcz_store.cc:276] 0709 12:27:55.432860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.645261 (+1212401us) service_pool.cc:224] Handling call
0709 12:27:56.645761 (+   500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.646970 (+  1209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.647164 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.649371  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1091) took 1215 ms. Trace:
I20260709 12:27:56.649734  6308 rpcz_store.cc:276] 0709 12:27:55.433717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.648345 (+1214628us) service_pool.cc:224] Handling call
0709 12:27:56.648790 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.649031 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.649059 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.649119 (+    60us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:56.649154 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.649207 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.649303 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.650579 13629 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:27:56.642637  6300 rpcz_store.cc:276] 0709 12:27:55.427615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.570473 (+1142858us) service_pool.cc:224] Handling call
0709 12:27:56.597042 (+ 26569us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.635457 (+ 38415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.635698 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.643723  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2861) took 1226 ms. Trace:
I20260709 12:27:56.641115  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:56.655118  6309 rpcz_store.cc:276] 0709 12:27:55.417190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.536105 (+1118915us) service_pool.cc:224] Handling call
0709 12:27:56.584021 (+ 47916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.643462 (+ 59441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.643665 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.656597  7320 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:27:56.644421  6304 rpcz_store.cc:276] 0709 12:27:55.425454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.060641 (+635187us) service_pool.cc:224] Handling call
0709 12:27:56.643835 (+583194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.632688 13639 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:56.678967  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:56.683576 13460 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.568s	user 0.011s	sys 0.008s
I20260709 12:27:56.684055 13460 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.691413 13433 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.692426 13433 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.695765 13703 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:56.696858 13221 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:56.697577 13221 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.660640 13627 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:56.663973 13705 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:56.664577  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 94302.9us
I20260709 12:27:56.701748  7314 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:27:56.665619 13392 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.702175 13654 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:56.598852 13210 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:56.674654  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2873) took 1209 ms. Trace:
I20260709 12:27:56.677031 13677 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:56.691619  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2879) took 1262 ms. Trace:
I20260709 12:27:56.738973  6310 rpcz_store.cc:276] 0709 12:27:55.429711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.690528 (+1260817us) service_pool.cc:224] Handling call
0709 12:27:56.690965 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.691282 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.691313 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.691355 (+    42us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:56.691384 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.691449 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.691562 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.662616  6459 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:56.668167 13672 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.640517 13630 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:56.720257  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.703670 13460 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.727458 13650 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.755716 13672 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.756089 13650 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.756510 13672 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:56.753334 13630 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.757382 13386 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:27:56.757992 13258 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.508s	user 0.015s	sys 0.002s
I20260709 12:27:56.758496  6308 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:56.728819  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 106068us
I20260709 12:27:56.759123  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1097) took 1283 ms. Trace:
I20260709 12:27:56.759577  6308 rpcz_store.cc:276] 0709 12:27:55.475410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.653691 (+1178281us) service_pool.cc:224] Handling call
0709 12:27:56.758273 (+104582us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.758885 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.759067 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.727530 13629 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:27:56.731757  6301 rpcz_store.cc:276] 0709 12:27:55.427855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.785707 (+357852us) service_pool.cc:224] Handling call
0709 12:27:56.601381 (+815674us) spinlock_profiling.cc:243] Waited 790 ms on lock 0x7b54002d28d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:56.636712 (+ 35331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1817401216}
I20260709 12:27:56.726332  6303 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:27:56.727757 13665 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:56.761401  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3161) took 1067 ms. Trace:
W20260709 12:27:56.734182  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.757828 13386 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:56.763319  6304 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:27:56.731359 12840 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20.
I20260709 12:27:56.732257 13654 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:27:56.766419  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.768258  6308 rpcz_store.cc:276] 0709 12:27:55.693551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.760453 (+1066902us) service_pool.cc:224] Handling call
0709 12:27:56.760795 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.761042 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.761078 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.761154 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:27:56.761189 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.761245 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.761347 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.764288  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2893) took 1094 ms. Trace:
W20260709 12:27:56.770351  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 192224us
W20260709 12:27:56.767395  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 104472us
I20260709 12:27:56.775738  6303 rpcz_store.cc:276] 0709 12:27:55.669711 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.306774 (+637063us) service_pool.cc:224] Handling call
0709 12:27:56.764220 (+457446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.771205  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1105) took 1256 ms. Trace:
I20260709 12:27:56.777952  6304 rpcz_store.cc:276] 0709 12:27:55.514619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.703762 (+1189143us) service_pool.cc:224] Handling call
0709 12:27:56.744467 (+ 40705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.770896 (+ 26429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.771143 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.776785 13433 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.782814 13704 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 13.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:27:56.772095 13650 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.759032 13547 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:27:56.787809  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:56.789186 13210 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.790031  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3166) took 1022 ms. Trace:
I20260709 12:27:56.790457  6307 raft_consensus.cc:2368] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18.
I20260709 12:27:56.783424  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:56.790928  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:56.794771  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:56.796244  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:56.796438  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.728420 13090 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:56.795431  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.799635  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.790534  6303 rpcz_store.cc:276] 0709 12:27:55.768102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.781252 (+1013150us) service_pool.cc:224] Handling call
0709 12:27:56.789279 (+  8027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.789601 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.789636 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.789708 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:56.789748 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.789814 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.789943 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.808655  6310 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:27:56.809572  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3160) took 1116 ms. Trace:
I20260709 12:27:56.810027  6310 rpcz_store.cc:276] 0709 12:27:55.693269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.750797 (+1057528us) service_pool.cc:224] Handling call
0709 12:27:56.779212 (+ 28415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.809273 (+ 30061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.809518 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.777446 13654 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:56.812469  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.777904 13258 raft_consensus.cc:2764] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:27:56.717115 13625 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.818270 13630 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.834343 13630 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.835445  6301 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 15. Current term is 16. Ops: []
I20260709 12:27:56.807999 13650 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:56.852170  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 89487.7us
I20260709 12:27:56.855679  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3162) took 1147 ms. Trace:
I20260709 12:27:56.855198 13632 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.856151  6301 rpcz_store.cc:276] 0709 12:27:55.708512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.780682 (+1072170us) service_pool.cc:224] Handling call
0709 12:27:56.835255 (+ 54573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.855396 (+ 20141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.855603 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.857697  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:56.831714  7288 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.581s (SENT)
I20260709 12:27:56.826905 13523 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:56.840200 13625 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.840795  6309 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:56.842746 13629 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:56.790571 13677 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:27:56.847661 13392 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.850764  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:56.850893  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54588.3us
I20260709 12:27:56.851447 13210 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:56.852466 13650 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:27:56.894307  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:56.895469  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.853964 13639 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.838776 13715 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:56.837661 13386 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.839164  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:56.825995 13515 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.002s
I20260709 12:27:56.854182  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:56.864494  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:56.866581 13523 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.858474  6301 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:56.852178 13617 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.859284 13713 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:56.952500  6302 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:56.862879 13382 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:56.908656  7320 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:56.824826 12840 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:56.910668  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 14.
W20260709 12:27:56.907699  6424 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.621s (SENT)
I20260709 12:27:56.912981 13709 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:56.914315 13090 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:56.914786  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 142263us
I20260709 12:27:56.914990 13488 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:27:56.915548  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:56.919209  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3169) took 1132 ms. Trace:
I20260709 12:27:56.919480 13677 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:56.921622  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:56.854095 13712 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:56.954465 12840 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:56.926034 13386 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:56.865831 13616 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:27:56.904064 13650 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:56.904580 13715 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:56.931053 13632 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:56.931756  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2866) took 1513 ms (client timeout 1968 ms). Trace:
I20260709 12:27:56.932847  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.933890  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1104) took 1422 ms. Trace:
I20260709 12:27:56.969594  6309 rpcz_store.cc:276] 0709 12:27:55.511067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.703776 (+1192709us) service_pool.cc:224] Handling call
0709 12:27:56.722161 (+ 18385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.865544 (+143383us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.933629 (+ 68085us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54002d19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:27:56.933806 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272317696}
W20260709 12:27:56.969108  6307 rpcz_store.cc:269] 0709 12:27:55.425078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.775294 (+350216us) service_pool.cc:224] Handling call
0709 12:27:56.931506 (+1156212us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54002e7f54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:56.931673 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1449841024}
W20260709 12:27:56.970975  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63333.6us
W20260709 12:27:56.971498  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.971537  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:56.934744 13523 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:56.973274 13715 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.866146 13629 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:56.974768 13715 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:56.974810 13629 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:56.937114 13704 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:56.979607 13090 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:27:56.980075 13090 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:56.936272 13221 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:56.947117  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:56.981042 13090 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.981485  6450 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:27:56.983219 12840 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.983767 13639 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.984624 12840 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:56.978663 13704 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:56.987581 13704 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:56.988963 13704 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:56.949798  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:56.990834  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:56.993948 13710 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:56.995344 13677 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:56.996343 13677 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.000492  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1103) took 1490 ms. Trace:
I20260709 12:27:57.001168  6300 rpcz_store.cc:276] 0709 12:27:55.509768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.658546 (+1148778us) service_pool.cc:224] Handling call
0709 12:27:56.679344 (+ 20798us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.772202 (+ 92858us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.772250 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.794262 (+ 22012us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.867752 (+ 73490us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:56.867816 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.867930 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.000076 (+132146us) raft_consensus.cc:1685] finished
0709 12:27:57.000130 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.000390 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104273,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":195,"thread_start_us":158999,"threads_started":2,"wal-append.queue_time_us":163243}
I20260709 12:27:57.005429  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3173) took 1217 ms. Trace:
I20260709 12:27:57.007041 13654 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.921902 13655 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:56.951179  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 153984us
I20260709 12:27:57.011408 13655 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:56.865571 13630 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:56.956763  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1090) took 1523 ms. Trace:
I20260709 12:27:57.012610 13713 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.012696  6305 rpcz_store.cc:276] 0709 12:27:55.433444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.645472 (+1212028us) service_pool.cc:224] Handling call
0709 12:27:56.652494 (+  7022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.659582 (+  7088us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:56.826857 (+167275us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.956141 (+129284us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.956219 (+    78us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:56.956326 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.956457 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.956681 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":199546,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":2206,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":158,"thread_start_us":158198,"threads_started":2,"wal-append.queue_time_us":101390}
I20260709 12:27:57.015151  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.015547  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3165) took 1289 ms. Trace:
I20260709 12:27:57.015836  6305 raft_consensus.cc:2376] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20. Re-sending same reply.
I20260709 12:27:57.015978  6302 rpcz_store.cc:276] 0709 12:27:55.726076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.379954 (+653878us) service_pool.cc:224] Handling call
0709 12:27:57.015359 (+635405us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54001b21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:57.015487 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968717184}
I20260709 12:27:57.017611  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3175) took 1229 ms. Trace:
I20260709 12:27:57.017971  6305 rpcz_store.cc:276] 0709 12:27:55.788249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.016659 (+1228410us) service_pool.cc:224] Handling call
0709 12:27:57.017027 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.017250 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.017275 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:57.017319 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:57.017384 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.017435 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.017555 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.019972  6305 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:57.020569  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3178) took 1106 ms. Trace:
W20260709 12:27:57.021272  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.020910  6305 rpcz_store.cc:276] 0709 12:27:55.913881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.019132 (+1105251us) service_pool.cc:224] Handling call
0709 12:27:57.019596 (+   464us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.020333 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.020520 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.022979  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:56.958858  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.024654  7320 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:27:57.024761 13090 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:57.025049  7320 pending_rounds.cc:85] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 14
I20260709 12:27:57.025444  7320 pending_rounds.cc:107] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:27:57.025750  7320 raft_consensus.cc:2889] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:57.025768 13616 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.026597 13616 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:56.958611  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 323474us
I20260709 12:27:56.954586  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.028417  6454 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:56.962581  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:56.923517 13672 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.028963 13616 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:57.029475 13672 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:56.957619  6304 rpcz_store.cc:276] 0709 12:27:55.786658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.797566 (+1010908us) service_pool.cc:224] Handling call
0709 12:27:56.843086 (+ 45520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.910236 (+ 67150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.910299 (+    63us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.910374 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:56.918730 (+  8356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.918862 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.919071 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67498,"raft.queue_time_us":64166,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":5187}
I20260709 12:27:56.956486 13709 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.964336  6455 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:56.959585  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:56.914124  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 60158.1us
I20260709 12:27:57.022794 13221 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:57.028762  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:56.971933  6307 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:57.023316  6305 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:57.026669  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:57.031208  6308 raft_consensus.cc:2376] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:27:57.017741  6302 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:27:57.006007  6300 rpcz_store.cc:276] 0709 12:27:55.787710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.003959 (+1216249us) service_pool.cc:224] Handling call
0709 12:27:57.004602 (+   643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.004913 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:57.004959 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:57.005064 (+   105us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:57.005121 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.005197 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.005348 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:57.036158  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.036118  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3176) took 1247 ms. Trace:
I20260709 12:27:57.036963  6302 rpcz_store.cc:276] 0709 12:27:55.788505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.016819 (+1228314us) service_pool.cc:224] Handling call
0709 12:27:57.017339 (+   520us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:57.035662 (+ 18323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.036054 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:57.033262 13709 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:57.039215 13655 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:27:57.040207 13655 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:57.041692  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.043303  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.044144  7331 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.044461  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.045285  7329 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:57.046557  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3171) took 1259 ms. Trace:
I20260709 12:27:57.047000  6307 rpcz_store.cc:276] 0709 12:27:55.787204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.971397 (+1184193us) service_pool.cc:224] Handling call
0709 12:27:56.971739 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.046264 (+ 74525us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.046460 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:56.964901 13382 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:57.058975  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.062404  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.009377 13654 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:57.061264  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:57.065534  7320 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:27:57.065569  7329 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:57.065850  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.066708  6305 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.059700  7314 raft_consensus.cc:2376] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17. Re-sending same reply.
I20260709 12:27:57.067857  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.046054 13655 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.071758  6304 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:57.012709 13630 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.077584  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.071414  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.059234 13639 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.068805 13713 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:57.081156 13672 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:57.093573 13523 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:57.084241  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57769.5us
I20260709 12:27:57.081377  7318 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.074034 13726 ts_tablet_manager.cc:933] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:57.083559  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.105731 13639 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.108045  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.119320  6302 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.113818  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.121011 13650 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.122732  7320 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.123194 13639 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:57.126839  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48255.8us
I20260709 12:27:57.126984  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.127847  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.128379  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:57.129058  7316 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:57.129091 13690 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:57.129613  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.130220  6302 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.133679  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.134320  6303 raft_consensus.cc:2376] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20. Re-sending same reply.
I20260709 12:27:57.134954 13683 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.135382  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.136041  6303 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:57.135772 13683 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.137442 13382 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.139442 13650 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:57.140647  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.141413  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
I20260709 12:27:57.126389  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.143150  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.143767  6303 raft_consensus.cc:2376] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17. Re-sending same reply.
I20260709 12:27:57.146800  6304 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:57.146831  7331 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:57.148360  6308 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:57.125165  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.153076  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.154320  6301 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.155259  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.155725 13721 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:57.156323 13721 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.156611  6459 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:57.156690 13721 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:27:57.159299  6424 leader_election.cc:336] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.862s (SENT)
I20260709 12:27:57.159749  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.161090 13616 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:57.163566  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.164731  6455 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:57.165689  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.165874  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.094237 13722 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:57.167678  6450 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:27:57.163548  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:57.169021  6454 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:57.169426 13547 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:57.159782  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.170882 13342 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:57.171295 13342 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:57.171523 13433 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:57.172451 13639 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:57.172310 13342 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.172761  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:57.155802  6301 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:57.165599 13693 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 12:27:57.174036 13342 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:57.174753 13616 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:57.175225 13616 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:57.175598 13616 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:57.154376  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70907.9us
I20260709 12:27:57.179165  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.181540  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.182864 13722 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:27:57.166579  6307 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.190232  7320 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:57.197815  6451 raft_consensus.cc:2368] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:27:57.197992  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:27:57.202944  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.207701  6302 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:57.209151  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.212988  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.213781  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.215281 13705 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:57.215893 13705 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.216356 13705 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.218727 13258 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:57.219273 13258 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.219720 13258 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.223214  6304 raft_consensus.cc:2376] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19. Re-sending same reply.
I20260709 12:27:57.224509  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.225116  6304 raft_consensus.cc:2376] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19. Re-sending same reply.
I20260709 12:27:57.225944  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.227080  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.231841 13523 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:57.233054  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60421.3us
I20260709 12:27:57.236413  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.236666  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.236976  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:57.237521  6308 raft_consensus.cc:2368] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:27:57.238396  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.238832  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.240101  6303 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.241113  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.241930  6303 raft_consensus.cc:2368] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
W20260709 12:27:57.242959  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61485us
I20260709 12:27:57.244027  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.244585 13616 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.246872  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.247560  6459 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:57.248081 13630 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:57.248762  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:57.249563  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55073.8us
I20260709 12:27:57.249641 13693 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:57.188140 13650 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:57.250689 13693 consensus_queue.cc:1243] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:27:57.251291 13693 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:57.251077 13650 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.251659 13693 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:57.252617 13693 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.253247 13650 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.253909 13693 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.255182  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:27:57.255630  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44406.1us
W20260709 12:27:57.256346  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.238945  6308 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.254179  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:57.089725 13629 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.257597  6304 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:27:57.257591  7335 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:57.261418 13632 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:57.262370 13632 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:57.262763 13632 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:57.262799 13616 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:57.263926 13632 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.265156 13632 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.266513  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.267499  6310 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.268966 13629 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.249675 13722 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:27:57.205138 13639 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:57.126163 13655 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:27:57.261261  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47608us
I20260709 12:27:57.246182 12840 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:57.296259  6309 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:57.273838  6300 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:27:57.290697  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.291059  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:27:57.279563  7318 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:57.295053  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.298621  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.256527 13683 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.301455  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3172) took 1514 ms. Trace:
I20260709 12:27:57.301858  6309 rpcz_store.cc:276] 0709 12:27:55.787457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.979743 (+1192286us) service_pool.cc:224] Handling call
0709 12:27:57.296097 (+316354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.296718 (+   621us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.301251 (+  4533us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5400140d50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:57.301400 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726299520}
I20260709 12:27:57.303296  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.303758  6309 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:57.299875 13713 raft_consensus.cc:2764] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:27:57.305593 13616 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:57.306195 13616 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:57.306846  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.302263  6307 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:57.312366  6305 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:27:57.312484  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.313277  6304 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.205442 13709 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.274127 13523 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:27:57.314309  6302 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:27:57.310757 13629 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.297981 13433 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.318324  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.325801  6307 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:57.331552  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.333838  6451 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:27:57.352202  6451 pending_rounds.cc:85] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 8
I20260709 12:27:57.352547  6451 pending_rounds.cc:107] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:27:57.352860  6451 raft_consensus.cc:2889] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:57.334558  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 122342us
I20260709 12:27:57.335675  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.307219 13616 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.338356  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.315620 12840 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:57.292163 13721 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.356946  6308 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:57.361927  6300 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:27:57.323150 13726 tablet_copy_client.cc:323] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:27:57.364686  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.365849  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:27:57.368261  6454 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 16: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:57.369277  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.369752 13722 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:27:57.370010 13721 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:57.370327 13655 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.373379 13705 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.362995 12840 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.374318 13709 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.386700 13632 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:57.388963  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.389405 13722 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:57.390086  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58667.7us
I20260709 12:27:57.391099  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:57.358834 13630 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:57.391947 13630 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.392333 13630 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:57.392875  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.393735 13632 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.353250 13713 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:27:57.381839  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115443us
I20260709 12:27:57.384264 13677 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:57.396298  6272 leader_election.cc:400] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:27:57.395035 13722 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.397125 13713 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.403826  6272 leader_election.cc:403] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:57.407693  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:57.414466  6310 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:57.414978  6455 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:57.416970  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.417855  6450 raft_consensus.cc:2368] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17.
I20260709 12:27:57.355533 13629 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.418931 13342 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:57.420815 13629 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.419632 13342 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.377135 13258 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.411471 13382 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:27:57.427443 13382 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.431278 13382 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:57.433008 13258 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:57.430595  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75330.8us
I20260709 12:27:57.425611 13639 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:57.436650  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.437676  6300 raft_consensus.cc:2368] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23.
I20260709 12:27:57.412344 13677 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.438411  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.439352 13650 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:57.439728 13677 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.440560  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.442617 13655 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:57.442845 13654 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:57.443204  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.443542 13654 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.444042 13090 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:57.444516 13090 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.444042 13654 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.446208  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.447081  6301 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.412575 13713 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:57.439522  6302 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.443359  6307 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:57.451738  6454 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:57.451797 13629 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.455549  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.458364  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:57.441812  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.455513  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:57.468083  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.467062  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:57.473059  6351 leader_election.cc:400] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:27:57.473356  6351 leader_election.cc:403] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:57.473870  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.474141  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.475739  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.479339  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.481539  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:57.482100  6456 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:57.482635  6454 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:57.474665  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.463281 13639 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:57.467401 13342 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.484395  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:57.508513  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:57.509145  6450 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.506738  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.469738 13672 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:57.512740  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.495975 13547 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:57.484037 13386 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.519095  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:57.520059  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61820.3us
I20260709 12:27:57.496657 13392 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:57.528621  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.463129 13090 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.526762  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.472070 13650 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:27:57.478782 13709 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:57.533821  6455 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:57.524873  6456 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:57.490767 13616 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.519768 13523 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:57.538573 13616 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.540302  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:57.518661 13712 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:27:57.551035  6451 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.529105 13655 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.552978  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.566610 13639 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.542423 13710 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:57.569104 13090 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.570583 13709 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.571758 13709 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.539860 13632 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.574846 13632 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.576191 13743 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:57.576846 13743 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.559074 13690 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 12:27:57.579248 13712 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:57.579782 13712 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.579629 13630 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.580879 13712 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:57.583421 13733 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:27:57.586165  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:57.587026  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.579852 13690 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:57.588145 13690 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:57.588380 13650 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:57.588716 13677 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:57.589365 13677 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.589169 13690 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.589959  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.557664  6459 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:57.587311  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:57.556043 13616 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:57.582240 13712 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.595254  6304 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:27:57.590624  6384 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
W20260709 12:27:57.605477  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 142110us
W20260709 12:27:57.606526  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80735.1us
I20260709 12:27:57.609344  6450 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:57.610093  6302 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:27:57.587230 13382 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.591114 13655 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.612917  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.591480 13690 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.614926  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.615772  6308 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:57.617374  6300 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:57.619117  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.620276  6307 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.622161  6454 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:27:57.589313 13650 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:57.625910  6453 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:27:57.624768  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:57.605253 13740 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:57.590814 13715 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:57.595168 13710 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:27:57.640604  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.641069  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.614948 13655 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.636397  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.649204  6308 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:57.652477  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.653381  6384 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.645332  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.628892 13650 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.635964 13715 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.590452 13677 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:57.657655  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50089us
I20260709 12:27:57.662015  6382 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.668553  7288 leader_election.cc:400] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:27:57.668852  7288 leader_election.cc:403] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:57.671118  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.671197  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.636152 13672 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.658020 13523 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:27:57.657764 13715 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:57.661474 13743 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.672681  6451 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:57.659231 13392 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.697324  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:57.691344 13693 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
W20260709 12:27:57.705574  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69295.6us
I20260709 12:27:57.705907  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.662427 13654 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.707985  6425 leader_election.cc:400] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23.
I20260709 12:27:57.708333  6425 leader_election.cc:403] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:57.710342 13616 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23.
W20260709 12:27:57.710669  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44530.2us
I20260709 12:27:57.711242 13616 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:57.711478  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.697952  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.697995  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.712792  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:57.717425 13433 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:57.717900 13433 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:57.718293 13433 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.718477 13713 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.720777 13713 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.721745 13672 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.723317 13523 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:57.723403 13650 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.739416  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.714115 12840 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:57.749199  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.749861  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.752211  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:57.756201  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.755069  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:57.760159  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.760288 13090 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:57.751585 13650 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.740368 13735 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:27:57.765941  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:57.739585 13616 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:27:57.753490 12840 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:57.767320  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.760892 13090 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.769809 13752 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:57.778321  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:57.770303 13746 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:27:57.761430 13750 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:57.713294 13710 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.775825 12840 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:57.777021  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.758932 13693 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:57.815125  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.758191 13709 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:57.793679  6300 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.798034  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:57.805035  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:57.802186 13743 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 12:27:57.765367 13748 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.807937 13722 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:57.798206 13750 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.813719  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:57.796478 13382 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:57.837039  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.800104 13655 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.838238  6382 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:57.798586 13392 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.839325 13745 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:27:57.844009  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.845000  6301 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:57.813730 13683 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:57.802017 13523 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:57.848651  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.821381 13709 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.850772  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.793771 13342 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:27:57.848794  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.882560  6308 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.849401  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:57.848731  7287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.585s (SENT) [suppressed 51 similar messages]
I20260709 12:27:57.871927  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.883009  6309 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.837574 13750 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:57.891681  6380 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:57.894694  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.896059  6382 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:57.826604 13735 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:27:57.850512 13683 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.852851 13690 raft_consensus.cc:2764] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:27:57.902724  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:57.882653 13709 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:57.796038 13752 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.917188  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:57.918279  6308 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:57.874426 13547 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:27:57.920475  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.921993  6380 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:57.912950 13683 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:57.931517  6384 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:27:57.932199 13754 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:27:57.930843 13755 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:27:57.933022  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.934340 13632 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:57.936949  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet d53ab19d424b4a85af2153ef24520678 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:27:57.937600  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet d53ab19d424b4a85af2153ef24520678 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-d53ab19d424b4a85af2153ef24520678
I20260709 12:27:57.940893  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:57.941677  6376 raft_consensus.cc:2376] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27. Re-sending same reply.
I20260709 12:27:57.943100  6376 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:57.943701  6382 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:57.924223 13433 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.932723  6378 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
W20260709 12:27:57.958617  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:57.929535 13758 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:27:57.932883 13754 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:57.895483 13748 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:57.962795  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:57.960462 13630 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:57.913949 13655 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:57.990202  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 141495us
W20260709 12:27:57.999943  7287 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.667s (SENT)
I20260709 12:27:58.003788  6308 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
W20260709 12:27:58.004889  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 161627us
I20260709 12:27:57.945348 13090 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:57.961661 13386 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:27:58.006759  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.011197  6382 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:27:58.009639 13433 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:58.008302  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 185911us
I20260709 12:27:58.015640  6376 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:58.020133  6380 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:58.022115  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.023018  6382 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:58.023188 13655 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.027311  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.010169 13714 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.030536 13748 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.030259 13714 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.029599  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.026906 13754 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.033291  6379 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:58.035889  6382 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
W20260709 12:27:58.037413  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.015671  7287 leader_election.cc:336] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.515s (SENT)
I20260709 12:27:58.045796  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:58.079242  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:58.082118  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.083336  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:58.084120 13693 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:58.086155  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.086834  6376 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:58.088521  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:58.096910  7331 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.094394  6304 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:27:58.102324  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.044449 13690 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.044665 13722 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:58.106083  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.107013  7320 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
W20260709 12:27:58.087867 13629 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.634s	user 0.008s	sys 0.004s
I20260709 12:27:58.110071  6382 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:58.110824  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.086573 13704 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:58.121089  7331 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:27:58.095283 13755 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:58.115898  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.134414  7317 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:27:58.130978  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:58.134213  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.155200  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:58.149914 13777 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:27:58.099500 13386 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.135907 13693 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.164621  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.138945  7331 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:58.154578  6377 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:27:58.157480  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:58.163053  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 147407us
I20260709 12:27:58.167529  6382 raft_consensus.cc:2376] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20. Re-sending same reply.
I20260709 12:27:58.122978 13629 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:27:58.168453  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.169437  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.123935 13392 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:27:58.170603  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.170964 13392 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:27:58.171283  7317 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:58.171435 13392 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:58.172407  6395 tablet_copy_source_session.cc:215] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:27:58.172701 13392 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.173864  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.174163 13392 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:27:58.176529  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.178627  6378 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:58.136482 13704 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:58.183418 13770 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 12:27:58.157577 13755 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:58.186555 13755 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.187641 13746 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:58.188115 13746 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:58.189257  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.180917  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.193449  6381 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:27:58.193599  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:58.189013 13746 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.185469 13654 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.197279  6376 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:27:58.205096  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.206540  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.207919  6376 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:58.209038 13382 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:58.182106 13704 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.211338  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.125919 13690 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.201510  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.202436  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.199829  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:58.194244  6378 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:27:58.212831  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.215385  6301 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:58.220369  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.226267  6305 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:58.227537  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:58.228168  6303 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.230000  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.230633  6378 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:27:58.168664 13386 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.234875  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:58.235754  6451 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:58.235603 13683 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.237274 13386 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.237764  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.238070  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:58.238381  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:58.220229 13639 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.238761  6454 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:58.239266  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.239034 13639 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.239403 13258 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.241683  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.242082  6385 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:58.242465  6382 raft_consensus.cc:2376] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19. Re-sending same reply.
I20260709 12:27:58.243620  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.246838  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.180089 13752 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.190856 13722 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.251013  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:27:58.264889  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.239995 13258 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.246863  6450 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:27:58.249584  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49837.4us
I20260709 12:27:58.280339  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.281383  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:58.272141  6454 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:27:58.217417 13705 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.273381  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.285358  7331 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:58.285482  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.263187 13386 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:58.280366  7318 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:27:58.292193  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80942us
I20260709 12:27:58.290050  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.311082  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:58.299702  7335 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:27:58.315758  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.315088  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:58.317911  6310 raft_consensus.cc:2368] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:58.289433  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:58.288717  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:58.319468  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.319646  6452 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:58.324153  6378 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:27:58.315837  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.326449  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.327685  6381 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:27:58.326968  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:58.329851  6450 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:58.325495  6303 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:58.332937  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.334479  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:58.340627  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.287617 13632 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:58.289592 13386 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.306283 13746 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.342350  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.343061  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.354446  7331 raft_consensus.cc:2368] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:58.345263  6351 leader_election.cc:400] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:58.254599 13750 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.290261 12840 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:58.359629  6457 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:58.355576  6351 leader_election.cc:403] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:58.363974  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.356494  6378 raft_consensus.cc:2376] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16. Re-sending same reply.
I20260709 12:27:58.288225 13714 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.369289  6451 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:58.366671  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:27:58.370361  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.364820  6381 raft_consensus.cc:2376] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19. Re-sending same reply.
I20260709 12:27:58.287825 13755 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:27:58.363008  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:58.358773 13386 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.385303  6384 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:58.385782  6381 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:27:58.389170 13783 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:27:58.389382 13749 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:58.390594 13769 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:27:58.389209 13763 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:27:58.391901 13761 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.392664 13761 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.394914  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:58.395388 13778 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:27:58.395586  6458 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:58.397439 13721 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:58.398759  6459 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:27:58.362146 13743 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.402179  6385 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
W20260709 12:27:58.402326  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87373.1us
I20260709 12:27:58.404608  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.406065  6310 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:58.406701  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.411935  6379 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:58.378556 12840 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:58.404373 13715 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.700s	user 0.011s	sys 0.001s
I20260709 12:27:58.419932  6381 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:58.401418 13523 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:27:58.426192  6379 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:58.383859 13735 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:27:58.407532 13221 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:58.415592 13655 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:58.437502  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.441403  6459 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:58.409575 13783 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:27:58.409211 13743 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.434825 13749 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.430078 13715 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.434667 13639 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.444000 13735 consensus_queue.cc:1243] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:27:58.465107  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58622.9us
I20260709 12:27:58.466940  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.445017 13221 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:58.477167  6307 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:58.483194 13258 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.483840 13761 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:58.484515 13761 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:58.485061 13761 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:58.463222 13783 consensus_queue.cc:1243] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:27:58.488220  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.463635 13690 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:58.478688 12840 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.486083 13761 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.499485 13726 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d53ab19d424b4a85af2153ef24520678. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:27:58.482645 13709 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
I20260709 12:27:58.474002 13221 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:58.517061  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 206549us
W20260709 12:27:58.506417 13775 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.279s	user 0.000s	sys 0.002s
I20260709 12:27:58.524273  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:58.474740 13616 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:58.525434  6304 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:58.526162  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.526793  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.526886  6300 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:27:58.491405 13748 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.509388 13785 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:58.507911 13750 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:27:58.521447 13779 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:27:58.539506  6305 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:27:58.540542  6302 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:27:58.542049  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:58.461043 13749 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:27:58.534946 13709 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.541596 13748 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.537631 13743 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.540342 13785 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:58.543960  6310 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:58.540493  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:58.547014 13767 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.558261 13655 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:58.574360  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:58.582731 13677 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:27:58.643075  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.579429 13785 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:58.686034  6378 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:27:58.688436  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 221614us
W20260709 12:27:58.689913  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 103146us
I20260709 12:27:58.690197  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.579700 13523 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:58.692541  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.701472  6459 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:58.703769  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.707885 13523 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.714368 13386 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.714941 13715 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.716064 13342 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:58.716472 13342 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:58.716547 13386 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.717350 13342 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.718158 13749 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.719466 13791 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:58.720002 13748 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:58.722255  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.810220  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.811103  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:58.821086  7317 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.811422  6304 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:58.688481 13629 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:58.813743  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 123085us
W20260709 12:27:58.752005  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.692118 13721 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:58.825218  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:27:58.849550  6452 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.522s	user 0.014s	sys 0.000s
I20260709 12:27:58.850306  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.850678  6452 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:27:58.857102 13722 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:58.859345 13722 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:58.862514 13342 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:58.862928  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.863795  6381 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:27:58.865125  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:58.865850  6381 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:58.867190 13715 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:58.867872 13715 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.871279 13639 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.871811 13639 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:58.873915 13639 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.852653 13629 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.808202 13616 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:58.875831 13616 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.876230 13616 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:58.876633 13629 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.809254 13258 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.878754 13258 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.870373 13715 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.879266  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:58.879815  6300 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:58.880905 13258 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:58.883934 13392 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.692s	user 0.008s	sys 0.002s
I20260709 12:27:58.884806  6382 raft_consensus.cc:2410] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:27:58.892813 13714 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.893522 13683 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.893471 13714 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.584892 13767 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:58.903812  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 329558us
I20260709 12:27:58.823882 13713 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.854203 13705 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.913659  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:27:58.907624  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 203913us
I20260709 12:27:58.908213  7317 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:58.914788  7331 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:58.908423  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.918228  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.919135  7320 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:27:58.919881  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:58.922442  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.923332  7335 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:27:58.913157 13392 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:27:58.936299  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:58.937925  7329 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
W20260709 12:27:58.941675  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.948544 13714 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.949182 13714 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:58.949554 13714 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.949914 13714 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:58.952476 13793 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.470s	user 0.007s	sys 0.000s
I20260709 12:27:58.952382 13785 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:58.957369  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 132245us
W20260709 12:27:58.965422  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45656.3us
W20260709 12:27:58.965440  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57100.2us
I20260709 12:27:58.966377  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:58.913774 13713 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.916671 13705 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.967240 13790 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:27:58.978431  6310 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:58.979673  6348 leader_election.cc:400] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17.
I20260709 12:27:58.980068  6348 leader_election.cc:403] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:58.980576  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:58.981191  7320 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:58.981761 13746 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.607s	user 0.007s	sys 0.005s
I20260709 12:27:58.982646  6379 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:27:58.982991  6379 pending_rounds.cc:85] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 14
I20260709 12:27:58.983197  6379 pending_rounds.cc:107] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:27:58.983403  6379 raft_consensus.cc:2889] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:58.987774 13767 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:58.989984  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:27:58.990690  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:27:58.991736 13797 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17.
I20260709 12:27:58.991843 13616 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:58.992262  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:58.994134 13616 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:58.994566 13768 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:58.996323 13705 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.997882 13713 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:58.998848 13767 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:59.000147 13655 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:59.018138 13785 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:59.018203 13258 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:59.000775 13715 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:58.974610 13709 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.019644 13258 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.020606 13767 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:59.020998 13767 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:59.022184 13258 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.008816  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:59.019804 13715 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.021931 13767 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.002148  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:58.973453 13804 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:59.003871 13714 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.026239  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:58.980062 13672 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:59.033685  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:59.034530  7320 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:27:59.034972  6304 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:59.016718 13790 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:59.016868 13768 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:59.036340  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.028813 13392 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:27:59.027444 13746 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:59.027760 13523 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:59.029297 13726 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43762 (request call id 996) took 1983 ms. Trace:
I20260709 12:27:59.048246 13523 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.031404 13672 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:59.048983 13523 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.049362 13801 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:27:59.049798 13672 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.033727 13804 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:59.052228 13804 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.048333 13726 rpcz_store.cc:276] 0709 12:27:57.036104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.042713 (+  6609us) service_pool.cc:224] Handling call
0709 12:27:57.129881 (+ 87168us) ts_tablet_manager.cc:934] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
0709 12:27:57.450928 (+321047us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4c002a55c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1573a20 00007f6fa1584a29 00007f6fa158513f 00007f6fa15492ab 00007f6fa1576f3c 00007f6fa1577461 00007f6fa1577768 00007f6fa72fd2eb 00007f6fa8222d3b 00007f6fa821d008 00007f6fa821c405
0709 12:27:58.428515 (+977587us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:27:58.889309 (+460794us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:58.889449 (+   140us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:27:59.005588 (+116139us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:27:59.005939 (+   351us) tablet_replica.cc:206] Creating consensus instance
0709 12:27:59.006323 (+   384us) ts_tablet_manager.cc:989] Replica registered.
0709 12:27:59.015538 (+  9215us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":44896,"spinlock_wait_cycles":24418560,"tablet-copy.queue_time_us":28389,"thread_start_us":161818,"threads_started":2}
I20260709 12:27:59.055464 13768 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.056162 13726 tablet_copy_client.cc:806] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:27:59.056919 13768 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:59.058372 13767 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.036571 13709 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:59.045722 13752 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:27:59.061673 13221 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.522s	user 0.008s	sys 0.005s
I20260709 12:27:59.050884 13672 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:59.064191 13796 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:27:59.065112 13796 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:59.065448 13796 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:59.044974  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:59.061789  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.070510 13752 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:59.077095 13714 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.078915 13715 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.079150  6351 leader_election.cc:400] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18.
I20260709 12:27:59.079571 13715 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:59.080116  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:59.081367  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:27:59.082607  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:59.066365 13796 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.085729  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:59.086345  7320 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.056919 13726 tablet_copy_client.cc:670] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
W20260709 12:27:59.121060 13754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.008s	sys 0.021s
I20260709 12:27:59.122048 13790 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:27:59.122270 13704 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.013s	sys 0.013s
I20260709 12:27:59.062168 13221 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:59.147837  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.StartTabletCopy -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 65711.1us
I20260709 12:27:59.223730  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:59.148403 13749 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:59.150970  6376 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
W20260709 12:27:59.151409  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50619us
I20260709 12:27:59.047041 13616 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:27:59.079424  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:59.151660  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1083) took 1218 ms. Trace:
I20260709 12:27:59.236949  6384 rpcz_store.cc:276] 0709 12:27:57.933708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.962356 (+ 28648us) service_pool.cc:224] Handling call
0709 12:27:59.087901 (+1125545us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:59.151523 (+ 63622us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969326336}
I20260709 12:27:59.157560 13090 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:58.404474  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:59.238669 12840 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:27:59.239208  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 88412.9us
I20260709 12:27:59.151017 13791 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.187384 13790 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:59.156423 13754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.008s	sys 0.022s
I20260709 12:27:59.189811  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.190678 13785 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:59.150405 13523 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.191493 13632 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:27:59.193212  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.193387 13796 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.248767  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.206732  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:27:59.249364  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59618.1us
I20260709 12:27:59.249995  6303 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:59.250222  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.208168  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1075) took 1383 ms. Trace:
I20260709 12:27:59.208885  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:59.210959  6309 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:59.212657  6300 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:59.215233  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:59.224536  6305 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:59.226352  7320 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
W20260709 12:27:59.216060 13704 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.013s	sys 0.013s
I20260709 12:27:59.230685  7335 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:59.258909  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.226786  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:27:59.238437  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 159085us
I20260709 12:27:59.260171  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.240020 12840 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.261129 12840 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:59.241073  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:59.261673  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46529.1us
I20260709 12:27:59.242201  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.262712  6452 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:59.243201  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:59.241056  6424 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.638s (SENT)
W20260709 12:27:59.195561  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.264007  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:59.264703  6452 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.263901  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.240880 13710 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:59.255980  6308 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:59.256652  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:59.266105  6451 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:59.264045 13221 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.263937  6458 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:59.271103 13488 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=8 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:27:59.272172 13488 tablet_bootstrap.cc:492] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:27:59.272909 13488 ts_tablet_manager.cc:1403] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 2.358s	user 0.055s	sys 0.055s
I20260709 12:27:59.260294  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:59.274006  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.259534  6453 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:59.252900  6385 rpcz_store.cc:276] 0709 12:27:57.827013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.827207 (+   194us) service_pool.cc:224] Handling call
0709 12:27:58.189094 (+361887us) spinlock_profiling.cc:243] Waited 9.53 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182e9c 00007f6fa0182a5b 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:27:59.207958 (+1018864us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b5400195ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:59.208092 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141879808}
I20260709 12:27:59.258361  6459 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:27:59.275440  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:27:59.279366  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 52661.7us
I20260709 12:27:59.291055  6303 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:27:59.286211 12840 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.245065 13749 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:59.298481  6451 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
W20260709 12:27:59.296576  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 138093us
I20260709 12:27:59.246443 13090 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:59.279515  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:59.300598  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:59.302974  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52822us
I20260709 12:27:59.303094  6451 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:27:59.304118 12840 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.244215 13767 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:27:59.301225  6452 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
W20260709 12:27:59.306293  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65291us
I20260709 12:27:59.326689 13802 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.328269  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.328161  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.274775 13258 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:27:59.333316  6456 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:27:59.319408  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.278692 13797 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.323503  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.332721  6455 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:59.334273  6385 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:59.324265  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.338995  6384 raft_consensus.cc:2368] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19.
I20260709 12:27:59.324630  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.325117  6379 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:59.248612 13785 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.301811 13755 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:59.295856 13639 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:59.344830  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:59.346796  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:59.341742  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:59.354287  6450 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:59.352218  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:59.295477 13629 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:59.297228 13710 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:27:59.305671 13616 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:59.360980  6458 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:59.361618  6453 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:59.371230  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.333597 13672 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:59.372020  6452 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:59.370168  6377 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:27:59.337136 13746 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:59.295657 13715 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:27:59.383818  6450 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:27:59.387952  6451 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:59.296028 13630 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
I20260709 12:27:59.360702 13639 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.360410 13629 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:59.337302 13392 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 12:27:59.392525  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.302006 13705 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:27:59.365314 13713 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.398540  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:27:59.402068  6452 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:27:59.398952  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:27:59.399616  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71547.1us
I20260709 12:27:59.404282  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:59.405195  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 108526us
I20260709 12:27:59.345916 13690 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:27:59.375953 13672 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.370960 13616 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:59.401427  6459 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:27:59.385519 13802 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:27:59.412758  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 73484.3us
I20260709 12:27:59.415753 13791 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:59.415119  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.416751 13791 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:27:59.417338  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:27:59.418551  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 111616us
W20260709 12:27:59.413162  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 133715us
I20260709 12:27:59.396616 13639 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:59.399819 13629 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.426110  6453 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:27:59.426930  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:59.413136 13221 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:59.412770 13672 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.431854  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.437716  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3047) took 1316 ms. Trace:
I20260709 12:27:59.439316  6376 rpcz_store.cc:276] 0709 12:27:58.121178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:58.208814 (+ 87636us) service_pool.cc:224] Handling call
0709 12:27:59.150791 (+941977us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.225832 (+ 75041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.437505 (+211673us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400037850. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:59.437653 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2165621504}
I20260709 12:27:59.447695  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:59.432897 13632 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.452972  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:59.455955  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.458598  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.458986  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.459786  6377 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:27:59.460208  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:27:59.461537  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.458654  6376 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:27:59.439036 13826 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.463469  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:59.433575 13802 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:59.459820  6382 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:27:59.439602 13713 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:59.470118  7287 leader_election.cc:400] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19.
I20260709 12:27:59.418629 13090 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.474714 13822 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:59.475086 13726 tablet_copy_client.cc:538] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:27:59.468418 13823 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:59.496657 13823 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.497082 13823 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.497633  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.498291  6381 raft_consensus.cc:2376] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17. Re-sending same reply.
W20260709 12:27:59.467406 13812 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:27:59.480132  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:59.500252  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.487888 13672 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.487988 13629 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:59.488832  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48418.9us
I20260709 12:27:59.503211  7331 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:27:59.494024  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.492008 13746 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:59.479647 13258 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:59.461318 13825 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:27:59.452105 13715 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.527802 13715 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.529471 13785 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:59.452840 13616 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.530064 13715 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.504118  6379 raft_consensus.cc:2368] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:27:59.530557 13616 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:59.504030  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:27:59.506036  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50919.5us
I20260709 12:27:59.420980 13749 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.452335 13714 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:27:59.498647 13713 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:27:59.507580 13650 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:27:59.519994 13672 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:27:59.556221  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52273.1us
I20260709 12:27:59.449131 13710 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:59.521572  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51606.5us
I20260709 12:27:59.508118 13523 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.522603 13382 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:59.522871 13639 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.521490  7316 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:27:59.526357 13824 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:59.526997  7329 pending_rounds.cc:85] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 11
I20260709 12:27:59.531383 13802 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.533201  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:27:59.479597  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:59.495296 13690 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.493870 13822 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.538825  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.546018  6385 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:27:59.543692  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:27:59.520612 13752 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:59.557123  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.505584 13746 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.494275 13797 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:27:59.558430  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.534093 13825 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:59.663357  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 84209.7us
W20260709 12:27:59.663903  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 106875us
I20260709 12:27:59.492507 13826 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:27:59.658623  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115011us
I20260709 12:27:59.656620 13683 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.671687 13683 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.675141 13713 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.662292 13690 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.680869 13690 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:59.666905 13824 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.685786 13677 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.658735 13823 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.686352 13677 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:59.611557  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.684476 13824 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.688164 13823 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.661357  7329 pending_rounds.cc:107] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:27:59.688763  7329 raft_consensus.cc:2889] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:59.665712  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:27:59.691622  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:27:59.695206  6305 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
W20260709 12:27:59.698275  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 159574us
I20260709 12:27:59.658067  7318 raft_consensus.cc:2410] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:27:59.701570  7317 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:27:59.701819  6303 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:27:59.703744  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.551963 13632 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.124s
I20260709 12:27:59.708580  6385 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:59.572160 13714 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:59.713070  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47466.2us
W20260709 12:27:59.708990  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 150634us
I20260709 12:27:59.715185  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:27:59.706122  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:59.571779 13629 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.660861 13752 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.719310  7331 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:59.711977  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.561913 13754 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:59.659451 13822 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:59.718014  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.669286 13746 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:59.718041  7316 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 14. (term mismatch)
I20260709 12:27:59.661828 13382 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:27:59.722252  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.738924  6381 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:59.722518  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.722625  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:59.722272  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.738339  7316 pending_rounds.cc:85] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 13
I20260709 12:27:59.725449  6308 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:59.740423  7316 pending_rounds.cc:107] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 17.14
I20260709 12:27:59.740476  7314 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:27:59.556042 13785 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.149s
I20260709 12:27:59.555444 13392 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.149s
I20260709 12:27:59.733352  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:27:59.703775 13804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.015s	sys 0.000s
W20260709 12:27:59.748180 13804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.016s	sys 0.000s
I20260709 12:27:59.696557 13683 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.688529 13824 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:59.754225 13824 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.737289  6384 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:27:59.688766 13825 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:27:59.668495 13650 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:59.740722  7316 raft_consensus.cc:2889] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:59.740288  6382 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:27:59.551700 13816 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.161s
I20260709 12:27:59.704111 13823 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:59.708997 13790 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:59.757377 13824 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.759912 13650 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.760491 13827 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:59.761288 13828 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:59.737767 13258 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:59.737905 13690 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:27:59.776526 12840 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:27:59.809369 12840 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.726012 13752 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:59.780754 13787 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.623s	user 0.004s	sys 0.000s
I20260709 12:27:59.723346 13386 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:59.727020 13754 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:59.798058  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64858.2us
W20260709 12:27:59.787629  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 308095us
I20260709 12:27:59.564047 13488 raft_consensus.cc:359] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:59.781633 13830 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:27:59.830618  6384 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:59.771845 13823 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.724156 13714 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:27:59.840976  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.739586 13712 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:27:59.785355 13650 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:59.850881  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.808277 13392 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:27:59.859413  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2913) took 2037 ms (client timeout 1897 ms). Trace:
W20260709 12:27:59.864308  6302 rpcz_store.cc:269] 0709 12:27:57.822230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.822375 (+   145us) service_pool.cc:224] Handling call
0709 12:27:59.859116 (+2036741us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b5400140fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:27:59.859304 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1543609600}
I20260709 12:27:59.825060 13829 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.810767 13828 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:27:59.752674 13382 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.877566  6384 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:59.859628 12840 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.827937 13827 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:27:59.796290 13090 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:27:59.844652 13754 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:59.878355  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:27:59.848528 13823 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:59.879858 13829 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:27:59.891057 13813 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.641s	user 0.004s	sys 0.000s
I20260709 12:27:59.905129  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:27:59.857689 13797 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:27:59.886510 13832 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:27:59.905565  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66439.4us
I20260709 12:27:59.878301 13828 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:59.912333  7331 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:59.892059 13382 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:59.919485  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:27:59.880735 13838 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:59.930032 12840 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:59.930673 12840 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:27:59.931317 12840 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:59.929384  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:27:59.936888  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:27:59.937750  6307 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:59.939661 13836 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:59.940404 13772 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:27:59.951035  6307 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:27:59.957563  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:27:59.960150  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:27:59.960922  7331 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:59.963960 13837 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:27:59.969542  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.919289 13547 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.970790 13826 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:27:59.924932 13488 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:27:59.970923 13547 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.972204 13827 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.973373 13547 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:27:59.922465 13832 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:27:59.944686 13386 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:27:59.988889 13386 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:59.989639 13639 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:27:59.934886 13748 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:27:59.951645 13824 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:59.992036 13824 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:27:59.992268 13817 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.837s	user 0.000s	sys 0.004s
W20260709 12:27:59.971284 13808 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.818s	user 0.003s	sys 0.001s
I20260709 12:27:59.983708  6309 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:27:59.992983 13824 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.023157  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:00.025013  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:00.031627  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:00.030833  6381 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:00.032379  6383 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:00.036453  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.037238  7320 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:59.955540 13632 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:00.047705  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:00.048161  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 123073us
I20260709 12:28:00.016535 13836 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:00.063651 13814 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:28:00.068830 13523 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:00.069365 13523 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:00.068935 13825 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.069872 13386 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.079697  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:00.081151  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:00.082252 13797 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:00.082526  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:00.082870  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:00.083490  6381 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:00.070426  6304 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:00.089178  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:00.070269 13523 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.071025  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:00.091255  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:28:00.092340  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:00.093019  6301 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:28:00.071707  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:00.072361  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:00.072458 13822 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.103943  7287 leader_election.cc:400] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18.
I20260709 12:28:00.073118  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:00.073259 13672 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:28:00.073706 13221 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.635s	user 0.007s	sys 0.008s
I20260709 12:28:00.032931 13714 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.114238 12840 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.074241 13829 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.076988 13746 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.047467 13630 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:00.134135  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47156.7us
I20260709 12:28:00.104884  6457 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:00.117630  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:00.118373  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:00.038432 13785 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:28:00.152921  6303 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:28:00.125000 13386 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.125304 13797 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:00.126915  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:00.036235 13488 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:00.149571  6308 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:00.133507  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:00.129527  6304 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:28:00.115190 13672 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:00.151496  6301 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:00.209961  6301 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:28:00.168793  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 80148.2us
I20260709 12:28:00.030757 13838 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.109s
I20260709 12:28:00.130228 13221 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:00.205303 12840 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:00.055097 13632 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.082937 13748 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 12:28:00.102687  7331 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:00.128614 13258 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:00.127534 13654 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:28:00.135039 13767 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:28:00.202414 13832 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.139053 13846 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:28:00.172842 13630 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.213963 13838 consensus_queue.cc:1243] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 27.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:00.255461 13488 ts_tablet_manager.cc:1434] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.982s	user 0.009s	sys 0.028s
W20260709 12:28:00.256055 13488 ts_tablet_manager.cc:1467] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Tablet startup took 982ms
I20260709 12:28:00.235586 13654 consensus_queue.cc:1243] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 16.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:28:00.243744 13709 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:28:00.265069  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 240676us
I20260709 12:28:00.272747  6275 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:00.274968  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 159880us
I20260709 12:28:00.276363  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:00.219600 13650 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:28:00.270720 13616 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.727s	user 0.013s	sys 0.000s
I20260709 12:28:00.316434  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:00.235865 13258 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.348837  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:00.284585 13796 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.017s	user 0.013s	sys 0.002s
I20260709 12:28:00.273202 13709 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.396261  6376 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:00.355902  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 200048us
I20260709 12:28:00.357025 13854 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:00.406786 13755 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:00.407454 13755 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:00.408021 13811 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.255s	user 0.008s	sys 0.000s
I20260709 12:28:00.409724 13755 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.420948 13822 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.427325 13746 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:00.441637  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 87687.8us
I20260709 12:28:00.442967  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:00.445807  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:00.446800 13854 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:00.447767 13854 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.449049 13854 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.450273 13258 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:00.450991 13547 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:00.452636 13632 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:00.221318 13836 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.397857  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:00.448043 13630 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:00.460657  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.466670  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:00.473351  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.478457 12840 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.339417 13825 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.540792  6457 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:00.479048 12840 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.337941 13639 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:00.264180 13849 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
W20260709 12:28:00.472628 13382 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.005s
I20260709 12:28:00.545576  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:00.262328 13672 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:00.545553  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 99797.1us
W20260709 12:28:00.549933  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 152190us
I20260709 12:28:00.459460 13832 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:00.477315 13630 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.550909 13864 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:28:00.611352  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.612177 12840 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.544325 13776 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:00.613720  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.618258  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:00.460108 13709 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:00.604920 13683 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:00.554572 13772 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:00.555011 13827 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:00.624933 13857 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:28:00.552306 13616 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:00.552457 13791 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:28:00.646847 12840 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:28:00.627951 13749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.864s	user 0.028s	sys 0.061s
I20260709 12:28:00.477598 13804 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
W20260709 12:28:00.651338  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 368751us
W20260709 12:28:00.652706  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53469.1us
W20260709 12:28:00.670881  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 543995us
W20260709 12:28:00.654213  6382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.001s	sys 0.017s
I20260709 12:28:00.695990  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:00.696384  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:00.551681 13258 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.099s
I20260709 12:28:00.698063  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:00.699541  6378 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:00.598441  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:00.619542  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:00.613646 13726 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:28:00.653856 13772 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:00.647770  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102245us
W20260709 12:28:00.615262 13790 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.830s	user 0.011s	sys 0.000s
W20260709 12:28:00.705113 13749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.942s	user 0.028s	sys 0.067s
I20260709 12:28:00.616451 13883 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:00.736802  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:00.723392 13828 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.772s	user 0.005s	sys 0.009s
I20260709 12:28:00.740172  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:00.652082 13639 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:00.650988 13776 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.616293 13630 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.653663 13746 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.702157 12840 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.653766 13832 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:00.616833 13815 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.701400 13342 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:00.705361 13827 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:00.732411  6458 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.008s	sys 0.009s
I20260709 12:28:00.644481 13864 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:00.738690  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:00.616173 13823 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.711s	user 0.013s	sys 0.008s
I20260709 12:28:00.735070 13868 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:00.737874  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 139460us
W20260709 12:28:00.711295  6382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.002s	sys 0.022s
I20260709 12:28:00.722039 13616 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:00.760123  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:00.772961 12840 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:00.732663 13772 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:00.775769  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:00.738271 13790 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:00.669332 13683 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:00.773105 13752 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.929s	user 0.008s	sys 0.005s
I20260709 12:28:00.742882 13632 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:00.838475  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:00.839140  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1017) took 1141 ms. Trace:
I20260709 12:28:00.738813 13816 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:28:00.742827 13693 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:00.840934  7316 pending_rounds.cc:85] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 18
I20260709 12:28:00.974071  7316 pending_rounds.cc:107] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 27.19
W20260709 12:28:00.823745  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112837us
I20260709 12:28:00.751339 13838 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:28:00.740769 13714 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 12:28:00.738862 13797 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:00.866948  6458 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.008s	sys 0.009s
I20260709 12:28:01.009809  6459 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:00.782325 13749 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.744982 13828 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.841231 13629 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:00.841509 13755 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:00.741302 13883 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:00.842094 13650 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.888195  6381 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:00.917407  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.932142  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.025612  6457 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:00.933262  7317 rpcz_store.cc:276] 0709 12:27:59.681249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.681465 (+   216us) service_pool.cc:224] Handling call
0709 12:27:59.701210 (+ 19745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:59.702010 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.822570 (+1120560us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b540043a3d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:00.822759 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44465408}
I20260709 12:28:00.820966 13746 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:00.888005 13861 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:28:00.884883 13705 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:00.974617  7316 raft_consensus.cc:2889] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:00.987979  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:00.989524  6307 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:00.990299  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:00.822846 13815 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:00.937297 13804 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.985713  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:01.017335  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:01.040164  6308 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:01.023864  7320 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:28:00.884402 13823 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:01.036427  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.037355  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:01.046960  6307 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 8
I20260709 12:28:01.047333  6307 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:28:01.047607  7314 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:01.047626  6307 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:01.009747 13677 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:00.879113 13639 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.012889 13767 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:01.063822  6380 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:01.064556  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.074467  6457 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:00.945446 13632 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.088399  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:01.056689 13754 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.147s	user 0.012s	sys 0.005s
I20260709 12:28:01.023985 13755 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:00.935941 13752 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.087076  6457 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:01.056742 12840 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.047289 13766 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.358s
I20260709 12:28:01.048265 13629 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.160786  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1313) took 1148 ms. Trace:
I20260709 12:28:00.889951 13868 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.016381 13797 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.052028 13873 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:00.944413 13832 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.025604 13749 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.182572 13797 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.064059 13616 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.184274 13749 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.089495 13824 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:01.103642 13755 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:01.130693 13880 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:28:01.111786 13870 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.100952 13869 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.394s	user 0.001s	sys 0.000s
I20260709 12:28:01.094539 13090 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:01.174474  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.169442 13790 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.161281  7329 rpcz_store.cc:276] 0709 12:28:00.012877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.013051 (+   174us) service_pool.cc:224] Handling call
0709 12:28:00.013511 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.016522 (+  3011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.016582 (+    60us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.037281 (+ 20699us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.314748 (+277467us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:00.700147 (+385399us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.700430 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.700528 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.160538 (+460010us) raft_consensus.cc:1685] finished
0709 12:28:01.160565 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.160721 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10418,"prepare.queue_time_us":458081,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":11636,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":15910,"thread_start_us":488346,"threads_started":2,"wal-append.queue_time_us":495778}
W20260709 12:28:00.256580 13488 ts_tablet_manager.cc:1469] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Trace:
0709 12:27:49.947623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.952321 (+  4698us) service_pool.cc:224] Handling call
0709 12:27:50.041228 (+ 88907us) ts_tablet_manager.cc:934] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
0709 12:27:51.096279 (+1055051us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4c002a5780. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1573a20 00007f6fa1584a29 00007f6fa158513f 00007f6fa15492ab 00007f6fa1576f3c 00007f6fa1577461 00007f6fa1577768 00007f6fa72fd2eb 00007f6fa8222d3b 00007f6fa821d008 00007f6fa821c405
0709 12:27:51.238702 (+142423us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:27:52.082896 (+844194us) tablet_metadata.cc:665] Metadata flushed
0709 12:27:52.100704 (+ 17808us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:27:52.477117 (+376413us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:27:52.527329 (+ 50212us) tablet_replica.cc:206] Creating consensus instance
0709 12:27:52.527768 (+   439us) ts_tablet_manager.cc:989] Replica registered.
0709 12:27:52.527920 (+   152us) inbound_call.cc:177] Queueing success response
0709 12:27:56.914299 (+4386379us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:27:59.273222 (+2358923us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:27:59.496238 (+223016us) tablet_replica.cc:261] Starting instrumentation
0709 12:27:59.496933 (+   695us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":37490,"mutex_wait_us":33870,"raft.queue_time_us":8049,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":15449,"spinlock_wait_cycles":33111424,"tablet-copy.queue_time_us":81278,"tablet-download-pool.queue_time_us":6217,"tablet-download-pool.run_cpu_time_us":1717,"tablet-download-pool.run_wall_time_us":3432689,"thread_start_us":301105,"threads_started":2}
I20260709 12:28:01.132254 13827 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:01.158349 13886 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:28:01.135470 13683 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.055434 13654 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 12:28:01.103049 13862 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.279s	user 0.001s	sys 0.001s
I20260709 12:28:01.059088 13804 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:01.134336 13876 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.312s	user 0.000s	sys 0.001s
W20260709 12:28:01.132890 13881 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
W20260709 12:28:01.131970 13882 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.294s	user 0.000s	sys 0.001s
W20260709 12:28:01.133713 13887 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:28:01.131312 13863 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
I20260709 12:28:01.186444 12840 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.090404 13677 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:28:01.103101 13776 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.139102 13743 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.130889 13650 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.090253 13767 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:01.175875 13709 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.533s	user 0.008s	sys 0.003s
I20260709 12:28:01.197515  6381 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:01.198071  6309 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:01.198760  6377 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:01.200191  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:01.138616 13828 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.137239 13848 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:01.054679 13693 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.263849  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.177313 13866 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.190363 13802 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.185066 13824 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:01.101739 13754 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.080670 13864 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:01.287192  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 301568us
I20260709 12:28:01.197748 13873 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:01.354702 13856 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:01.260043 13630 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.263751 13895 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.293008 13683 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.347620 13885 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
W20260709 12:28:01.348141  7373 heartbeater.cc:646] Failed to heartbeat to 127.5.4.126:35813 (6 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.4.126:35813 timed out after 15.000s (SENT)
I20260709 12:28:01.348439 13841 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:28:01.297421 13854 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.003s
I20260709 12:28:01.266973 13090 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.266731 13790 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.363128  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:01.366324  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 167625us
I20260709 12:28:01.260012 13870 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.377434 13875 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.446775 13875 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.447701 13693 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.390021  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3355) took 1477 ms. Trace:
I20260709 12:28:01.393901 13904 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.395030  6384 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:01.449404 13875 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.378324 13802 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.377812 13828 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:28:01.425043  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.425213  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3308) took 1324 ms. Trace:
I20260709 12:28:01.438961  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3309) took 1332 ms. Trace:
I20260709 12:28:01.406755 13823 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.373003 13883 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.417449 13816 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.468807 13816 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.469151 13824 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.469789 13802 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.470366 13868 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.419945 13895 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.396642 13815 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.439465 13090 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.471186 13873 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.472697 13895 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.415869 13867 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:01.473538 13867 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.461012 13904 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.475924 13867 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.444098 13790 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.407464 13848 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.423717 13683 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.359678 13650 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.445652  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.501370  6457 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:01.447160  6309 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:28:01.359678 13749 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:01.371694 13709 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.451488  6382 rpcz_store.cc:276] 0709 12:27:59.912417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.912605 (+   188us) service_pool.cc:224] Handling call
0709 12:27:59.933261 (+ 20656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.089769 (+156508us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.089793 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.090178 (+   385us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.883094 (+792916us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:00.883148 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.883249 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.389768 (+506519us) raft_consensus.cc:1685] finished
0709 12:28:01.389793 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.389957 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33051,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"thread_start_us":894027,"threads_started":2,"wal-append.queue_time_us":609453}
I20260709 12:28:01.455848  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:01.459784  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1137) took 1519 ms. Trace:
W20260709 12:28:01.460044 13893 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:28:01.511266  6302 rpcz_store.cc:276] 0709 12:27:59.935177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.935803 (+   626us) service_pool.cc:224] Handling call
0709 12:27:59.936138 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.953903 (+ 17765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:59.953945 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.999439 (+ 45494us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.247402 (+247963us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:00.302310 (+ 54908us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.347059 (+ 44749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.347167 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.454589 (+1107422us) raft_consensus.cc:1685] finished
0709 12:28:01.454616 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.454757 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40766,"prepare.queue_time_us":75748,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":58487,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":286,"thread_start_us":161837,"threads_started":1,"wal-append.queue_time_us":657603}
W20260709 12:28:01.465234  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.357366 13743 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:01.515645  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60128.3us
I20260709 12:28:01.471208 13816 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.516427  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:01.443621 13629 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.461730  6450 rpcz_store.cc:276] 0709 12:28:00.094321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.094517 (+   196us) service_pool.cc:224] Handling call
0709 12:28:00.094879 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.333275 (+238396us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.333326 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.346211 (+ 12885us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.741635 (+395424us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:00.741718 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.741856 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.418591 (+676735us) raft_consensus.cc:1685] finished
0709 12:28:01.418621 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.418830 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80491,"raft.queue_time_us":175779,"raft.run_cpu_time_us":9092,"raft.run_wall_time_us":75701,"thread_start_us":399657,"threads_started":2,"wal-append.queue_time_us":380288}
I20260709 12:28:01.419994 13841 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:01.504781  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1144) took 1439 ms. Trace:
I20260709 12:28:01.447731 13870 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:01.464076 13889 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:28:01.377900 13866 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.460708 13908 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:01.506026 13754 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.523829  6310 rpcz_store.cc:276] 0709 12:28:00.065491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.116092 (+ 50601us) service_pool.cc:224] Handling call
0709 12:28:00.143426 (+ 27334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:00.459383 (+315957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.459419 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:00.459475 (+    56us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:00.459512 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.504399 (+1044887us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54002dfad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:01.504451 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.504692 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726057856}
I20260709 12:28:01.554782  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:01.555757  6382 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:01.523231  6384 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:01.529028  6457 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:01.531773  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3312) took 1421 ms. Trace:
I20260709 12:28:01.415310 13856 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:01.529870 13772 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.595s	user 0.015s	sys 0.002s
I20260709 12:28:01.461815  6454 rpcz_store.cc:276] 0709 12:28:00.106355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.106563 (+   208us) service_pool.cc:224] Handling call
0709 12:28:00.107080 (+   517us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.312954 (+205874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.312991 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.353722 (+ 40731us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.741265 (+387543us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:00.918263 (+176998us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.973478 (+ 55215us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.050449 (+ 76971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.050561 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.136614 (+ 86053us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b48000cccd8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:01.438670 (+302056us) raft_consensus.cc:1685] finished
0709 12:28:01.438711 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.438886 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":290829,"prepare.queue_time_us":351330,"raft.queue_time_us":380627,"raft.run_cpu_time_us":2902,"raft.run_wall_time_us":112623,"spinlock_wait_cycles":24447616,"thread_start_us":477281,"threads_started":3,"wal-append.queue_time_us":460139}
I20260709 12:28:01.518723 13790 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:01.514709 13823 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:01.560266  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.568743  6450 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:01.568953  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:01.569900  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.570598  6303 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:28:01.562260  6379 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: []
W20260709 12:28:01.549139  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.001s	sys 0.022s
I20260709 12:28:01.571547  6308 raft_consensus.cc:2368] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:01.530081 13629 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:01.560043  6458 rpcz_store.cc:276] 0709 12:28:00.110338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.110501 (+   163us) service_pool.cc:224] Handling call
0709 12:28:00.110940 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:01.010146 (+899206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.010177 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:01.010773 (+   596us) log.cc:844] Serialized 52 byte log entry
0709 12:28:01.360987 (+350214us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:01.514057 (+153070us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.531404 (+ 17347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.531495 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.531542 (+    47us) raft_consensus.cc:1685] finished
0709 12:28:01.531557 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.531712 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102477,"raft.queue_time_us":357885,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":74763,"thread_start_us":1271832,"threads_started":3,"wal-append.queue_time_us":402487}
I20260709 12:28:01.595741  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:01.553360 13713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.793s	user 0.036s	sys 0.018s
I20260709 12:28:01.530090 13867 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:01.606738 13629 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.608121 13709 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.608996 13754 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.609660 13754 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.571635  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.566406  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.574759 13841 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.533186 13908 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:01.576781 13743 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.576926 13848 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.532580 13815 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:28:01.577111  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.614517 13908 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.615159 13841 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:01.615566  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.615952 13908 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:01.616611  6454 raft_consensus.cc:2368] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:01.617663  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.578363 13866 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:01.580741 13892 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.224s	user 0.003s	sys 0.000s
I20260709 12:28:01.580556  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:01.620083 13856 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.620518  6382 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:01.581173 13890 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:28:01.584508  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:01.622153  6307 raft_consensus.cc:2368] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:01.585749 13767 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.512899 13749 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.587325 13823 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.567800 13804 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.589102  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.629777  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:01.589826 13903 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.632030  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.631888 13903 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.632844 13749 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.629683 13848 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.636144 13848 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.591486  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.639074  6452 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:01.553275 13677 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:01.640056 13677 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.641687 13840 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:01.567281 13772 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.626724 13867 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:01.643139 13867 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:01.643868 13850 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.293s	user 0.000s	sys 0.002s
I20260709 12:28:01.644150 13772 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.644691  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3238) took 1563 ms. Trace:
I20260709 12:28:01.594874  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:01.604660 13650 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.645219  6305 rpcz_store.cc:276] 0709 12:28:00.081676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.115106 (+ 33430us) service_pool.cc:224] Handling call
0709 12:28:00.115433 (+   327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.115799 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:00.550809 (+435010us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.866065 (+315256us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.866792 (+   727us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.882290 (+ 15498us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:00.882666 (+   376us) log.cc:844] Serialized 14 byte log entry
0709 12:28:00.882894 (+   228us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.882986 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.644387 (+761401us) raft_consensus.cc:1685] finished
0709 12:28:01.644417 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.644613 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44830,"prepare.queue_time_us":126991,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":108,"raft.queue_time_us":23183,"raft.run_cpu_time_us":561,"raft.run_wall_time_us":58720,"thread_start_us":704680,"threads_started":2,"wal-append.queue_time_us":709834}
I20260709 12:28:01.606148 13683 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.609097 13629 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.612069  6457 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:01.571899  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.001s	sys 0.023s
I20260709 12:28:01.612100  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.621871  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.625875  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:01.612347 13713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.853s	user 0.037s	sys 0.018s
I20260709 12:28:01.580606 13904 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:01.630185  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:01.630291  6301 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:01.631171  6348 leader_election.cc:400] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:01.638842 13823 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.646884  6310 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:01.601358  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.634197 13903 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.653026  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:01.642279 13677 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.628454 13755 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.657637  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3311) took 1547 ms. Trace:
I20260709 12:28:01.670604  6458 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:01.642266 13840 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:01.656428  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 607773us
I20260709 12:28:01.678301  6348 leader_election.cc:403] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:28:01.673822  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.645280 13867 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:01.679778  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:01.802439 13848 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.738096 13713 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:01.808491 13868 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:01.809237 13713 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:01.809656  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.810747  6308 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:01.810329  6454 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
W20260709 12:28:01.809533  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 180338us
W20260709 12:28:01.818881 13814 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.006s	sys 0.019s
W20260709 12:28:01.845073 13814 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.021s
I20260709 12:28:01.696300  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3232) took 2011 ms. Trace:
I20260709 12:28:01.811580 13713 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.707607 13754 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:01.846467  6300 rpcz_store.cc:276] 0709 12:27:59.684434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.704422 (+ 19988us) service_pool.cc:224] Handling call
0709 12:27:59.704853 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.705152 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:59.705192 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.713616 (+  8424us) log.cc:844] Serialized 26 byte log entry
0709 12:27:59.935829 (+222213us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:27:59.935863 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:59.935923 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.642951 (+1707028us) spinlock_profiling.cc:243] Waited 62.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:01.673580 (+ 30629us) raft_consensus.cc:1685] finished
0709 12:28:01.673610 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.696192 (+ 22582us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143637888,"thread_start_us":186346,"threads_started":1,"wal-append.queue_time_us":498446}
I20260709 12:28:01.837566  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.849766 13840 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:01.850562  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:01.817641  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 205102us
I20260709 12:28:01.851570 13903 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:01.811787  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.681738  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.915879  6450 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
W20260709 12:28:01.855598 13853 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.606s	user 0.003s	sys 0.000s
I20260709 12:28:01.810011  6453 rpcz_store.cc:276] 0709 12:28:00.109996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.110171 (+   175us) service_pool.cc:224] Handling call
0709 12:28:00.138878 (+ 28707us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.551654 (+412776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.551712 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.564118 (+ 12406us) log.cc:844] Serialized 26 byte log entry
0709 12:28:00.871125 (+307007us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:00.871259 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.871555 (+   296us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.657422 (+785867us) raft_consensus.cc:1685] finished
0709 12:28:01.657447 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.657576 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128915,"raft.queue_time_us":333800,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":66786,"thread_start_us":523919,"threads_started":2,"wal-append.queue_time_us":701807}
I20260709 12:28:01.893862  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1317) took 1112 ms. Trace:
I20260709 12:28:01.897083  6382 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:01.846167 13755 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:01.898797  6377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3346) took 2199 ms (client timeout 1982 ms). Trace:
I20260709 12:28:01.926527  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.927695  6308 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:28:01.707235  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:01.928318  6383 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:01.906508  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.925948  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.894956 13840 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:01.929452  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3623) took 1216 ms. Trace:
I20260709 12:28:01.942890  7314 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:01.918419  7316 rpcz_store.cc:276] 0709 12:28:00.781382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.781579 (+   197us) service_pool.cc:224] Handling call
0709 12:28:00.837760 (+ 56181us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:01.413253 (+575493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.413550 (+   297us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:01.414388 (+   838us) log.cc:844] Serialized 52 byte log entry
0709 12:28:01.634515 (+220127us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:01.634555 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.634619 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.893550 (+258931us) raft_consensus.cc:1685] finished
0709 12:28:01.893580 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.893781 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":348328,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":9880,"thread_start_us":498871,"threads_started":2,"wal-append.queue_time_us":416521}
I20260709 12:28:01.941401  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:01.926782  6377 rpcz_store.cc:269] 0709 12:27:59.699344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.699462 (+   118us) service_pool.cc:224] Handling call
0709 12:28:01.864805 (+2165343us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540022ef54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:01.898684 (+ 33879us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3354277632}
I20260709 12:28:01.951198  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.975943  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:01.952705  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:01.942039  6458 rpcz_store.cc:276] 0709 12:28:00.713133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.857296 (+1144163us) service_pool.cc:224] Handling call
0709 12:28:01.929378 (+ 72082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:01.979079  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:01.981662  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3350) took 2267 ms. Trace:
I20260709 12:28:01.982331  6385 rpcz_store.cc:276] 0709 12:27:59.710577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.710792 (+   215us) service_pool.cc:224] Handling call
0709 12:27:59.735216 (+ 24424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:59.858673 (+123457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:59.858709 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:59.867361 (+  8652us) log.cc:844] Serialized 26 byte log entry
0709 12:27:59.938657 (+ 71296us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:59.938709 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:00.441793 (+503084us) spinlock_profiling.cc:243] Waited 96.9 ms on lock 0x7b54003d40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:00.441857 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.854936 (+1413079us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:01.967828 (+112892us) raft_consensus.cc:1685] finished
0709 12:28:01.967852 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.977817 (+  9965us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5167,"raft.queue_time_us":16734,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":11358,"spinlock_wait_cycles":232196608,"thread_start_us":29080,"threads_started":1,"wal-append.queue_time_us":315416}
I20260709 12:28:01.977803  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:01.985306  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:01.985332  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.990157  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:01.991174  7331 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:01.991966  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:01.992163  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:01.991986  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:01.995004  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:01.996346  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:01.998672 13920 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.195s	user 0.002s	sys 0.000s
W20260709 12:28:02.000135 13897 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.852s	user 0.011s	sys 0.002s
I20260709 12:28:02.002024 13748 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.003029  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.003679  6378 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:01.993057  6454 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:28:02.004107  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1102) took 1304 ms. Trace:
I20260709 12:28:02.004465  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3077) took 1306 ms. Trace:
I20260709 12:28:02.005053  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.004953  6379 rpcz_store.cc:276] 0709 12:28:00.698055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.698546 (+   491us) service_pool.cc:224] Handling call
0709 12:28:00.714566 (+ 16020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.571513 (+856947us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.004244 (+432731us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b540023f5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:02.004402 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882660608}
I20260709 12:28:02.005697  6303 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:02.006034  6385 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:28:02.008195  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:01.987672 13823 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:01.940832 13903 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.004575  6378 rpcz_store.cc:276] 0709 12:28:00.699954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.002779 (+1302825us) service_pool.cc:224] Handling call
0709 12:28:02.004057 (+  1278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.010428  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.010569  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.011375  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.011318  7329 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:02.013692  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:02.018117  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.013823  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1151) took 1270 ms. Trace:
I20260709 12:28:02.021540  7329 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:28:02.024356  7314 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:02.025624  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.026196 13841 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:28:02.026866  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.027544  6379 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:28:02.028406  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.028915  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:02.029739  6377 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:02.031031 13903 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.031500  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.032586  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.033592  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.020918 13683 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.035038  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.013239 13845 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:02.036329 13845 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:02.037209 13845 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.038270 13632 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.039294  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.040331  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.040637  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:02.040840  6382 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:02.041193  6457 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:02.041667  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.025668  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.020263  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:02.035917  7335 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:02.045352  6457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "34bae875d7104f298673c223da6caf2e"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:02.042701  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:02.046552 13841 consensus_queue.cc:1243] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:28:02.047194 13841 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:02.047567 13841 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:02.047753 13814 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:02.047950  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:02.048214 13814 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.048629 13814 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:02.028925  6379 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:02.048969 13849 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:02.051123 13849 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.049358  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.021420  6304 rpcz_store.cc:276] 0709 12:28:00.743557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.744040 (+   483us) service_pool.cc:224] Handling call
0709 12:28:00.744466 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.821217 (+ 76751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.822636 (+  1419us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.840420 (+ 17784us) log.cc:844] Serialized 52 byte log entry
0709 12:28:01.049044 (+208624us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:01.093323 (+ 44279us) log.cc:844] Serialized 14 byte log entry
0709 12:28:01.095108 (+  1785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.095308 (+   200us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.999460 (+904152us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:02.007093 (+  7633us) raft_consensus.cc:1685] finished
0709 12:28:02.013579 (+  6486us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.013747 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27847,"prepare.queue_time_us":30293,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":97373,"raft.run_cpu_time_us":387,"raft.run_wall_time_us":22894,"spinlock_wait_cycles":19906944,"thread_start_us":164845,"threads_started":1,"wal-append.queue_time_us":223937}
I20260709 12:28:02.052702  6377 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:02.051991  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.052239  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.053836  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:02.048470 13841 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.053185  6379 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:02.056067  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3364) took 1011 ms. Trace:
I20260709 12:28:02.056437  6379 rpcz_store.cc:276] 0709 12:28:01.044842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.051372 (+1006530us) service_pool.cc:224] Handling call
0709 12:28:02.056016 (+  4644us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.057508  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.057864  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.015465 13888 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.060151  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.051540 13849 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:02.035173 13683 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.012266 13823 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:02.059538  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:02.061743  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.062386  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:02.059715  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.068604  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1100) took 1369 ms. Trace:
I20260709 12:28:02.069007  6377 rpcz_store.cc:276] 0709 12:28:00.699313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.067787 (+1368474us) service_pool.cc:224] Handling call
0709 12:28:02.068092 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.068261 (+   169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.068289 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.068356 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:02.068394 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.068450 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.068546 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.069928  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:02.070231  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.072898  6377 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:02.006583 13748 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.056177 13903 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:02.082379  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:28:02.059188 13632 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:02.083382  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 409633us
W20260709 12:28:02.089588  6381 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1104) took 1389 ms (client timeout 1695 ms). Trace:
I20260709 12:28:02.093317  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3078) took 1395 ms. Trace:
I20260709 12:28:02.067557 13823 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:02.090215  6381 rpcz_store.cc:269] 0709 12:28:00.700395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.051710 (+351315us) service_pool.cc:224] Handling call
0709 12:28:02.089365 (+1037655us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:02.089506 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":249053952}
I20260709 12:28:02.087770  6383 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:02.096153  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.074203 13822 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:02.076855 13908 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:28:02.093788  6380 rpcz_store.cc:276] 0709 12:28:00.698312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.698452 (+   140us) service_pool.cc:224] Handling call
0709 12:28:00.698789 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.073944 (+375155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.088794 (+1014850us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b5400195ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:02.093247 (+  4453us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":838675840}
I20260709 12:28:02.078701 13841 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:02.067698 13888 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.108745 13683 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:02.109598  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:02.063443 13755 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:28:02.066556 13868 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:28:02.111593  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3315) took 1402 ms. Trace:
I20260709 12:28:02.126349  6454 rpcz_store.cc:276] 0709 12:28:00.708877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.088320 (+1379443us) service_pool.cc:224] Handling call
0709 12:28:02.110994 (+ 22674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.111219 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.111247 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.111296 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:28:02.111325 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.111380 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.111517 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.098618 13822 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.138672 13822 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:02.142246  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.129643  6206 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 8 to 11, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 11 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:02.143378 13888 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.152000 13748 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:02.153596  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:02.116710  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:02.117251  6309 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1153) took 1358 ms (client timeout 1737 ms). Trace:
W20260709 12:28:02.154026  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44520.1us
W20260709 12:28:02.155359  6309 rpcz_store.cc:269] 0709 12:28:00.758738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.759733 (+   995us) service_pool.cc:224] Handling call
0709 12:28:02.116993 (+1357260us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54002df5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:02.117174 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718194560}
I20260709 12:28:02.118979  6377 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:28:02.076933 13629 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:28:02.086207 13632 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.128391  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.115581 13908 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:02.159487  7332 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:02.145892  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:02.154428  6454 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:28:02.171594  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.179371  6383 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:02.171169  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1106) took 1463 ms. Trace:
I20260709 12:28:02.211556  6378 rpcz_store.cc:276] 0709 12:28:00.708083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.170026 (+1461943us) service_pool.cc:224] Handling call
0709 12:28:02.170478 (+   452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.170739 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.170777 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.170838 (+    61us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:02.170884 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.170967 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.171095 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:02.240211  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71130.4us
I20260709 12:28:02.173831 13864 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:02.246742  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.180647  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:02.248203  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67673.8us
I20260709 12:28:02.248999  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:02.162707 13712 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:02.242889  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:02.167905 13908 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:02.163389 13632 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:02.163506 13866 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.268677  6351 leader_election.cc:400] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:02.269224  6351 leader_election.cc:403] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:28:02.287966  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.255802 13840 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.302147  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3368) took 1256 ms. Trace:
I20260709 12:28:02.259644 13683 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:02.285771 13848 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.142475 13823 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.266386 13888 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.325255 13840 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.325485 13888 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.327581 13840 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.328176 13840 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:02.329645 13825 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.330364 13825 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.259289 13867 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.317584  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.320219  6351 leader_election.cc:400] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:02.271821 13748 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:02.287237 13854 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:02.316578  6378 rpcz_store.cc:276] 0709 12:28:01.046085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.253962 (+1207877us) service_pool.cc:224] Handling call
0709 12:28:02.254447 (+   485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.300343 (+ 45896us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:02.300394 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.300462 (+    68us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:02.300511 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.301853 (+  1342us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.302064 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45311,"raft.queue_time_us":45467,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":9036}
I20260709 12:28:02.290799 13846 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.338449  6380 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:02.347836  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.350935 13823 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:02.351667  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.352331  7332 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:02.353158  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.353670  7332 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:02.354395  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.355383 13523 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:02.355506  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.357816  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.341445 13848 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.364440 13848 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.346879 13814 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.378343 13814 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.372896  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3616) took 1728 ms. Trace:
I20260709 12:28:02.376633  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1103) took 1676 ms. Trace:
I20260709 12:28:02.378544  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3622) took 1665 ms. Trace:
I20260709 12:28:02.376515  6376 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:02.388522  6376 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:02.382174  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.328198 13888 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.368023  7318 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:02.391219 13672 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:02.266738 13864 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.385193  6377 rpcz_store.cc:276] 0709 12:28:00.700166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.074702 (+1374536us) service_pool.cc:224] Handling call
0709 12:28:02.075054 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.157254 (+ 82200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.376313 (+219059us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b540010a4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:02.376565 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100201344}
I20260709 12:28:02.391767  6458 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:02.386401  6459 rpcz_store.cc:276] 0709 12:28:00.712866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.377523 (+1664657us) service_pool.cc:224] Handling call
0709 12:28:02.377917 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.378130 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.378163 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.378221 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:02.378260 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.378327 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.378450 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.393711 13090 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.399271  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.399971  6457 raft_consensus.cc:606] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:02.399744 13090 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.400367  6457 raft_consensus.cc:614] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:02.348953  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:02.385040  6456 rpcz_store.cc:276] 0709 12:28:00.644262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.644369 (+   107us) service_pool.cc:224] Handling call
0709 12:28:00.644816 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.080584 (+435768us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.080620 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:01.101571 (+ 20951us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.351273 (+249702us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:01.351338 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.372367 (+1021029us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54000eb854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:02.372428 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.372495 (+    67us) raft_consensus.cc:1685] finished
0709 12:28:02.372511 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.372815 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6207,"raft.queue_time_us":6772,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":11138,"spinlock_wait_cycles":986842368,"thread_start_us":90352,"threads_started":1,"wal-append.queue_time_us":266086}
I20260709 12:28:02.339900 13825 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.410269 13710 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:02.398286 13846 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.350102 13748 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.410791 13841 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:02.412134  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.442710  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:02.412998 13930 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:28:02.348008 13867 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.398269 13221 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 12:28:02.310855 13866 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.415199 13848 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:02.415421 13823 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.415218 13849 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.416118 13904 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.416602 13342 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:02.417546 13822 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.421257 13824 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.402343  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.422479  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:02.423130 13864 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:02.423367 13931 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 12:28:02.423552  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.490156  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.424990  6303 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:02.426786  6304 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:28:02.426796  6384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3354) took 2539 ms (client timeout 2432 ms). Trace:
I20260709 12:28:02.427776  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3317) took 1718 ms. Trace:
W20260709 12:28:02.492880  6384 rpcz_store.cc:269] 0709 12:27:59.887824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.887996 (+   172us) service_pool.cc:224] Handling call
0709 12:28:02.426464 (+2538468us) spinlock_profiling.cc:243] Waited 1.49 s on lock 0x7b54001f54d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:02.426682 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3420475776}
I20260709 12:28:02.428654  6300 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 12:28:02.430797  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.430737 13090 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.495041  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.493033  6454 rpcz_store.cc:276] 0709 12:28:00.709627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.255880 (+1546253us) service_pool.cc:224] Handling call
0709 12:28:02.286048 (+ 30168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.328582 (+ 42534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.328627 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.328682 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:02.328728 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.427455 (+ 98727us) spinlock_profiling.cc:243] Waited 42.3 ms on lock 0x7b5400344254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:02.427523 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.427698 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97264896}
W20260709 12:28:02.404098  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55244.6us
I20260709 12:28:02.431895  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.435801  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:02.444195  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55986.8us
W20260709 12:28:02.498172  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62493.2us
I20260709 12:28:02.451025  6377 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:28:02.499060  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:02.501729  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.502887  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.469615  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3630) took 1460 ms. Trace:
I20260709 12:28:02.503746  6459 rpcz_store.cc:276] 0709 12:28:01.009270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.399445 (+1390175us) service_pool.cc:224] Handling call
0709 12:28:02.468767 (+ 69322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.469196 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.469235 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:02.469289 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:02.469333 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.469406 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.469548 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2872,"raft.queue_time_us":1657,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":7407}
I20260709 12:28:02.471035  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.429508 13677 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.443420 13710 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.466885 13748 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:02.487062  6376 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:28:02.507961  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.517091  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.508769  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.509931  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:02.517292  6305 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:02.522248  6304 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:28:02.522471  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:02.510303  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.511642  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:02.528918 13090 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:02.529410 13090 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:02.529827 13090 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:02.511431  6454 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:02.530941  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3323) took 1019 ms. Trace:
I20260709 12:28:02.531402  6454 rpcz_store.cc:276] 0709 12:28:01.511691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.510577 (+998886us) service_pool.cc:224] Handling call
0709 12:28:02.511002 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.530567 (+ 19565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.530850 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.489728 13791 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:02.537858  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.539337  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.541450  6459 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:28:02.541543  6303 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:02.542133  6459 pending_rounds.cc:85] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:28:02.542963  6459 pending_rounds.cc:107] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:28:02.543291  6459 raft_consensus.cc:2889] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:02.544135  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3324) took 1028 ms. Trace:
I20260709 12:28:02.544548  6459 rpcz_store.cc:276] 0709 12:28:01.515797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.512385 (+996588us) service_pool.cc:224] Handling call
0709 12:28:02.540990 (+ 28605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.543818 (+  2828us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.544068 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:02.516362 13710 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:02.484743 13904 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.546890  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.549041  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:02.558427  6454 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:02.559700  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:02.510877 13845 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:02.570231  6301 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:02.483487 13823 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.515408 13825 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:02.573709  6305 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:02.577972  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:02.580231  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.584357  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.519505 13258 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:02.586270  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.589284  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:02.597226  6459 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:02.599354  6307 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:28:02.594129  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:02.617497  6206 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 15 to 16, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 16 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:02.620138  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.548280 13915 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:02.621245  6309 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:02.621196  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:02.540653 13791 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:02.500849 13824 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:02.607770 13937 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:28:02.569780 13755 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.596580 13655 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:02.516558 13677 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:02.634563  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 370301us
I20260709 12:28:02.610831 13845 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:02.610379 13713 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:02.634192 13616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.019s	sys 0.015s
I20260709 12:28:02.626729 13915 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.674172  6379 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:28:02.648347 13090 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:02.624248 13715 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.020s	sys 0.014s
I20260709 12:28:02.539873 13632 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:02.668182 13616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.020s	sys 0.015s
I20260709 12:28:02.656911  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:02.713856  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57034.9us
I20260709 12:28:02.657917  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:02.663174  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.709950  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1099) took 2011 ms. Trace:
I20260709 12:28:02.612666 13629 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:02.714691  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:02.758180  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:02.759320  6454 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:02.760488  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:02.773283 13090 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:02.686022 13715 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.021s	sys 0.014s
I20260709 12:28:02.782403  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:02.716066  6382 rpcz_store.cc:276] 0709 12:28:00.699022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.064879 (+1365857us) service_pool.cc:224] Handling call
0709 12:28:02.065175 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.075771 (+ 10596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.075823 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.079924 (+  4101us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.481921 (+401997us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:02.505653 (+ 23732us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.513925 (+  8272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.514019 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.709646 (+195627us) raft_consensus.cc:1685] finished
0709 12:28:02.709684 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.709871 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16432,"prepare.queue_time_us":22734,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":22026,"raft.queue_time_us":18026,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":5987,"thread_start_us":361229,"threads_started":1,"wal-append.queue_time_us":579079}
I20260709 12:28:02.789059  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.791903  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:02.679090 13713 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:02.686218 13655 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:02.810801  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:02.778326 13761 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:02.796732  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:02.672694 13915 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:02.771214 13790 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:02.728911 13629 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.778614 13848 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:02.655264 13755 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.769069 13884 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.721308 13841 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:02.780148 13846 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.813027  6384 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:02.814337 13761 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.874058  6382 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
W20260709 12:28:02.839468 13908 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.001s
I20260709 12:28:02.826211 13790 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.843889 13629 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:02.714437 13791 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.851649 13866 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.895386  6382 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
W20260709 12:28:02.881403  7332 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.002s	sys 0.019s
I20260709 12:28:02.870158 13867 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.855312 13845 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:02.971093 13523 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.615s	user 0.007s	sys 0.007s
W20260709 12:28:02.972218 13854 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.004s
I20260709 12:28:02.972651 13854 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:02.973526  6451 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:02.860898 13848 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:02.958801 13937 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:28:02.898602 13790 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:02.976267 13848 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.869451 13884 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.980715 13884 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:02.871794 13841 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.983266 13841 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.983645 13755 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:02.952908 13822 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.987870  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3373) took 1597 ms. Trace:
I20260709 12:28:02.988300  6378 rpcz_store.cc:276] 0709 12:28:01.390885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.348981 (+958096us) service_pool.cc:224] Handling call
0709 12:28:02.349435 (+   454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.349839 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.349870 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.356447 (+  6577us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.720988 (+364541us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:02.721047 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.721131 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:02.987636 (+266505us) raft_consensus.cc:1685] finished
0709 12:28:02.987660 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:02.987807 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17318,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":117,"thread_start_us":364098,"threads_started":1,"wal-append.queue_time_us":595214}
I20260709 12:28:02.991016 13854 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.991593 13854 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:02.896885 13908 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:02.993135  7314 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 13. Current term is 14. Ops: []
I20260709 12:28:02.993755 13854 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:02.994141 13908 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:02.877354 13761 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:02.974277 13873 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.997709 13823 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:02.998181 13713 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.001097 13655 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:03.004171 13948 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:28:03.005424  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:02.958673  7332 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.006s	sys 0.023s
I20260709 12:28:03.010421 13904 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:03.011255  6380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.004s	sys 0.037s
W20260709 12:28:03.016477  6380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.004s	sys 0.037s
W20260709 12:28:03.018703  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 440800us
I20260709 12:28:03.033481  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:03.014150  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 489735us
I20260709 12:28:03.043699  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:03.012735 13873 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:03.054584  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:03.038038 13824 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:03.057850  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:03.045454 13823 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:03.063622  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:03.063815  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:03.059163  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:03.069485  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3371) took 1680 ms. Trace:
I20260709 12:28:03.069926  6383 rpcz_store.cc:276] 0709 12:28:01.389083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.239592 (+850509us) service_pool.cc:224] Handling call
0709 12:28:02.256852 (+ 17260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.257167 (+   315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.257209 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.265669 (+  8460us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.654937 (+389268us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:03.041824 (+386887us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.042147 (+   323us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.042255 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.069284 (+ 27029us) raft_consensus.cc:1685] finished
0709 12:28:03.069305 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.069430 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31760,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":138,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":5337,"thread_start_us":706256,"threads_started":2,"wal-append.queue_time_us":519636}
I20260709 12:28:03.078334  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:03.082258  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:03.058622 13715 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:03.086845  6307 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:03.088645  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1263) took 2466 ms. Trace:
I20260709 12:28:03.065812 13825 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:03.066025 13755 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:03.092509  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:03.089103  6455 rpcz_store.cc:276] 0709 12:28:00.618150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.621147 (+  2997us) service_pool.cc:224] Handling call
0709 12:28:00.621436 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.063802 (+442366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.063849 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:01.063906 (+    57us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:01.063951 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.083878 (+2019927us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54003c78d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:03.083951 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.084089 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016740608}
I20260709 12:28:03.052416 13822 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:28:03.069509 13632 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:03.093568  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:03.088809  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:03.086928 13824 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:03.103698  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-521c92260b1b4a6abba1ce9d3c7c5ad2 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:28:03.104636  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-521c92260b1b4a6abba1ce9d3c7c5ad2 on tablet 521c92260b1b4a6abba1ce9d3c7c5ad2 with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:03.106150 13710 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.537s	user 0.011s	sys 0.000s
I20260709 12:28:03.106632 13710 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.107191 13822 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:03.107899 13822 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:03.108829 13710 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:03.108983 13822 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.096294 13884 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:03.112185 13884 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.112983  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:03.088542 13904 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:03.093514 13867 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.305773  6308 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:03.338836  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:03.340764  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:03.095610 13755 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.341542  6455 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:03.382987  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:03.120956 13824 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:03.342329 13948 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.249s	user 0.000s	sys 0.000s
I20260709 12:28:03.131136 13632 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.114280 13746 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:03.417631  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:03.096922 13915 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.380393 13755 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:03.426182  6455 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:03.455413  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 363679us
I20260709 12:28:03.403668 13904 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:03.416664 13824 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:03.491189 13956 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.422s	user 0.001s	sys 0.001s
I20260709 12:28:03.492870  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2942) took 1062 ms. Trace:
W20260709 12:28:03.493256 13952 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.400s	user 0.000s	sys 0.000s
W20260709 12:28:03.494611 13955 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.404s	user 0.001s	sys 0.000s
I20260709 12:28:03.493455  6302 rpcz_store.cc:276] 0709 12:28:02.430358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.430504 (+   146us) service_pool.cc:224] Handling call
0709 12:28:02.449020 (+ 18516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.449305 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.449344 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.449805 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.897925 (+448120us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:02.897986 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.898083 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.492483 (+594400us) raft_consensus.cc:1685] finished
0709 12:28:03.492519 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.492771 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":428766,"threads_started":1,"wal-append.queue_time_us":596521}
I20260709 12:28:03.496043  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3384) took 1387 ms. Trace:
W20260709 12:28:03.496295 13957 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.419s	user 0.000s	sys 0.000s
I20260709 12:28:03.497888  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3374) took 2106 ms. Trace:
I20260709 12:28:03.498309  6380 rpcz_store.cc:276] 0709 12:28:01.391200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.360180 (+968980us) service_pool.cc:224] Handling call
0709 12:28:02.360511 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:02.367320 (+  6809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.367348 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:02.401952 (+ 34604us) log.cc:844] Serialized 52 byte log entry
0709 12:28:03.016909 (+614957us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:03.016950 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.017030 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.497631 (+480601us) raft_consensus.cc:1685] finished
0709 12:28:03.497660 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.497823 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6333,"raft.queue_time_us":6633,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":76,"thread_start_us":538855,"threads_started":1,"wal-append.queue_time_us":596477}
I20260709 12:28:03.465258  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:03.496552  6381 rpcz_store.cc:276] 0709 12:28:02.108681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.417514 (+308833us) service_pool.cc:224] Handling call
0709 12:28:02.417895 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.418234 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.418266 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.424161 (+  5895us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.855218 (+431057us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:02.855260 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.855325 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.495758 (+640433us) raft_consensus.cc:1685] finished
0709 12:28:03.495786 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.495973 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5348,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":104,"thread_start_us":430612,"threads_started":1,"wal-append.queue_time_us":633720}
I20260709 12:28:03.506011  6454 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:03.507725  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:03.519456 13966 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:28:03.404134 13867 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.553315 13841 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.421422 13761 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:03.559489 13965 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:28:03.429234 13632 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.581276  6450 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:03.572266 13841 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:03.609814 13972 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:28:03.626674 13977 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:28:03.632264 13841 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:03.637041 13790 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.662s	user 0.010s	sys 0.001s
I20260709 12:28:03.637815 13915 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:03.598963 13904 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:03.640682 13629 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.002s
I20260709 12:28:03.699142 13629 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.642024 13867 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:03.655773 13632 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:03.658846 13761 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:03.620272 13884 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:03.666882 13748 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.128s	user 0.012s	sys 0.002s
I20260709 12:28:03.703622 13748 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.705026  6385 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:03.709654 13748 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:03.713549 13790 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:03.739132  6454 raft_consensus.cc:2368] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:03.741849 13790 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:03.747707  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 287249us
I20260709 12:28:03.750111  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:03.734395 13629 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:03.759781 13713 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.014s	sys 0.027s
I20260709 12:28:03.770952  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:03.773381 13873 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:03.457722  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.515s (SENT) [suppressed 16 similar messages]
I20260709 12:28:03.777943  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:03.463701  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:03.814841  6455 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:03.791550  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:03.783849 13755 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.784945  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:03.728379 13824 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:03.788569 13814 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:03.791044  6452 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:03.816187  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:03.998365  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 182291us
I20260709 12:28:03.999774  6272 leader_election.cc:400] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:04.000190  6272 leader_election.cc:403] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:03.819505  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.001928  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:03.813661 13713 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.014s	sys 0.027s
I20260709 12:28:04.003041  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:03.817974 13258 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:04.013556  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:03.824380 13974 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:04.026527  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3341) took 1270 ms. Trace:
W20260709 12:28:04.011158 13726 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:28:04.033373  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3338) took 1278 ms. Trace:
I20260709 12:28:04.008562 13840 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:04.034053  6459 rpcz_store.cc:276] 0709 12:28:02.755065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.816313 (+ 61248us) service_pool.cc:224] Handling call
0709 12:28:02.816741 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.985002 (+168261us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:03.077573 (+ 92571us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.426855 (+349282us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.426949 (+    94us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:03.427016 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.032967 (+605951us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54002c9554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:04.033047 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.033276 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39843,"prepare.queue_time_us":124870,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":45342,"spinlock_wait_cycles":444521472,"thread_start_us":400727,"threads_started":2,"wal-append.queue_time_us":615741}
W20260709 12:28:04.047945  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48395us
I20260709 12:28:04.049283  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:04.027118  6452 rpcz_store.cc:276] 0709 12:28:02.755752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.025398 (+1269646us) service_pool.cc:224] Handling call
0709 12:28:04.025828 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.026067 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.026106 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.026167 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:04.026217 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.026293 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.026436 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.010864 13629 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:04.008273  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.017383 13258 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.052023 13904 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.008096 13814 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.023594 13974 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.059993 13629 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:04.096313  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47164.4us
I20260709 12:28:04.097497  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:04.067267 13258 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:04.118472  6454 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:04.069881 13974 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:04.095299 13845 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:04.103475 13629 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.132426 13915 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.148341  6455 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:04.154934  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3636) took 1058 ms. Trace:
I20260709 12:28:04.155335  6455 rpcz_store.cc:276] 0709 12:28:03.096135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.463013 (+366878us) service_pool.cc:224] Handling call
0709 12:28:04.154870 (+691857us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.156029 13908 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.156878  6455 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:04.156798 13908 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.157729  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1276) took 1073 ms. Trace:
I20260709 12:28:04.158170  6455 rpcz_store.cc:276] 0709 12:28:03.084626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.156177 (+1071551us) service_pool.cc:224] Handling call
0709 12:28:04.156510 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.157454 (+   944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.157674 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.160733 13342 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:04.161449  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1277) took 1076 ms. Trace:
I20260709 12:28:04.161357 13342 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.161942  6455 rpcz_store.cc:276] 0709 12:28:03.084976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.160649 (+1075673us) service_pool.cc:224] Handling call
0709 12:28:04.160948 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.161132 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.161159 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.161202 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:04.161236 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.161289 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.161397 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.162688 13776 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:04.163833 13221 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:04.164399 13221 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.164925 13776 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.165395 13776 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.161305  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1106) took 2012 ms. Trace:
I20260709 12:28:04.166749 13772 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.167616 13772 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.168203 13342 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.168882  7332 rpcz_store.cc:276] 0709 12:28:02.148690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.358737 (+210047us) service_pool.cc:224] Handling call
0709 12:28:02.359077 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.359314 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:03.029013 (+669699us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.160950 (+1131937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.161005 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:04.161054 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.161123 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.161248 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1059713,"prepare.queue_time_us":61748,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":1066447,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":727788,"threads_started":2,"wal-append.queue_time_us":374173}
I20260709 12:28:04.170862  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1330) took 1840 ms. Trace:
I20260709 12:28:04.177647  7318 rpcz_store.cc:276] 0709 12:28:02.329933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.413580 (+ 83647us) service_pool.cc:224] Handling call
0709 12:28:02.413884 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.417492 (+  3608us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:02.780822 (+363330us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.158724 (+1377902us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.159445 (+   721us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.159661 (+   216us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:04.159713 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.159811 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.170581 (+ 10770us) raft_consensus.cc:1685] finished
0709 12:28:04.170606 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.170772 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1128377,"prepare.queue_time_us":3801,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":1135367,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":187,"thread_start_us":573178,"threads_started":2,"wal-append.queue_time_us":273915}
I20260709 12:28:04.171408 13221 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.170451 13772 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:04.187181  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 729457us
I20260709 12:28:04.149628  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3343) took 1393 ms. Trace:
I20260709 12:28:04.150916 13901 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:04.152176  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1321) took 2157 ms. Trace:
W20260709 12:28:04.150943  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 1.64353e+06us
I20260709 12:28:04.189468  6459 rpcz_store.cc:276] 0709 12:28:02.756210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.130391 (+1374181us) service_pool.cc:224] Handling call
0709 12:28:04.148339 (+ 17948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.148635 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.148675 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.148741 (+    66us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:04.149301 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.149390 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.149548 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.190572  7331 rpcz_store.cc:276] 0709 12:28:01.994921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.995068 (+   147us) service_pool.cc:224] Handling call
0709 12:28:01.997290 (+  2222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:01.997502 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:02.256857 (+259355us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.151694 (+1894837us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.151761 (+    67us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:04.151818 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.151923 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.152065 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1496359,"prepare.queue_time_us":423140,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":1499106,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":62,"thread_start_us":657039,"threads_started":2,"wal-append.queue_time_us":727157,"wal-append.run_cpu_time_us":1229,"wal-append.run_wall_time_us":1001250}
I20260709 12:28:04.194864  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:04.185691 13915 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.200029  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.202667  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.208664  6459 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:28:04.210297  6425 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.897s (SENT)
W20260709 12:28:04.212217  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:04.212859 13936 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.816s	user 0.011s	sys 0.032s
I20260709 12:28:04.213402 13864 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:04.214054  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:04.215076 13683 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:04.214737 13873 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:04.216202 13873 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:04.216692 13873 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:04.209573  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3633) took 1114 ms. Trace:
I20260709 12:28:04.219866  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.220436  6459 rpcz_store.cc:276] 0709 12:28:03.095448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.193754 (+1098306us) service_pool.cc:224] Handling call
0709 12:28:04.205763 (+ 12009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.209186 (+  3423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.209490 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:04.190310 13901 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:04.221936  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.225736  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.222937  7318 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:04.201696 13342 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:04.201525 13755 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.236670  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1326) took 2120 ms. Trace:
I20260709 12:28:04.239287  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:04.217828 13873 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.222900 13824 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.207894 13916 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:04.233006  7318 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:28:04.225621 13864 consensus_queue.cc:1243] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:04.243016  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.214754  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:04.238231  7335 rpcz_store.cc:276] 0709 12:28:02.116718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.116866 (+   148us) service_pool.cc:224] Handling call
0709 12:28:02.128896 (+ 12030us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.129103 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:02.437215 (+308112us) log.cc:844] Serialized 14 byte log entry
0709 12:28:02.471515 (+ 34300us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.775868 (+1304353us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b18003d38a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba557 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:04.148672 (+372804us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b18003d38a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:04.236249 (+ 87577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.236309 (+    60us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:04.236365 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.236454 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.236601 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1681154,"prepare.queue_time_us":294432,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":1688970,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":7253,"spinlock_wait_cycles":20136960,"thread_start_us":394895,"threads_started":2}
I20260709 12:28:04.236549 13766 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:04.268765  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54089.6us
I20260709 12:28:04.270144  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.267015 13772 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:04.271488 13814 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:04.272107 13814 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:04.272722 13814 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:04.247489 13342 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.274024 13342 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.275374 13683 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:04.222149 13915 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.280272  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.281530  6309 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:04.282941  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:04.283270  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.283728  6309 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:04.284251  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.285751  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:04.287021  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.262151 13755 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.289860  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.290836  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.291769  6309 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:04.293299  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.293362  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.294150  6302 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:28:04.294906  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.277046 13873 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:04.269104 13864 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:04.301623  6303 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:04.281656 13824 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.289434 13790 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:04.308983  6309 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:04.307349  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.307060  6455 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:04.309535  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:04.313462  6302 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:28:04.313800  6305 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:04.276139 13683 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.294333 13908 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:04.315027  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43659.9us
I20260709 12:28:04.286212 13916 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:28:04.296023 13986 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:04.327093  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87943.3us
I20260709 12:28:04.328153  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:04.310297 13790 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:04.320964 13683 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.318738 13982 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:04.307444 13755 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.343509 13845 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:04.352557  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55385.9us
I20260709 12:28:04.312655 13824 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.324939 13986 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.354218 13988 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:04.422998  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 81147.5us
I20260709 12:28:04.371965  6305 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
W20260709 12:28:04.417464  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100344us
W20260709 12:28:04.392962  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66068.1us
I20260709 12:28:04.373574  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55284 (request call id 27) took 3295 ms. Trace:
I20260709 12:28:04.443097  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.444396  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:04.447016  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 79240.9us
I20260709 12:28:04.445799  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.443879  6457 rpcz_store.cc:276] 0709 12:28:01.079124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.045224 (+966100us) service_pool.cc:224] Handling call
0709 12:28:04.373320 (+2328096us) spinlock_profiling.cc:243] Waited 354 ms on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:04.373471 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":814219008}
W20260709 12:28:04.495434  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83211.4us
I20260709 12:28:04.496388  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:04.445230  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.406697 13752 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.501787  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:04.384075 13824 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.373641 13755 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.413640 13989 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:04.383409 13845 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.491829 13915 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.507369  6457 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:04.439899 13988 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:04.509693  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:04.413762 13908 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.094s
I20260709 12:28:04.500793 13916 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:04.439597 13986 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:04.420291 13705 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:28:04.481050 13974 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.534328 13710 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.421794 13790 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.470752 13629 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.510818 13989 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:04.520223 13258 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.518209 13988 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:04.548243  6307 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:04.521286 13908 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:04.526001  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3091) took 2090 ms. Trace:
W20260709 12:28:04.550371  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52282.1us
I20260709 12:28:04.537009 13983 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:28:04.529855  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3632) took 1434 ms. Trace:
I20260709 12:28:04.530820 13748 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:04.531170  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1269) took 2637 ms (client timeout 2068 ms). Trace:
W20260709 12:28:04.559273  6458 rpcz_store.cc:269] 0709 12:28:01.893531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.984966 (+ 91435us) service_pool.cc:224] Handling call
0709 12:28:04.530908 (+2545942us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54000c1cd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:04.531056 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":931173760}
I20260709 12:28:04.560613  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.561393  6458 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:04.531379 13655 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.531682 13916 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:04.531723 13903 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.532706  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:04.533493  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1262) took 3915 ms (client timeout 1686 ms). Trace:
I20260709 12:28:04.533856 13797 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:04.569591  6451 rpcz_store.cc:269] 0709 12:28:00.617770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.619242 (+  1472us) service_pool.cc:224] Handling call
0709 12:28:04.533297 (+3914055us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b5400343854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:04.533418 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5189146496}
I20260709 12:28:04.572882  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:04.534016 13999 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:04.514683 13984 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.175s	user 0.002s	sys 0.000s
I20260709 12:28:04.533563  6456 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:04.369125 13982 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.547129  6457 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:04.514328 13845 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.526923 13776 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.550710  6384 rpcz_store.cc:276] 0709 12:28:02.435182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.890388 (+455206us) service_pool.cc:224] Handling call
0709 12:28:02.891697 (+  1309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.892106 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:02.957223 (+ 65117us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.087862 (+130639us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.525366 (+1437504us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b1800410560. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:04.525567 (+   201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:04.525631 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:04.525691 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.525776 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.525928 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2155,"prepare.queue_time_us":66067,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":146,"raft.queue_time_us":1440095,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":4615,"spinlock_wait_cycles":10898688,"thread_start_us":129837,"threads_started":1,"wal-append.queue_time_us":146214,"wal-append.run_cpu_time_us":1652,"wal-append.run_wall_time_us":1013536}
I20260709 12:28:04.553493  6455 rpcz_store.cc:276] 0709 12:28:03.095177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.163025 (+1067848us) service_pool.cc:224] Handling call
0709 12:28:04.286145 (+123120us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.313326 (+ 27181us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.529554 (+216228us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b54001e19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:04.529780 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47252352}
I20260709 12:28:04.586889  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.557019 13849 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.536592 13683 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:04.563913  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:04.599723  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.600135 13776 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:04.600553  6458 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:28:04.601778  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:04.603236  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3085) took 3082 ms (client timeout 3083 ms). Trace:
W20260709 12:28:04.603798  6379 rpcz_store.cc:269] 0709 12:28:01.520569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.057217 (+536648us) service_pool.cc:224] Handling call
0709 12:28:04.602985 (+2545768us) spinlock_profiling.cc:243] Waited 616 ms on lock 0x7b5400038254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:04.603158 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1415693056}
I20260709 12:28:04.566301  6304 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:04.576316 13900 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:04.608587 13938 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.742s	user 0.005s	sys 0.011s
I20260709 12:28:04.567301 13655 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.559286 13790 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:04.573892  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:04.610828  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.611769  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.536428 13710 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.578927  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:04.581327  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.552433 13908 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:04.583315  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1035) took 2000 ms. Trace:
I20260709 12:28:04.617204  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.587518  6457 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:04.589625 13856 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:04.575748 13903 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.577554 13999 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:04.592751 13974 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:04.624840  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.558951 13748 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.593194 13989 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.585758 13726 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=1 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:04.594818  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:04.595737 13988 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.614358  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.578018 13797 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.626784 13999 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:04.637388 13797 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.615362  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.615463  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:04.618842  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:04.639454  7332 raft_consensus.cc:2410] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:04.639668  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.640789  6457 raft_consensus.cc:2410] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:28:04.623152 13755 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.617163  7320 rpcz_store.cc:276] 0709 12:28:02.583076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.583243 (+   167us) service_pool.cc:224] Handling call
0709 12:28:02.583641 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:04.151605 (+1567964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.151639 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:04.152165 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:28:04.263528 (+111363us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:04.263575 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.263656 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.583072 (+319416us) raft_consensus.cc:1685] finished
0709 12:28:04.583096 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.583249 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1567458,"raft.queue_time_us":1570019,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":126,"thread_start_us":110932,"threads_started":1,"wal-append.queue_time_us":146272}
I20260709 12:28:04.622910  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.619908  6458 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:28:04.625291  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.630811  6455 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:04.622624 13900 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:04.620087  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.627206 13991 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.655428 13655 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.629276 13915 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:04.661185  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.676750  6458 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:28:04.677441  6458 pending_rounds.cc:85] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 9
I20260709 12:28:04.677812  6458 pending_rounds.cc:107] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:28:04.678194  6458 raft_consensus.cc:2889] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:04.679628  6459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:04.680289  6459 raft_consensus.cc:606] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.687461 13903 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.688405 13748 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.689523 13851 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:04.649065 13873 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:04.717372  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 274333us
W20260709 12:28:04.720279  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 138840us
I20260709 12:28:04.723122  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:04.725397 14005 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:04.725945 14005 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.726809  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:04.727628  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.728299 14005 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.728554  7332 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.730002 13888 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:04.730531 13888 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:04.731012 13888 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.731434  7320 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:04.732604  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49402.8us
I20260709 12:28:04.734193  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.735862 13752 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:04.739210  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.739982  7316 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.744071  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.738346 13752 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:04.745471 13752 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.745904 13752 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:04.746074 13888 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.750460 13888 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.757555  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:04.758419 14014 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:04.759603  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.760320 14014 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:04.760547  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:04.760807 14014 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:04.761413 13888 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:04.761951 13888 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.762437 13888 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:04.763655 13994 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:04.764806 13994 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:04.765206 13994 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:04.766079 13994 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.767374 13994 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.762967 14005 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:04.771507 13888 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.763103  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.773860  7320 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:04.774457 14014 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:04.781674 14014 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.773945 13888 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:04.784538  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.785844  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:04.786131  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.786656  6382 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:04.786684 14014 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.787227  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.659484 13900 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.787716  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.787870  6378 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:04.675812 13754 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:04.788772  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.789283  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.790014  6384 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:04.786862  6380 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:04.792870  7316 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:04.763402  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.794035  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.788445  6382 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:04.794910  6384 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 19: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:04.795382  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:04.796065  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 115649us
I20260709 12:28:04.796648  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.797269  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:04.797252  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.685745 13797 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:04.799242  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.800009  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:04.799317  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.800416  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.805546  6378 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 25 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:28:04.796090  6382 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:04.807094  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:04.808367  7332 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:04.810536  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.811223  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:04.784303 13766 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.815304  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.816237  6384 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:28:04.659024 13755 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:04.799410 13904 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.725s	user 0.014s	sys 0.001s
I20260709 12:28:04.676522 13991 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.796339 13900 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:04.797447 13754 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.818164  6459 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:04.814467 13726 tablet_bootstrap.cc:492] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:28:04.813445 13987 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:04.827605  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.831332  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:04.838639  6377 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:04.684650 13915 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.804767 13797 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:04.837042  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:04.824461  6382 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:28:04.832998  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.844930  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.814692 14002 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:04.850137  6457 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:04.851166  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.851742  6457 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:04.853296 13822 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.853956 13822 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.854697 13755 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.856004 13755 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:04.788122 13873 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.817173 13888 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:04.833994 13904 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:04.817967 13766 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:04.860044  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.864339  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.867949  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:04.868474  6382 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:04.870726  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.783234 13908 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:04.871362  6380 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:04.843937 13987 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.837900 13754 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:04.857609 13797 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:04.856313 13822 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.883332  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:04.884192  6457 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:28:04.856390 13755 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.860975 13258 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.882844  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:04.880968  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.860370 13776 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.886940  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.872085 13766 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:04.889716  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:04.875085 13726 ts_tablet_manager.cc:1403] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 4.327s	user 0.063s	sys 0.056s
I20260709 12:28:04.890982  6380 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 18.
I20260709 12:28:04.903362  6380 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:28:04.890331  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:04.874014 13849 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:04.877676 13987 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:04.895320  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3094) took 2448 ms. Trace:
I20260709 12:28:04.911059  6385 rpcz_store.cc:276] 0709 12:28:02.447104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.896155 (+449051us) service_pool.cc:224] Handling call
0709 12:28:02.957738 (+ 61583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.705475 (+747737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.895044 (+1189569us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400038754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:04.895247 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1718097664}
I20260709 12:28:04.876508 13888 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:04.889940 13755 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:04.920789  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:04.895699 14005 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:04.925964  7332 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:04.895262 13710 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.893534 13776 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:04.911222 13849 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.951483 13677 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:04.916024 13994 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:04.955289  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.958593 13915 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:04.959656  7319 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:04.931123  6382 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:04.961424  7332 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:04.963786  6453 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3331) took 3385 ms (client timeout 2341 ms). Trace:
I20260709 12:28:04.930202 14005 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.961009 13994 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:04.965027  6453 rpcz_store.cc:269] 0709 12:28:01.578128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.963497 (+3385369us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:04.963692 (+   195us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:28:04.933418 13776 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:04.954259 13849 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.002246  6453 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:05.003154  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:05.003330  6453 raft_consensus.cc:606] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:05.003873  6453 raft_consensus.cc:614] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:05.010758  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:04.963970 14013 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:04.973424 14005 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.000375 13994 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:05.038100  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 170505us
I20260709 12:28:05.039600  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:05.039804  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.047739  6453 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:04.987640 13752 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:04.965283 13888 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.052727  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.036542 13988 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.053817  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:05.059209  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:05.055258 13993 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:28:05.071144 13868 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:05.063197  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.071762 13868 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.072206 13868 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.072175  6385 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:05.073912  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:05.074877 14012 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:05.075412 14012 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.075773 14012 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:05.077602 14002 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:05.078159 14002 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:05.078610 14002 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.070442  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:05.053607 14003 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
I20260709 12:28:05.060047 13991 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.083709 13868 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.084334 13988 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:05.085510 13988 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:05.085978 13988 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.087437 13988 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.088688 13988 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.089586 13868 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.097958 13629 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.099116 13797 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.104133 13752 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.104780 13710 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:05.109607 13999 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:05.137753  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 98244us
I20260709 12:28:05.142520  6381 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:05.145555  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 348414us
I20260709 12:28:05.146610  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:05.137949 14010 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:28:05.144220  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.150645 13797 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:05.151088 14016 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
W20260709 12:28:05.141289 13790 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.000s
I20260709 12:28:05.159641  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:05.150309 14022 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:28:05.164961  6453 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:05.163508  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:05.155323 13986 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.617s	user 0.005s	sys 0.007s
I20260709 12:28:05.143677 13710 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:05.170783  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:05.163801 14009 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:28:05.158998 13629 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:05.173475 13974 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:28:05.174026 13710 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:05.170040 14017 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 12:28:05.166316 14027 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:05.184768  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.185585  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:05.173849 13988 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:05.187114  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.189008  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.195466  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.197093  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:05.179876 14004 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:05.179638 13991 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:05.202265  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:05.207552  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.187307 14027 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:05.210016  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.210744 14027 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:05.211213  6309 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:05.218906  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.184861 13629 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:05.196307 13988 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:05.228808  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.221944  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.246656  6459 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:05.234148  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.204130 14004 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:05.248358 14004 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.240376 13726 raft_consensus.cc:359] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.228101 13997 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:05.175154 13986 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.188031 13989 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:28:05.252285  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.272146  6303 raft_consensus.cc:2376] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19. Re-sending same reply.
I20260709 12:28:05.255746 13776 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.258298 13755 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.259981  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:05.252715 13964 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.950s	user 0.005s	sys 0.019s
I20260709 12:28:05.259956 13766 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.204228 14025 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:28:05.205861 13991 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:05.270866  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:05.253829  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56812.4us
I20260709 12:28:05.277182  6455 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:05.250835 13988 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:05.278716  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:05.280817  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51750.2us
I20260709 12:28:05.282400  6303 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.281231  6309 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:05.262159 13754 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.284324  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.284421  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:05.270180 13994 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.290212  7316 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:05.288056  6304 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:05.290863  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:05.293251 13986 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.293766 13900 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.282168 13991 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:05.296470  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.276679 14032 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:05.320444 13726 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:28:05.321346 13726 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:05.298786  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.299757 13677 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.323026 13677 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.303310  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:05.323906  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.325109 13752 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.325316 13677 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.325927 13752 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:05.298910  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.255746 13849 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.327455  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.300272  6383 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:28:05.328070 13726 ts_tablet_manager.cc:1434] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.422s	user 0.003s	sys 0.007s
I20260709 12:28:05.303001  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:05.329015  6455 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:05.329147 13754 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.329986  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:05.330330  6450 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1265) took 3438 ms (client timeout 1951 ms). Trace:
I20260709 12:28:05.330585  6455 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:05.330761  6450 rpcz_store.cc:269] 0709 12:28:01.892358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.005085 (+1112727us) service_pool.cc:224] Handling call
0709 12:28:05.330129 (+2325044us) spinlock_profiling.cc:243] Waited 566 ms on lock 0x7b54001617d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:05.330268 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1301656448}
I20260709 12:28:05.328114 13849 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:05.305012 14026 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:05.332032  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.305181  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2943) took 2874 ms. Trace:
I20260709 12:28:05.332789  6450 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:05.309692 13629 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.333686  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.333776 13900 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.334913  6451 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:28:05.336171 13845 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.332875  6310 rpcz_store.cc:276] 0709 12:28:02.430813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.430937 (+   124us) service_pool.cc:224] Handling call
0709 12:28:02.471738 (+ 40801us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.624935 (+153197us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:02.624977 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.641404 (+ 16427us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.020676 (+379272us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:03.020731 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.511989 (+1491258us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400140fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:04.512030 (+    41us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:05.252009 (+739979us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:05.304905 (+ 52896us) raft_consensus.cc:1685] finished
0709 12:28:05.304927 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.305089 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17136,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":14662,"spinlock_wait_cycles":405439104,"thread_start_us":322433,"threads_started":1,"wal-append.queue_time_us":419885}
I20260709 12:28:05.337646  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.337827  6451 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:05.338692  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.311884 13776 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.314386 13999 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.302780 13797 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:28:05.331614  6376 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:05.340214 13999 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:05.340345  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.344458  6459 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:05.346140 13987 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.348482 13840 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.334347  6450 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:05.352322  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.354688  6459 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:05.365288  6376 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:05.361930  6304 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:05.363982  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:05.366124  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.367497  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:05.362898  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:05.369454  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.368511  6385 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:05.372290  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.370903  6304 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:05.370448 13790 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:05.370935 13999 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:05.384173  6455 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:05.376497  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.378895  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:05.320259 13755 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.384172 13752 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:05.368342 13840 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.348766 13987 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:05.374272 13342 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:05.320259 13766 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.313977 14012 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.400345  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.411854  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.424152  6450 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:05.421036  6304 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:05.434877  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.363819 13828 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:05.437601  6450 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
W20260709 12:28:05.412696  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49927.2us
I20260709 12:28:05.431507  6303 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:28:05.385409 13616 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.441113  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:05.436446  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:05.447367 13994 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.449493 14012 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.438834 13677 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:05.450887 13901 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.451824 13752 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:05.452819 13752 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.432540 14027 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.453297 13752 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:05.455435 14002 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.457772 14002 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.458709 13988 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.438663 13868 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:05.460150 13616 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.461903 13988 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:05.463105 13616 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:05.446077 13991 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.464906  6309 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:05.466217 13991 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:05.466548 13967 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.908s	user 0.003s	sys 0.018s
I20260709 12:28:05.378067 14005 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.469105 14005 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:05.470903  6274 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.110s (SENT)
I20260709 12:28:05.419690 13851 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:05.420711 13856 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:05.472198 13856 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.472637 13856 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.385752 13999 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:05.474751 13999 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:05.477465  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3647) took 2071 ms (client timeout 2249 ms). Trace:
I20260709 12:28:05.480064 14027 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:05.478037  6454 rpcz_store.cc:269] 0709 12:28:03.406198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.007851 (+601653us) service_pool.cc:224] Handling call
0709 12:28:05.477172 (+1469321us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b54001e23d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:05.477350 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":143004544}
I20260709 12:28:05.485397 13982 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.486835 13677 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:28:05.487071  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46045.8us
I20260709 12:28:05.487391 13840 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.451249 13791 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.488094 13677 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:05.488858 13710 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.489603 13873 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:05.491628 13851 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:05.492071 13851 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:05.492221 13868 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:28:05.493388 13677 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:05.470989  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3321) took 4006 ms. Trace:
I20260709 12:28:05.493806 13677 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:05.494140  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:05.493891  6456 rpcz_store.cc:276] 0709 12:28:01.464574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.403288 (+938714us) service_pool.cc:224] Handling call
0709 12:28:04.533151 (+2129863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:04.577540 (+ 44389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.470749 (+893209us) spinlock_profiling.cc:243] Waited 2.13 s on lock 0x7b54000c1cd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:05.470923 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4897588480}
I20260709 12:28:05.479007  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:05.420486 13997 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.495756  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:05.496345 13655 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:05.496441  7335 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:05.496796 13997 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.425736  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c413b06caa444496aecb13ac2142c246"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:05.501350  6455 raft_consensus.cc:606] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:05.502399  6455 raft_consensus.cc:614] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:05.492972 13851 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.384565 13790 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:05.505347  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.506546  6303 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:05.508456  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3273) took 2924 ms. Trace:
I20260709 12:28:05.509364  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1160) took 2430 ms. Trace:
I20260709 12:28:05.513540  7316 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:05.501478  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:05.513964  6308 rpcz_store.cc:276] 0709 12:28:03.078540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.079429 (+   889us) service_pool.cc:224] Handling call
0709 12:28:03.305315 (+225886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.306355 (+  1040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.509115 (+2202760us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54002067d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:05.509292 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":518703360}
I20260709 12:28:05.509435  6378 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:05.515756  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.510401  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1327) took 3373 ms. Trace:
I20260709 12:28:05.519627  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.520521  6382 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:05.517088  7314 rpcz_store.cc:276] 0709 12:28:02.136951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.137101 (+   150us) service_pool.cc:224] Handling call
0709 12:28:02.137491 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.993712 (+856221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.510118 (+2516406us) spinlock_profiling.cc:243] Waited 855 ms on lock 0x7b54006f7a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:05.510343 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1967207168}
I20260709 12:28:05.495891 13710 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.523303 14004 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.561524  6310 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 17: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:05.561896  6381 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:28:05.563550 13851 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:05.564567 13790 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.565279 13856 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.565856 13790 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:05.566475  6377 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:05.566597 13814 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:05.567510 13814 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:05.567876 13814 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:05.568635  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.568684 13814 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.569470 13840 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:05.571528  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:05.572073 14004 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.572113  6455 raft_consensus.cc:606] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:05.573719  6455 raft_consensus.cc:614] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:05.578369  7318 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4"
dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:41092
I20260709 12:28:05.579095  7318 raft_consensus.cc:606] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:05.578087 13999 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.495666 13677 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.582894 13677 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:05.523496  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.586381  6380 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:05.587035 13999 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.513685  6300 rpcz_store.cc:276] 0709 12:28:02.583872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.584919 (+  1047us) service_pool.cc:224] Handling call
0709 12:28:02.585255 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.893521 (+308266us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:03.006737 (+113216us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.409346 (+402609us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.409446 (+   100us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:03.409521 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.508196 (+2098675us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54002df0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:05.508247 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:05.508381 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":148389,"prepare.queue_time_us":125792,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":36660,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":26278,"spinlock_wait_cycles":708432640,"thread_start_us":379546,"threads_started":1,"wal-append.queue_time_us":520871,"wal-append.run_cpu_time_us":2297,"wal-append.run_wall_time_us":1073474}
I20260709 12:28:05.597160  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.599942  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.600740  6459 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.607466  7335 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:05.594321  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.610003  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:05.594015  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.613512  6380 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:28:05.519678 13982 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.525154 13828 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:05.620736 13999 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:05.590222  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:05.625069  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.627031  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.632110  7316 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:05.632319  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2963) took 1349 ms. Trace:
I20260709 12:28:05.632762  6307 rpcz_store.cc:276] 0709 12:28:04.282908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.283981 (+  1073us) service_pool.cc:224] Handling call
0709 12:28:05.632122 (+1348141us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54001b2bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:05.632255 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605264640}
I20260709 12:28:05.633904  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.634382  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.623412  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.634605  6455 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:05.637403  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.589912  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:05.639308  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49478.2us
I20260709 12:28:05.615862 13790 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:28:05.621078 13221 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:28:05.642985 13873 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:05.632366 13982 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.645663 13856 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:05.634835 13814 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:05.648674  6452 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:28:05.649078  6452 pending_rounds.cc:85] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:28:05.649307  6452 pending_rounds.cc:107] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:28:05.649169  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.649798  6452 raft_consensus.cc:2889] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:05.650467  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.652190  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.643170 13790 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:05.643870 13873 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:05.624274 14005 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:05.643935 13221 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:05.658082  6384 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:05.620913 14035 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:05.662680  6455 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:05.658787  6456 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:05.653090 14013 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.653402 13991 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:05.646377 13982 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.653824 13999 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:05.633074 13828 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:05.684624  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.669417  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:05.681559  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.693511  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:05.672209  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.697633  6384 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:05.687661  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:05.678218  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.700309  7335 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:05.702191  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.701273  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:05.706166  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 184982us
I20260709 12:28:05.678694 13713 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:05.673903 13221 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.671655 13790 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:05.792686  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.673830 13755 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:05.712697  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:05.682806 14035 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.703672 13888 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:05.688712 13994 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.126s
W20260709 12:28:05.813643  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 105202us
I20260709 12:28:05.705687 13983 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:28:05.823252  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 110649us
W20260709 12:28:05.824149  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.708827 13995 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:05.684043 14005 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:05.830986  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.833109  6383 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:05.680961 13845 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.134s
I20260709 12:28:05.682582 13629 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.132s
I20260709 12:28:05.835246  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.836293  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.844343  6456 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:05.844386  6459 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:05.811010 13713 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:05.859099 13713 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:05.866524 13999 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.868388 14005 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:05.869607 14039 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:28:05.828068 13755 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:05.870869 13828 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.871609 13755 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.872768 14035 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.705449 14025 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.122s
I20260709 12:28:05.843154  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.852246  7314 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:05.875046  6450 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:05.687232 14013 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:05.817878 13655 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:28:05.867071 13629 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:28:05.836910  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.955853  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.956252  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.853127  6459 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:05.879633  6456 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:05.705880 13752 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.896678  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1119) took 1070 ms. Trace:
I20260709 12:28:05.961519  6378 rpcz_store.cc:276] 0709 12:28:04.826241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.826375 (+   134us) service_pool.cc:224] Handling call
0709 12:28:05.509268 (+682893us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.515690 (+  6422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.896372 (+380682us) spinlock_profiling.cc:243] Waited 682 ms on lock 0x7b54003d36d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:05.896596 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1569651968}
W20260709 12:28:05.897228  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56305.7us
I20260709 12:28:05.816346 13856 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:28:05.826713 13989 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:05.909693 13705 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:05.912147  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.913281 13997 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:05.979351 13997 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:05.979813 13997 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.980722 13997 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.982003 13997 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:05.914256 13983 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.927220 13221 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.983208 13221 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:05.915123  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.983446 13982 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:05.984655  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.985298  6384 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:05.985832 13845 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:05.916302  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.986341 13845 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.987532  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.988550  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.988909 13845 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:05.989594  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.917503 14025 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:05.920003  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.990818 14025 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.992097  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.993309 13989 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.993799 13989 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:05.993821  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.885488 13888 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:05.821836 13873 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.884702 13995 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:05.938293  6455 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:05.926754 13790 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.944659  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:06.009878  6383 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:06.011029  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:05.962003  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 149130us
I20260709 12:28:06.011620  6383 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:05.963171  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.973161 13705 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:05.973284  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-d53ab19d424b4a85af2153ef24520678 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:28:05.974524 13752 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.028662  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:06.033716  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.974988  6385 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:06.034940  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:06.035246  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:05.962538 13994 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:05.976519  6272 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.002s	sys 0.020s
W20260709 12:28:06.036389  6272 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.004s	sys 0.021s
W20260709 12:28:06.036654  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 627197us
I20260709 12:28:06.036736  6303 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:06.037884  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.039235  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.033465  6302 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:05.984308 13982 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.991933 14025 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:05.917326  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:06.057976  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.839895 13991 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:05.839228 13868 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.094s
I20260709 12:28:05.972427  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:06.070289  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97917us
I20260709 12:28:06.070691  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:06.071511  6377 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:06.001410  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:05.998245  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:05.957409 13629 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:05.972218  6381 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:06.021159  6300 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:06.075982  6453 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:28:06.076572  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.026916  6384 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:06.077245  6451 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:06.011698 13683 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:28:06.042631  6455 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:06.044296  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.045589  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:06.082280  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.054247  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:06.082319  6305 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:06.054678  6303 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:06.055251  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.058230 14002 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:28:06.058770  6382 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:06.044042  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-d53ab19d424b4a85af2153ef24520678 on tablet d53ab19d424b4a85af2153ef24520678 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.095415  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.096274  6455 raft_consensus.cc:2410] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:28:06.060654  6456 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:06.097752  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.067178  7331 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:28:06.066380  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.068739  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:06.071429  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:06.100438  6301 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:06.072569  6380 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:06.067739  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.073331  6383 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:06.070940  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:06.078260  6384 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:28:06.101857  6384 pending_rounds.cc:85] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 10
I20260709 12:28:06.042090  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:06.057147 13772 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:06.085256  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.055981  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:05.972718  6450 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:06.088469  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:06.059149 13994 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.113368  6456 raft_consensus.cc:2410] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:28:06.091598  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.028949 13904 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:28:05.983225 13983 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.072558  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:06.117082  6377 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 15 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 6.
I20260709 12:28:06.117704  6450 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:28:06.124893  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.111593  6454 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: [17.9-19.10]
I20260709 12:28:06.113996  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:06.089507  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.107788  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.079757 14047 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:06.091118 13616 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.129953  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.129484  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.131824  6300 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:28:06.069262 13991 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.132632  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.133340  6300 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:06.127091  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:06.107618  6302 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:06.141144  6450 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:28:06.143374  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.143961  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.149994  6450 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:06.102134  6384 pending_rounds.cc:107] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 17.11
I20260709 12:28:06.151410  6384 raft_consensus.cc:2917] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Aborting config change with OpId 17.11: Aborted: Op aborted by new leader
I20260709 12:28:06.103627 14025 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:06.160481  6451 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:06.153223  6384 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:06.161358  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.162182  6305 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.158551  6450 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:28:06.163304  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:06.159546  6382 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:06.087805 13888 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.080180 13705 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.172511  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 9) took 12076 ms. Trace:
W20260709 12:28:06.173602 14048 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:28:06.178630  6384 rpcz_store.cc:276] 0709 12:27:54.095708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.098380 (+  2672us) service_pool.cc:224] Handling call
0709 12:27:55.100249 (+1001869us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b2400297ed8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa003dcb9 00007f6fa003bdcb 00007f6fa013a2b1 00007f6fa0135848 00007f6fa01357ad 00007f6fa15769a7 00007f6fa320385c 00007f6fa3203699 00007f6fa32006ad 00007f6fa3200527
0709 12:27:55.119417 (+ 19168us) log.cc:844] Serialized 485 byte log entry
0709 12:28:06.172380 (+11052963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14807,"raft.queue_time_us":230463,"raft.run_cpu_time_us":1900,"raft.run_wall_time_us":107604,"spinlock_wait_cycles":8394496,"thread_start_us":70305,"threads_started":1}
I20260709 12:28:06.180751  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.182533  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3642) took 3062 ms. Trace:
I20260709 12:28:06.183059  6452 rpcz_store.cc:276] 0709 12:28:03.120359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.220293 (+1099934us) service_pool.cc:224] Handling call
0709 12:28:04.220669 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.650318 (+1429649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.182189 (+531871us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400162bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:06.182427 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282630400}
I20260709 12:28:06.176182  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:06.176965  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49986us
I20260709 12:28:06.130435 13900 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:06.195458  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:06.196130  6455 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:06.179214  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.179914  6380 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:06.183437  6303 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:06.198518  6302 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:06.185182 14042 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:28:06.228281  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 205490us
I20260709 12:28:06.243676  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.099131 13995 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.151557 13772 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 12:28:06.251287  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.252439  6309 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:06.189013 13677 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:06.258471 13677 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.259397 13677 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.200484 13705 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.256199  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.194648  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.263466  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:06.266450  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71903.1us
I20260709 12:28:06.199896 14047 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.268076  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:06.273043 14050 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 12:28:06.276794  6450 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:28:06.286233  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.274873  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.281782  6300 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:06.283252 14059 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:06.238220  6305 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:06.296036  6450 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 22. Ops: []
I20260709 12:28:06.181368 13713 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.300061  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.301590  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.303287  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.303985 13994 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:06.304456  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.304817 13994 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:06.305352  6302 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 29 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:28:06.306214  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.308663  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.311565  6309 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:06.310585  6302 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:06.313045  6450 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:06.311921  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2981) took 1122 ms. Trace:
I20260709 12:28:06.318552  6303 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:06.286473 13748 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:06.165460 13629 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.323349  6455 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 15.
I20260709 12:28:06.325969  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.307819 13908 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:06.386154  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.330711 14047 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.295819 14059 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.332234 13713 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.295585 13797 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:06.334802 13982 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.325479 13991 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.316732  6304 rpcz_store.cc:276] 0709 12:28:05.189268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.439289 (+250021us) service_pool.cc:224] Handling call
0709 12:28:05.439766 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.440041 (+   275us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:05.704704 (+264663us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.311569 (+606865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:06.311629 (+    60us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:06.311678 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.311745 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.311865 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":67100,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":45,"thread_start_us":807262,"threads_started":3,"wal-append.queue_time_us":555770}
I20260709 12:28:06.405049  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:06.311343 13916 consensus_queue.cc:1175] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" responder_term: 23 status { last_received { term: 22 index: 16 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:06.343981  7320 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:28:06.350610 13221 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:06.407464 13221 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:06.408349 13221 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.355700  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.411362  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.412364  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.413278  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:06.362470 13814 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.715s	user 0.009s	sys 0.005s
W20260709 12:28:06.364322  6351 connection.cc:256] RPC call timeout handler was delayed by -2.0817376470004092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 12:28:06.414959  6351 connection.cc:256] RPC call timeout handler was delayed by -2.0133376470002986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
I20260709 12:28:06.416895  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.418660 13655 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.419411 14067 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:06.422955 13982 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:06.423628 13982 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.424539  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.425539  6455 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:06.424793 13982 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.406430  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.431788 14075 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:06.432711 14075 consensus_queue.cc:1243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:06.433813 13868 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:06.431800  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.382236 13748 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.435657 13868 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:06.436576 14065 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:28:06.436870 13851 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:06.437033 14065 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.364760 13856 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.379287  6307 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:06.379895  6309 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:06.380573  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:06.365099 13710 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.383697  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3355) took 1167 ms. Trace:
I20260709 12:28:06.437315 13851 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:06.384130  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 314512us
W20260709 12:28:06.330739  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.390096  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1283) took 1802 ms. Trace:
I20260709 12:28:06.397159  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.402581 13713 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.403028 13797 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.462976  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:06.470649  7316 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:06.342065 13997 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.438119  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.438757  6300 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:28:06.464402  7331 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:06.435683  6455 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:06.374703  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 289507us
I20260709 12:28:06.473770  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.474257  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:06.444331  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.476651  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.452982  6457 rpcz_store.cc:276] 0709 12:28:05.216220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.218232 (+  2012us) service_pool.cc:224] Handling call
0709 12:28:05.477334 (+259102us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:05.477586 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:28:05.477885 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:28:05.686714 (+208829us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:05.686798 (+    84us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:05.686872 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.686977 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.383393 (+696416us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54001e23d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:06.383605 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3643,"prepare.queue_time_us":207,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":3795,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":8164,"spinlock_wait_cycles":595383808,"thread_start_us":204673,"threads_started":1,"wal-append.queue_time_us":455679}
I20260709 12:28:06.466861  6450 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 14. Current term is 15. Ops: []
I20260709 12:28:06.456355  6458 rpcz_store.cc:276] 0709 12:28:04.587970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.681225 (+ 93255us) service_pool.cc:224] Handling call
0709 12:28:04.681579 (+   354us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:04.682060 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:04.682087 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:04.682807 (+   720us) log.cc:844] Serialized 52 byte log entry
0709 12:28:05.464603 (+781796us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b18001b1240. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:05.707959 (+243356us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:06.102944 (+394985us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.385098 (+282154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.385188 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:06.385235 (+    47us) raft_consensus.cc:1685] finished
0709 12:28:06.389775 (+  4540us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.390018 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":567310,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":9807,"raft.queue_time_us":203707,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":5277,"spinlock_wait_cycles":202765056,"thread_start_us":799789,"threads_started":3,"wal-append.queue_time_us":1252030}
I20260709 12:28:06.479542  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.475205  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.481690  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.482062  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.414397 13616 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:06.485181  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:06.485913  7335 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:06.486649  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:06.503937  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.454880 14069 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.505599  6452 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:06.465482 14060 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:06.477412 13873 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:28:06.508610  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.478271 14057 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.485134 13994 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.515169  7319 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:06.516769 13221 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.519367  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:06.523463  7314 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:06.528702 13221 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.533509  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:06.514974 14069 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.505501 14064 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:06.525646 14060 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.542800 13797 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:06.464804 13856 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.542941 14047 raft_consensus.cc:2764] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:28:06.454262 13989 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.543677 14066 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:06.546932 13999 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.554064  6381 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:06.505718 13677 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:06.475966 13710 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.437047 13748 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.478546 13713 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.487872 13705 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:28:06.587703  7335 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:06.511080 14063 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:06.596354  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 116906us
I20260709 12:28:06.583067 13683 raft_consensus.cc:993] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: : Instructing follower 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:28:06.599427 14047 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:06.604418 14047 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:06.604777 14047 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.605185 13856 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:06.600189 13900 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.600400  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.603488 13683 raft_consensus.cc:1081] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Signalling peer 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:28:06.526124 14065 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.616207  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:06.618257  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 124252us
I20260709 12:28:06.534910 13851 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:06.620339  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 111860us
I20260709 12:28:06.624001  7319 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:06.625456 13713 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.627454 13710 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:06.628490 13851 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:06.628532 14069 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.629980 14063 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.631661 14060 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:06.631372 14063 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.619659 13856 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:06.630101 14040 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.988s	user 0.002s	sys 0.000s
I20260709 12:28:06.622021 13677 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:06.617594 13705 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:06.635948  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3121) took 1284 ms. Trace:
I20260709 12:28:06.636096  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.619246 13748 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:06.637401  6379 rpcz_store.cc:276] 0709 12:28:05.351893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.352090 (+   197us) service_pool.cc:224] Handling call
0709 12:28:05.352519 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:05.352978 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.353021 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:05.353636 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:28:05.506962 (+153326us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:05.507003 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.507066 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:06.629547 (+1122481us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:06.634377 (+  4830us) raft_consensus.cc:1685] finished
0709 12:28:06.634414 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.635883 (+  1469us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11655,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6683,"spinlock_wait_cycles":12342016,"thread_start_us":131071,"threads_started":1,"wal-append.queue_time_us":266582}
I20260709 12:28:06.640322  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.641812  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.642580 13873 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.647956 13999 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:06.656073 13873 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.642050 14069 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.656706 13873 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.644310  7317 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:06.663019 14059 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:06.664252 14049 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.805s	user 0.004s	sys 0.000s
I20260709 12:28:06.664723 13873 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:06.665163  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:06.666383 14002 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.639171 13989 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:06.666255 14063 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:06.667485 14002 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:06.671459  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.675383 13994 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:06.675606 14089 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.676374 14069 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:06.676501 14089 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.677452 14069 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:06.677873 14069 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:06.679011 13991 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:06.679356 14089 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:06.679495 13991 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.680301 13991 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.681102 13991 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:06.664288 14059 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.689481 13982 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:06.690855 14069 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:06.691715  7317 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1126) took 1948 ms (client timeout 2105 ms). Trace:
I20260709 12:28:06.691859 13790 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:06.692225  7317 rpcz_store.cc:269] 0709 12:28:04.743514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:04.743722 (+   208us) service_pool.cc:224] Handling call
0709 12:28:06.691516 (+1947794us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54003bc4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:06.691622 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4340757248}
I20260709 12:28:06.698555 14047 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.700053 14047 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.692319 14069 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:06.713816 14065 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:06.694841 13873 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:06.716992 13873 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:06.717468 13873 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:06.673190 14005 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.737731 14005 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.775555  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:28:06.598222  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 224563us
I20260709 12:28:06.788901  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.669142  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:06.805361  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 136034us
W20260709 12:28:06.807041  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 185660us
I20260709 12:28:06.808041  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:06.815282  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 2989) took 1563 ms. Trace:
I20260709 12:28:06.815536  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:06.815824  6308 rpcz_store.cc:276] 0709 12:28:05.253018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.514948 (+261930us) service_pool.cc:224] Handling call
0709 12:28:05.515318 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:05.520561 (+  5243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:05.520603 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:05.521129 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:28:05.565894 (+ 44765us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b18003c5aa0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:05.707226 (+141332us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:05.707951 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:05.708040 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:06.663609 (+955569us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:06.801217 (+137608us) raft_consensus.cc:1685] finished
0709 12:28:06.814524 (+ 13307us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:06.814976 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4841,"raft.queue_time_us":5720,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":31333,"spinlock_wait_cycles":39096448,"thread_start_us":139188,"threads_started":1,"wal-append.queue_time_us":300864}
W20260709 12:28:06.819447  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.821972  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.822599  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.820964  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:06.821152  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:06.829188  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.673564 13900 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.844405 13851 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:06.844699  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:06.846853  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.848335 14069 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:06.849236 14069 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:06.849452 14090 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:06.851567 14069 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.853533 13705 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:06.857661 13755 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:06.858872  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:06.860587  6305 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:28:06.862937 13982 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:28:06.865727  6310 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 2984) took 1661 ms (client timeout 2169 ms). Trace:
I20260709 12:28:06.846340  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:06.867292  7335 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:06.848268 13849 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:28:06.870193  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49586.3us
I20260709 12:28:06.871052  7317 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
W20260709 12:28:06.868013  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47221.4us
I20260709 12:28:06.838186 14005 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:06.866345  6310 rpcz_store.cc:269] 0709 12:28:05.204519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.338404 (+133885us) service_pool.cc:224] Handling call
0709 12:28:06.865131 (+1526727us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54002e7054. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:06.865466 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508117248}
I20260709 12:28:06.850288 14090 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:06.829675 13713 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:28:06.875352 14059 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:06.851208 14066 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:28:06.883296 14073 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:06.884126 13655 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:06.878733 13849 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:06.885031 14073 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:06.886339 14073 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.887296 13991 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:06.887535 13849 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.888278 14073 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.890894 13713 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:06.891654 13713 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:06.882573 13705 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.010s
W20260709 12:28:06.893733  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95406.8us
I20260709 12:28:06.896024 13991 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:06.893889 13705 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:06.892446 13713 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:06.895136 14073 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:06.899679 13982 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:06.900101  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.897234 13991 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.898161 13705 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.914785 13791 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:06.921846  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.922575 14059 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:28:06.918391 13791 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:06.925449 13791 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:06.926352  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:06.926672 14004 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:06.928143 13791 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:06.918008  6452 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:06.945873 14059 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:06.946435 14012 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:06.951692 14012 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:06.952910 14012 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.954314  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3388) took 1450 ms. Trace:
I20260709 12:28:06.891755 14090 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.954758  6385 rpcz_store.cc:276] 0709 12:28:05.503824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.503981 (+   157us) service_pool.cc:224] Handling call
0709 12:28:05.519258 (+ 15277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.034756 (+515498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:06.953776 (+919020us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b5400065754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:06.954116 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1047654656}
I20260709 12:28:06.956256  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:06.955945 13873 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:06.957710  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.960707 14073 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:06.961678 13873 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:06.962836 14080 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:06.963961 14073 consensus_queue.cc:1243] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:06.964599 14080 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:06.965093 14080 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:06.966416 13901 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:06.966558 14080 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.968245 14080 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:06.969570  6458 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:06.967207  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:06.971701  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 67124.8us
I20260709 12:28:06.972591  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.972824  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:06.973589  6458 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:06.948431 14004 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:28:06.977341 13991 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.978114  6459 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:06.950417 14013 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:06.978819 13814 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:06.948201 14096 raft_consensus.cc:2764] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:28:06.980262  6453 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:06.968423  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:06.991063  6452 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:06.978519 14004 consensus_queue.cc:1243] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:06.950323 13705 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:06.993952 14012 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:06.948259  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:06.998831 13873 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:06.999365 13873 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.999815 13873 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:06.946611 14059 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.002432 14059 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.007025  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.008344  6458 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:28:07.011597  6458 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:07.022763  6454 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:07.024920 13982 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:07.025488 13982 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:07.038383 14097 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:28:07.039961 13629 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:28:06.976507  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:07.049625  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76991.6us
I20260709 12:28:07.027732  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.055199  7335 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:07.057837 14033 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:07.058425 14033 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:07.058877 13828 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:07.059999  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.060707 13752 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:07.061133 13982 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.061407 13752 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.063131 13982 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:07.050432 14059 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.060009 14033 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.079658 14033 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:07.081385  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.082095  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.062186 13713 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.053270 13705 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:07.053942  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 105847us
W20260709 12:28:07.105173  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 185910us
I20260709 12:28:07.107213  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:07.113348 13713 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.114084 13755 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:07.114872 13845 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:28:07.115293  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 215355us
I20260709 12:28:07.115701 13845 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.118264 14062 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:07.125897  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:07.120507 13828 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.130736 13828 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.133919 13828 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.120522 13755 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.140480 13755 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.142059  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:07.144927  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.145812 13221 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.121305 13845 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.148222 13752 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.138592 13655 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.149328 13655 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.151858 13655 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.142642 13991 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:07.156217  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 197797us
I20260709 12:28:07.155879 14002 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:07.158946  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.159766  7332 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
W20260709 12:28:07.163551  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46664.1us
I20260709 12:28:07.162081 13629 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:07.169998 13766 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:07.180408  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57269.4us
I20260709 12:28:07.185343  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.188414  6385 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:07.188139  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.188727  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.189710  6381 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:07.190601 13713 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:07.180032 13766 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.190791  6452 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:28:07.191273 13713 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.191936  6453 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:07.192514 13713 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:07.194590  7332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1342) took 2184 ms (client timeout 1938 ms). Trace:
W20260709 12:28:07.195094  7332 rpcz_store.cc:269] 0709 12:28:05.009794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.010372 (+   578us) service_pool.cc:224] Handling call
0709 12:28:07.194347 (+2183975us) spinlock_profiling.cc:243] Waited 2.14 s on lock 0x7b54005c1a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:07.194503 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4933063936}
I20260709 12:28:07.167140 13900 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:07.197624  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 95648.7us
I20260709 12:28:07.188117  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.200567  6454 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
W20260709 12:28:07.203431  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81401.7us
I20260709 12:28:07.206113  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:07.197969 13888 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:07.211079  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.220432  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.221395  6385 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:07.222681 13655 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.223393 13655 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.234084  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.241547  6385 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:28:07.242195  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:07.241294  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.244831  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:07.249277 14090 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:07.250176  7314 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:07.252707  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.252317  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.253715  6379 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:07.232285 13791 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.256778  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:07.258234  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.258287  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.260681 13221 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.003s
W20260709 12:28:07.265122  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.234522 13845 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:28:07.265468  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:07.266707  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 82908.2us
I20260709 12:28:07.219213 13900 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.268967  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.250694 13655 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.269788  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.271502  6382 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:07.266848  6385 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:07.275621 13752 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:07.276108  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.270371  6378 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:07.278137  6383 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:07.279289 13705 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:07.277995  6378 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:07.280287  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.283876 13705 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:07.284126 13752 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:07.284174  6378 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:07.285362  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51097.6us
I20260709 12:28:07.281095  6380 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:28:07.262048 14002 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:28:07.281992  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:07.279686  6376 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:07.264470 13791 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.287312  6381 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:28:07.289319 13901 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:07.296274 13901 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.296697 13901 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:07.286862 13705 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:07.292119 14002 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:07.292240  6454 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:28:07.289245  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.300248  6378 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.302340  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.303313 14062 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:07.304469  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.305466 14062 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:07.300256 13629 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:28:07.307030 13629 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:07.307407 13629 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.309311 13221 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:28:07.310205  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.310958  6453 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:07.311177 13772 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.313262  6385 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:07.312974 13629 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.313722 13983 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.314828 13629 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:07.314699 13983 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.313071 13772 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.321045 13766 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:07.323509 13655 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:07.323987 13655 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.324331 13655 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.326875 13752 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:07.328413 13868 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:07.329192 13868 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:07.327039 13772 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.330473 13868 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.332306 13868 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.333442 13748 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:07.334545 13748 consensus_queue.cc:1243] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:07.335141 14047 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:07.335224  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.329121  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.336843  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.308038 13791 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.330018 13766 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:07.335688 14047 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:07.348357  6378 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:07.311520 13873 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:07.350755  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 134121us
I20260709 12:28:07.300676 14002 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:28:07.351305  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.359393 13755 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.360570 13752 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:07.361047 13752 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.361409 13752 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:07.352871 13748 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:07.370007  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:07.357558 13828 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
W20260709 12:28:07.377424 14099 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.152s	user 0.004s	sys 0.000s
I20260709 12:28:07.385929  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.363327 13713 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:07.386992  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51852.5us
I20260709 12:28:07.386926 13748 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:07.395896  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 59134.1us
I20260709 12:28:07.397293  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:07.398254  7335 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.398932  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:07.385740 13755 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.380628 13983 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.402423 13748 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.410306 13748 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:28:07.412787 13790 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.672s	user 0.015s	sys 0.015s
I20260709 12:28:07.413823  6451 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:07.417470 13705 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.426077 13888 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:07.425980  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:07.427395 13873 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:07.429036 13705 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.438467  7332 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:07.441053  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.442085 14004 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:07.442723 14004 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.443969 14004 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.445631 13900 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.457238  7314 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:07.461452 14090 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:07.463881  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.464720  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.470961 14073 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:07.471577 14073 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.472738 14073 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.463003 13791 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:28:07.503737  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 113013us
I20260709 12:28:07.504908 14012 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:07.505707  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.506879 13900 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:07.507494 13900 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:07.507750 13629 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:07.508488  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:07.509311 13629 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:07.509550  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:07.509004 13900 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.466162 13901 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.513021 14059 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:07.527933  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 268) took 20136 ms (client timeout 15000 ms). Trace:
W20260709 12:28:07.529888  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 155001us
I20260709 12:28:07.532204  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.537297  6378 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:07.482321 14047 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.540340  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.543764  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.547973  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:07.523561 14059 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:07.540375 14103 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:28:07.485612 14002 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.542409 14047 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:07.559358  6381 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:28:07.563586  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.553027 14059 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.562165  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.571355 13991 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.572382 13991 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.574726 13991 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:07.571484  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet d53ab19d424b4a85af2153ef24520678: Aborted: Op aborted by new leader
I20260709 12:28:07.550028 13790 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:28:07.578588 13791 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:07.556947 14004 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:07.579808 13766 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.582717 14090 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:07.586777  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.587332  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:07.588006  6310 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:07.588820  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.589727  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.592289 14111 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:07.571623  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.593443  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:07.601305  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3677) took 1434 ms (client timeout 1544 ms). Trace:
I20260709 12:28:07.603544 13900 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.607692  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.608718  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:07.586799  7335 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
W20260709 12:28:07.529291  6202 rpcz_store.cc:269] 0709 12:27:47.227062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.227372 (+   310us) service_pool.cc:224] Handling call
0709 12:27:47.545203 (+317831us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:07.357632 (+19812429us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:55.635049 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:55.635777 (+   728us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:55.635814 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:27:55.635834 (+    20us) tablet.cc:662] Decoding operations
0709 12:27:55.637831 (+  1997us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:55.637937 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:55.637976 (+    39us) tablet.cc:685] Acquiring locks for 28 operations
0709 12:27:55.797560 (+159584us) tablet.cc:701] Row locks acquired
0709 12:27:55.797582 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:27:55.797758 (+   176us) write_op.cc:270] Start()
0709 12:27:55.797888 (+   130us) write_op.cc:276] Timestamp: P: 1783600075797703 usec, L: 0
0709 12:27:55.797972 (+    84us) op_driver.cc:348] REPLICATION: starting
0709 12:27:55.958633 (+160661us) log.cc:844] Serialized 10721 byte log entry
0709 12:27:56.458767 (+500134us) op_driver.cc:464] REPLICATION: finished
0709 12:27:56.607008 (+148241us) write_op.cc:301] APPLY: starting
0709 12:27:56.607207 (+   199us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:57.124996 (+517789us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:57.125121 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:57.573712 (+448591us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:57.574229 (+   517us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=0,mrs_lookups=0
0709 12:27:57.574304 (+    75us) write_op.cc:312] APPLY: finished
0709 12:27:57.724275 (+149971us) log.cc:844] Serialized 244 byte log entry
0709 12:27:57.806176 (+ 81901us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:57.806618 (+   442us) write_op.cc:454] Released schema lock
0709 12:27:57.806723 (+   105us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:57.896214 (+ 89491us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:57.899768 (+  3554us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1125,"reported_tablets":43,"spinlock_wait_cycles":585186560,"child_traces":[["op",{"apply.queue_time_us":129652,"apply.run_cpu_time_us":69558,"apply.run_wall_time_us":1245897,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121223,"num_ops":28,"prepare.queue_time_us":112759,"prepare.run_cpu_time_us":9511,"prepare.run_wall_time_us":339805,"raft.queue_time_us":385816,"raft.run_cpu_time_us":39286,"raft.run_wall_time_us":136636,"replication_time_us":660528,"thread_start_us":576562,"threads_started":5,"wal-append.queue_time_us":136714,"wal-append.run_cpu_time_us":19338,"wal-append.run_wall_time_us":2501847}]]}
I20260709 12:28:07.606389  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:07.624150  6451 rpcz_store.cc:269] 0709 12:28:06.166721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.175920 (+  9199us) service_pool.cc:224] Handling call
0709 12:28:07.601061 (+1425141us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001c1e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:07.601223 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2823312128}
I20260709 12:28:07.629011  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.632383  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.638180  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.638922  6302 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:07.641712  6302 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:07.642683  6303 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:28:07.643723  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:07.644644  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:07.645682 13655 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.646927  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:07.648185 13766 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.648461 13791 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:07.650421  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57090.9us
I20260709 12:28:07.650732 13982 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:07.651883  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.652351  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:07.651561  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.652772  6379 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:07.632764  6305 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:07.636794  6453 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:07.622572 14090 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.671058  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:07.651252  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.683359  6309 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:07.683068  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:07.660935 13655 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.681013  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.651311 13982 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:07.687731  6305 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:28:07.688695  6457 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:07.690608  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44310.5us
I20260709 12:28:07.689610  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:07.674646 13873 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:07.664268 14047 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:07.706367  6384 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:07.676159 13845 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:07.719462 14047 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.721911 14047 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.682230 14012 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.723120 14012 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:07.725389 14012 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.681473 13849 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:07.700317  6304 raft_consensus.cc:2376] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22. Re-sending same reply.
I20260709 12:28:07.730664 14113 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:07.733718  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.734212  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.745729 14002 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.749235 13983 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:07.699592  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112374us
I20260709 12:28:07.750974  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.751207  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:07.679744 14090 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:07.706913 13221 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:07.710561  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.753530 13221 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.710633  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.755635  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.711928  6305 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:28:07.757553  6305 pending_rounds.cc:85] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 9
I20260709 12:28:07.757886  6305 pending_rounds.cc:107] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:28:07.758173  6305 raft_consensus.cc:2889] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:07.712566  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.759052  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.700184 14111 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.714963 13916 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:07.715018  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 107434us
I20260709 12:28:07.760514  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:07.760778  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.761318  6456 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.761102 13982 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.761850 14073 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:28:07.715921  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 129460us
I20260709 12:28:07.762971  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.718662 13901 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:07.713356  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.757136  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:07.773355  6451 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:07.777143  7335 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:07.751746  6457 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:07.772429 13873 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.764132 14033 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:07.749953 13983 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.790589  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:07.798106  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47176.5us
I20260709 12:28:07.805194  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:07.807255  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94115.5us
I20260709 12:28:07.812108  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.806103  6302 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:28:07.818764  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:07.789568 13873 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:07.824178  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:07.789798 13790 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:07.794723 14033 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:07.831009  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.806926 14047 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:07.844622  6378 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:07.832726  6310 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:28:07.849264  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.777613 13916 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:28:07.864141 13916 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.864625 13916 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.858680 13655 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:07.858750  6302 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:07.820770 13901 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:28:07.861207 14059 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.832183 14114 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:07.862627 13766 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:07.867161  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55167.6us
I20260709 12:28:07.789160  6303 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:53970
I20260709 12:28:07.868280 13901 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:07.868919  6303 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:07.869328  6303 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:07.830574 13888 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:07.871436 13888 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:07.871524  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.871909 13888 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:07.872224  6453 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:07.873226  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:07.873740  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:07.874085  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 92155.1us
I20260709 12:28:07.874945  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:07.874971 13748 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:07.875780 13748 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:07.876245 13748 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:07.865990  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:07.864959  6451 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:07.866601 14114 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.880389  6457 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:28:07.842689 14033 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:07.890365  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.847530 13845 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:07.899787  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76047.6us
I20260709 12:28:07.882028 13629 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:07.854138 14047 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:07.903308  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:07.903131  6451 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:07.881570 14059 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.907027  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:07.891240 13655 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:07.925143  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.926340  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.927006  6302 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:07.927860  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.928787  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:07.929692  6309 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:28:07.930043 13790 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:07.909857  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:07.908275  6310 raft_consensus.cc:2376] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22. Re-sending same reply.
I20260709 12:28:07.908651  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:07.883433 14114 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:07.912851  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.915355 14111 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:07.915912 14124 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:28:07.995316  6304 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:07.919044  6301 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:07.906692 14047 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:07.920542 13752 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.005s
I20260709 12:28:07.999912  6303 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:07.959501  6351 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:07.988929  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 80956us
I20260709 12:28:07.884719 14073 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
W20260709 12:28:08.003279 13772 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.007s	sys 0.013s
I20260709 12:28:07.942644 13790 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.011719  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.015336  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.016074  6378 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:08.015558  6454 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
W20260709 12:28:08.012107  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 185065us
I20260709 12:28:08.009748  6305 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 22. Ops: []
I20260709 12:28:08.013481  6457 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:07.998009 14080 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.012586 13752 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.023905  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:08.015848 13772 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.007s	sys 0.014s
I20260709 12:28:08.026041  6303 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.026962  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.028484  6381 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:08.030450  6379 pending_rounds.cc:85] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 16
I20260709 12:28:08.032184  6379 pending_rounds.cc:107] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:28:08.032498  6379 raft_consensus.cc:2889] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:08.009554 14111 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.016793 13713 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.998910 13655 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:28:08.019023 14128 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:08.029896 13791 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:28:08.105724 14131 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:08.037875 13868 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:08.099532  6274 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.742s (SENT)
W20260709 12:28:08.113471  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 280639us
I20260709 12:28:08.034190 14080 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.119472 14135 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.120015 14135 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.122298 14135 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.123008 14135 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.123592 14135 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.127007  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:08.131412  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 95964.5us
I20260709 12:28:08.091451 14111 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.111479 14128 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:08.146922 14128 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.149093 14128 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.150089 13752 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.150755  6381 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:08.097500 13713 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.131737 14129 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:08.138998  6385 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:08.143922 13845 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:08.144251 14111 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.144651  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.146023 14080 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.154155  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:08.173866  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 268181us
I20260709 12:28:08.174290  6456 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:08.173686  6301 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 14.
I20260709 12:28:08.179553  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.162731  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:08.181141  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.149003 14002 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:08.161446 13916 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.172813 14129 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.199445 14002 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.210829 14002 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:08.211959  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 118427us
I20260709 12:28:08.212580  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:08.213214  7314 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:08.213549  7314 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:08.206763  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:08.199847 13791 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.216189  6308 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:08.220451  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45736.9us
I20260709 12:28:08.219233  6383 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:08.173065 13873 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.224903  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.225497  7317 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 14.
I20260709 12:28:08.206784 13752 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.200747 14134 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:08.208528 14115 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:08.194435 14090 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.209880 14129 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:08.228567 14127 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
W20260709 12:28:08.241843  6274 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.887s (SENT)
I20260709 12:28:08.245213  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:08.247191  6274 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.994s (SENT)
W20260709 12:28:08.248208  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.248947 13845 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:08.250352 14133 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:08.251191 14133 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.253278 14133 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.250272  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:08.259140 14130 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:28:08.261216 14117 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.390s	user 0.003s	sys 0.000s
I20260709 12:28:08.261045 13888 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.242898  7314 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.223201 14131 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:08.239264 14126 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:28:08.226176 13748 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:08.265719  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59044.9us
I20260709 12:28:08.267060  7314 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:08.239078 14134 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:08.250478 13752 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.262677 13828 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:08.270624 13790 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:08.273706 14120 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
W20260709 12:28:08.274082 14116 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
I20260709 12:28:08.274523 14145 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:08.291615 14134 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.226789 13791 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.294636 14142 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.032s
I20260709 12:28:08.295706 13888 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.281486 13748 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.283228 13713 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.284976 13221 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.295373 13791 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.298343 13790 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.305835  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:08.306910  7314 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:08.316496 14090 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.317497 14090 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:08.319980 14138 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:28:08.323838 13873 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.327222  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:08.328354  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:08.329418 13221 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:08.329452  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.329886 13221 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.330294 13221 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:08.330806  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.333415  6385 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:08.334641 13752 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.337332 13772 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:08.337765 13772 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.338099 13772 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.338373  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.338387 13713 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.355216  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.360706  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.361647  6378 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:28:08.353757  6459 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.012s	sys 0.044s
I20260709 12:28:08.363780  6310 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:08.365381  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.392638  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.392500 14131 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.393373  6380 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.393545 14150 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:08.394683 13901 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.395064 14136 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.395262 13901 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:08.396881  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48135.1us
I20260709 12:28:08.398380  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.375877  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.371266  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:08.372093  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 82704.8us
I20260709 12:28:08.372817  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.401388  6385 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:08.372911  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.372162  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:08.402760  6458 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:08.403543  6453 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.034s	sys 0.051s
W20260709 12:28:08.403960  6453 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.035s	sys 0.051s
I20260709 12:28:08.407501  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.411873  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.413339  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:08.414328  6307 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:08.414846  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.395684 14136 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.415499  6309 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:08.339572 13713 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.431720  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.422160  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.396462 13849 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:28:08.436267  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.420913  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.440088  6456 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 13.
I20260709 12:28:08.447752  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.448390  6456 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:08.441025  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.441327 13752 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.442714 14135 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.444167  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.444068  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:08.449648  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.412266 13748 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:08.455464  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.456202  6456 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 13.
I20260709 12:28:08.452199 13221 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.454269  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.461567  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.462517  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.463304  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.464074  6301 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:08.455675 13748 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:08.444454  6452 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:08.465909  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.484092  6303 raft_consensus.cc:1217] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Deduplicated request from leader. Original: 15.13->[18.14-18.14]   Dedup: 18.14->[]
I20260709 12:28:08.484923  6303 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:28:08.497009  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.488358  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.493669 13221 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:08.503794  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.503868  6378 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:08.500528  6377 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:08.501754  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.502642  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.503293  7317 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:08.505911  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.503470  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.507277  6309 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:08.509181  6309 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:08.505576  6457 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:08.470343 13748 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:08.505419  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:08.470551 14109 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:28:08.484820  6425 connection.cc:256] RPC call timeout handler was delayed by -1.6946716070001457s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
I20260709 12:28:08.520193  6455 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:28:08.520804  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79868.3us
I20260709 12:28:08.513958  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:08.463155 14080 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:08.529489  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:08.529827  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 121291us
I20260709 12:28:08.532274  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.488421 13888 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.530979  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.531842 13221 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:08.533401 13221 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.533622  6303 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:08.533800 13221 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.459368 13828 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:28:08.549757  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.552073 13221 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.553203  6380 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:08.554757  6380 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:08.455677 14114 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.557160 13221 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.489143 13752 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.564769 13752 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.565433  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.567670  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:08.538035 13900 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.907s	user 0.016s	sys 0.002s
I20260709 12:28:08.560232  6457 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 15.
I20260709 12:28:08.577203  6458 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:08.575847  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:08.584090  6303 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:08.584592  6455 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:28:08.466765 14147 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.102s
I20260709 12:28:08.582969 14114 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.588792  6453 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:08.591912  6453 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:08.589017  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 65710.5us
I20260709 12:28:08.583035  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.592872  7317 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:08.592868  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.593456  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:08.596377  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:08.597030  6458 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:28:08.615777  6450 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:08.526356 14111 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:28:08.616773 14111 consensus_queue.cc:1243] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:28:08.556488 14012 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:28:08.622452  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.630805  6453 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.591626 13916 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:08.631961  6453 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:08.634104  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 171084us
I20260709 12:28:08.562335 13888 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:08.650259  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57472.8us
I20260709 12:28:08.615641 14147 consensus_queue.cc:1243] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:28:08.659847  6301 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:08.661768  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:08.639277 14033 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.732s	user 0.008s	sys 0.004s
I20260709 12:28:08.623888 13900 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:08.672945  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.677562  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.683000  6458 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 15.
W20260709 12:28:08.665141 14159 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:28:08.659847 13916 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.691671  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.697314  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.693707 14147 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:08.695684 14111 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:08.693534 13752 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:08.671443 14033 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.693328 14109 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:08.689429 14142 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:28:08.693179 14128 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.725080  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:08.735838  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:08.689833 13655 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:28:08.717867 14004 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.732577 14164 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:28:08.732328 14033 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.733047  7332 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:08.733783 14142 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:08.689661 14134 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:28:08.703356 13900 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:28:08.702131 14012 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:28:08.701488 13916 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:08.759817  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44045.2us
I20260709 12:28:08.727118 14111 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.734157 14109 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.768361  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.769945  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.770713  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.770963  6384 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:08.773579 14170 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:08.782626 14171 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.783263 14171 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.784452  6380 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:28:08.785583 14166 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:08.786121 14166 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:08.786525 14166 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:08.777840 13888 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.744204 13828 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:08.738893 14136 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.778581 13982 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.782155  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.787802  6385 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:08.785967  6380 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:08.744388 14059 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.800566  6382 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:28:08.790730 13828 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:08.793742 14122 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.890s	user 0.003s	sys 0.000s
I20260709 12:28:08.797076  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.803797 14115 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:08.804221 13982 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.804915  6384 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:08.803720 14163 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:28:08.809890 13901 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.744169 14128 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.776489 14111 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:08.816622 14059 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.819953  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.823702 14012 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.826674 14174 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.828085 14121 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.972s	user 0.004s	sys 0.000s
I20260709 12:28:08.828428 14115 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:08.816622 13900 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:08.864645  6383 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 22. Ops: []
I20260709 12:28:08.807938 13916 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.831897  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3310) took 1242 ms. Trace:
I20260709 12:28:08.834870 13868 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.816753 13828 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:08.827893  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.870636  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:08.834257 14090 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.516s	user 0.007s	sys 0.001s
I20260709 12:28:08.842805 14124 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.763799 14004 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.852429  6453 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
W20260709 12:28:08.854537  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.873498  6453 pending_rounds.cc:85] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 13
I20260709 12:28:08.873915  6453 pending_rounds.cc:107] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:28:08.856508  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.874166  6453 raft_consensus.cc:2889] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:08.859627 14012 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.880281 14012 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.881871 14004 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.882447 14124 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.883709 14123 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:08.884322 14123 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:08.884713 14124 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.861795  7314 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:08.885401 14123 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.860819 13982 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:08.859073 14047 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.847s	user 0.012s	sys 0.000s
W20260709 12:28:08.819214 14002 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.594s	user 0.008s	sys 0.004s
I20260709 12:28:08.862892  6380 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:28:08.869894  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.845338 14142 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.872016  6379 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:08.859153  7335 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:08.871317 13868 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:08.887714 13772 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.532s	user 0.007s	sys 0.004s
I20260709 12:28:08.918659 13772 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.919450  7329 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:08.919831 14059 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.914307  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:08.877969 14090 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:08.883894 14128 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.861387 14174 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.922019  6455 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:08.922049 13772 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.923913 14174 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.926644 14012 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.927477  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:08.927266 14012 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.928915 14004 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:28:08.929806 13991 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:08.929855 14004 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:08.930912  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.931088 14012 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.931720 14109 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:08.932339 14109 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.934324  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:08.934726  7332 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:28:08.935395 14131 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.936003 14131 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.939173 14131 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.892477 14002 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.941169  7335 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:08.933929 14115 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.944265 14080 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:08.946074 14080 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:08.887992  6453 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:08.890473 14047 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:08.947322 14167 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.349s	user 0.004s	sys 0.000s
I20260709 12:28:08.949447 14047 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.894268 13983 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.894342  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.894342  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.950865  6380 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:08.951834  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:08.894886 14142 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.959883  6304 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:08.961133  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.961748 14142 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.961809  6301 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:08.903677 13748 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.963796  6381 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:08.963896 14147 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:08.964247  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.868114 13900 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.964900  6452 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:08.965848  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:08.966711  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 461378us
I20260709 12:28:08.968232  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.968881  6450 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:08.969385  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.969751  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.970034  6307 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:08.900130  7318 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:08.970660 13888 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.970961  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.971248 13888 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.971645  6307 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
W20260709 12:28:08.972096  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:08.900002 13868 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:08.902647  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 45579ad5e67d4843a6f90bf59939a2d7: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:08.973233  7317 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:08.978089  6308 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:08.906006  6376 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:28:08.906698 13982 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:08.907545 14171 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:08.892742 14129 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.626s	user 0.006s	sys 0.006s
I20260709 12:28:08.910707  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.981743  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.982057  6383 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:08.909395 13713 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.913873  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:08.917656 14166 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:08.986539  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.986750  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.929208 14137 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.987908  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.988370  6302 pending_rounds.cc:85] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 9
I20260709 12:28:08.988277 14137 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:08.989007  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.989226  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.867640  6300 rpcz_store.cc:276] 0709 12:28:07.589383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.589823 (+   440us) service_pool.cc:224] Handling call
0709 12:28:07.606772 (+ 16949us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:07.607361 (+   589us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:07.607390 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:07.608087 (+   697us) log.cc:844] Serialized 52 byte log entry
0709 12:28:07.698921 (+ 90834us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:07.698980 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.699054 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:08.792680 (+1093626us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:08.831509 (+ 38829us) raft_consensus.cc:1685] finished
0709 12:28:08.831534 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:08.831732 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2251,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":9468,"spinlock_wait_cycles":4678528,"thread_start_us":90362,"threads_started":1,"wal-append.queue_time_us":213889}
I20260709 12:28:08.945513  6457 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:08.990585  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:08.950194  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:08.991050  7331 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:08.987931 14166 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.966681 13900 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:08.986747  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.990651 14137 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:08.944902 14002 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.876832  7320 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:08.993566  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.959340  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.989226  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.994800  6380 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:28:08.962805  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:08.983390  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69611.5us
I20260709 12:28:08.996280  6456 raft_consensus.cc:2410] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:08.983438  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:08.997108  6310 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:08.953295  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.970250 14090 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.900401  6378 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:08.900584  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.972627  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.895779 14123 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:08.972640  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:08.958091  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:08.974001  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:08.974890  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:08.977361  7329 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:08.955523  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:08.903753  6382 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:08.980270  6274 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:08.912091  7319 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
W20260709 12:28:09.009552  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54135.7us
I20260709 12:28:08.984827  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:09.010438  6308 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:09.010720 14114 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:09.012079  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.012409  7318 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:08.988600 13888 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:09.017423  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:08.973233 13873 raft_consensus.cc:2764] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:28:08.892994  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1186) took 1100 ms. Trace:
I20260709 12:28:08.964390 14147 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:09.026170  6380 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:08.966900 14150 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:09.023742  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:09.032001  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 74007.1us
I20260709 12:28:09.003249 14142 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:08.997794  6453 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:08.971711 14113 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:09.003280  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:09.038111  7332 raft_consensus.cc:2410] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:09.001183  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:08.995147  6459 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:09.010843 13752 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:09.013267  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:09.007112  6303 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:09.014117  6454 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:08.934356 14109 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:08.988765  6302 pending_rounds.cc:107] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:28:09.014777  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:08.991524  6300 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:09.042965  6302 raft_consensus.cc:2889] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:09.025202  6307 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:08.956570  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:09.020782  6305 rpcz_store.cc:276] 0709 12:28:07.792157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.792434 (+   277us) service_pool.cc:224] Handling call
0709 12:28:07.794752 (+  2318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:08.020255 (+225503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:08.892709 (+872454us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:08.892925 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":479863168}
I20260709 12:28:09.026576  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:08.990180  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:08.958763 13828 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.021127 14137 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:28:09.043394  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.020941 13868 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:09.026093 13713 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.040560 13752 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:09.057900  6376 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
W20260709 12:28:09.040987  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60805us
I20260709 12:28:09.042665  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:09.045477  6309 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:09.046141  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.021413 14115 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:09.006948 13900 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:28:09.038947 14142 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.037546 14113 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:09.067515  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52823.5us
I20260709 12:28:09.069155  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:09.026846 14114 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:28:09.062247  6453 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:09.041421 13982 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:28:09.002712 14129 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.042591 14170 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:09.013502 14090 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.051168 13868 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.074527  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.084012  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:09.084631  6453 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:09.079101  6379 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:09.084782  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:09.076232  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.045991 14171 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:09.048053 14150 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:09.091048 14135 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:09.091621 14171 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:09.091914 14150 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.092389  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:09.093076  6453 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.071493 14142 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:09.087391  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:09.080102 13790 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:28:09.083506  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:09.116451  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.086740 13791 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:09.148967  6455 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:09.152463 13791 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:09.152892 13791 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:09.153036 13766 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:28:09.153501  6455 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:09.153718 13766 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:09.145812  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:09.144758  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.083217 14109 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:09.086843 13868 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:09.154853 13766 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:09.156365  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69042.7us
W20260709 12:28:09.203243  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 150902us
I20260709 12:28:09.205333  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:09.094949 14149 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:09.063591 14174 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:28:09.063249 14136 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:28:09.157682 14168 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.095s
I20260709 12:28:09.159374 14090 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:09.160082 14188 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:28:09.160929 14129 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:09.163959  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80100us
W20260709 12:28:09.165899  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 164797us
I20260709 12:28:09.226166  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:09.167303  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.226997  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:09.227762  6456 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:09.227931  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:09.037546 14080 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.077183 13982 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.174706  6310 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:09.080296 14137 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:09.212906 13772 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:09.178692  7318 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:09.067711 13713 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.203152  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:09.176954 14115 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:09.217984  6453 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:09.096168 14133 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.111s
I20260709 12:28:09.229077  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.080403 14170 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.169234 14111 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.196748 14059 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:28:09.221362 14149 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:09.251127 14137 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.251271 14149 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.246641 14170 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:09.254602 14183 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:09.247058  7316 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:09.251891 13713 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:09.255623  6385 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:09.256547 14184 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:28:09.257450 14012 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:09.269325 14012 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.269781 14012 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:09.271250 14155 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.695s	user 0.000s	sys 0.003s
I20260709 12:28:09.272091 14145 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:09.258118 14181 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 12:28:09.244415 13888 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:09.275215  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:09.275770 14191 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:09.276162 13888 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:09.276881 13888 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:09.277562 14194 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.277733 13888 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:09.243600  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:09.278169 14194 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.279234 14184 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:09.279750 14184 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:09.280354 13713 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:09.280678 14184 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.282780 14090 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:09.284332 14090 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:09.213502 14073 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 12:28:09.281841 13868 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:09.265802  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.267215 14183 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:09.288551  6309 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:09.234400 13982 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:28:09.229689 13748 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:09.290966 14189 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:28:09.278146 13888 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:09.280678 14194 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.292284 13828 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:09.222321 14124 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:28:09.246901 14196 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:09.240432 13772 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:09.306994  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 268816us
I20260709 12:28:09.274416 14115 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:09.262933 13629 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:09.297196 14080 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:09.220247 13752 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.243513 14168 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
W20260709 12:28:09.327988  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 264534us
I20260709 12:28:09.326946  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:09.320708 14128 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.329312  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 268433us
I20260709 12:28:09.322743 13748 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:09.336112  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:09.324235 14136 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:09.326393 14196 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.327684 13772 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:09.212065 14113 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.351719  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.329075 13629 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:09.373308 14195 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.081s
I20260709 12:28:09.375507  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:09.340969 14136 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:09.377493 14136 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.378710 14136 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:09.379730 14203 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:09.380208 14203 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.380551 14203 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:09.373539 14174 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:09.381723 14012 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.387038  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:09.394634 14168 consensus_queue.cc:1175] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 21 status { last_received { term: 20 index: 12 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:09.395697 14113 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:09.396210 14168 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:09.396708 14168 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.395200 13748 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:09.397060 14168 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:09.399740 13748 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:09.400236 13748 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:09.399731 14174 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:09.402827  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.406042  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:09.410908  6452 raft_consensus.cc:2368] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:09.403088 14174 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.349488 14196 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:09.460242  6450 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:09.341616 13845 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:28:09.348858 14128 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.352895 14194 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.103s
I20260709 12:28:09.443594 14174 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:09.477015  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 123509us
W20260709 12:28:09.479003  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103760us
I20260709 12:28:09.464835 14124 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.421734 14115 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:28:09.480448  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:09.441403 13791 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.496330  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.404618 13888 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:09.497246  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 262801us
I20260709 12:28:09.476326 14178 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:09.469161 13772 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.506251  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:09.512833  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:09.513579 14198 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:28:09.521143  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:09.523931  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:09.504484 14149 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:09.524974  6309 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:09.527006  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 298015us
I20260709 12:28:09.533854  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:09.461556 14183 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.505568 14178 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:09.480273 14204 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:28:09.539888  6309 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:09.530050  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.582353  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:09.524107 14123 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.004s
I20260709 12:28:09.498066 14124 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.582916  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:09.533069 13845 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:09.495631 13629 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.533677 14149 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:09.538116 14205 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:09.595925  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:09.533190 13868 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.599666  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:09.606002  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:09.606657  6377 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.609052  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:09.589056 14199 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:28:09.589731 14194 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:09.614137  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.589579 13828 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.617486  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.622295  6381 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:28:09.624759  6379 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:09.592082 13713 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:09.631019  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:09.631692  6384 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:28:09.631935  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:09.633881  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.591281 14210 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:09.635052  6376 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:09.635929  6308 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:09.637140  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.637857  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:09.638628  6380 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:09.637962  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:09.639680  6301 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
W20260709 12:28:09.641029  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 128298us
I20260709 12:28:09.644070  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.644485  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:09.645512  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115582us
I20260709 12:28:09.646313  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.642014 14012 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.633349 14184 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:09.738754  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.642686 13983 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:09.744673  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:09.745995  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.639546 13916 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:28:09.752408 14216 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:09.753252 14206 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.753959  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.763767  6376 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:09.769851  6380 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
W20260709 12:28:09.770313  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.773568  6457 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:09.668769 13828 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.785539 14222 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:09.788942 14149 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.790716 13791 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:09.791405 14194 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.792014 14194 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:09.792840 14201 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.243s	user 0.003s	sys 0.007s
I20260709 12:28:09.793407  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:09.794047  6456 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:09.792294 13772 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.795674  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.651492 14210 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.805132  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:09.762307 14216 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.765275 13983 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:09.809458  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 159289us
I20260709 12:28:09.814474  6380 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:09.822744  6384 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:09.650828 14178 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.757407 14184 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.794288 14194 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.794576 14004 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.777664 14203 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.844852 14220 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:09.845495 14205 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.838836 14156 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:28:09.837844 14207 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.249s	user 0.001s	sys 0.001s
I20260709 12:28:09.846429 14205 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.838146 14213 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:09.848034 14213 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.848423 14213 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:09.852021 13828 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:09.838801 14211 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:28:09.852711 13828 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:09.853073 13828 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:09.823863 14149 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:09.855034 14220 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:09.843391 14004 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.855640 14220 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.841886 14169 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:09.857517 14169 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.857843 14004 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.845717 14203 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.859589 14169 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.859536 14220 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:09.862834 14208 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
W20260709 12:28:09.864037 14209 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.345s	user 0.001s	sys 0.000s
I20260709 12:28:09.840086 14218 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:28:09.867515 14210 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:09.804922 14206 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.872233  6383 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:09.873353 14206 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.824725 14216 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:09.879119  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.882900  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:09.886415  6384 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:09.896010  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.911895 14196 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:09.913167 14109 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.716s	user 0.012s	sys 0.000s
I20260709 12:28:09.913872 14109 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:09.854017 13828 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:09.915474 13748 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:09.839409 14200 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.292s	user 0.002s	sys 0.000s
I20260709 12:28:09.817451 14128 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.829160 14080 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.934366 14221 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:09.934523  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.935086 14221 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.935590  6459 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:09.937639 14221 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:09.939929  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.833333 14184 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:09.919849  6376 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:09.824709 13983 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:09.914211 14196 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.948035  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.922672  6377 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:09.957093  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.952427  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.959486  6380 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:09.952590  6456 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:09.950531  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:09.962688  6384 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:09.963183  6454 raft_consensus.cc:2410] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:28:09.963455  6383 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 12:28:09.942021 14142 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.786s	user 0.004s	sys 0.008s
I20260709 12:28:09.962381  6377 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:09.923321 13748 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.972628  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:09.966612  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.967885  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.973688  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.974436  6384 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22.
I20260709 12:28:09.977514  6459 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:09.978699  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.983238  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:09.843034 14214 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.127s
I20260709 12:28:09.954104 13713 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:09.995085  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:09.962393 14124 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.997077  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:09.946230 14194 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:28:10.001888  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:09.943161 14213 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.967825 14218 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.968011 14210 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:10.004112  6274 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22.
I20260709 12:28:10.006071 14135 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:10.076975 14135 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:10.008165 14205 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:09.992542 13845 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:10.018808  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet dd3cb6d1db5d481b9589e84d04e83d8e: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:09.974373 14142 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.020519 13888 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:10.021375 14168 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.000s
I20260709 12:28:10.024982  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:10.025310  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:10.026364  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.026365 14230 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.029240  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:10.029771 14115 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:28:10.093014  6377 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:10.080017 14183 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:10.095094  6377 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:10.096287  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71064us
W20260709 12:28:10.031687 13982 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.717s	user 0.007s	sys 0.004s
I20260709 12:28:10.031867 14220 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:10.033578  6451 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
W20260709 12:28:10.099830  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.100734  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.101461  6451 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:10.040114 13752 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:10.040709  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3330) took 1071 ms. Trace:
I20260709 12:28:10.041045  6348 leader_election.cc:400] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:10.042443 13828 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.049703 13791 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:10.050165 14111 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.051360 14150 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:10.104017 14002 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.123871 14213 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.078107 14135 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.103819  6302 rpcz_store.cc:276] 0709 12:28:08.969297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.987740 (+ 18443us) service_pool.cc:224] Handling call
0709 12:28:08.988105 (+   365us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:09.216581 (+228476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.216616 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:09.217201 (+   585us) log.cc:844] Serialized 52 byte log entry
0709 12:28:09.297164 (+ 79963us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:09.297203 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:09.297277 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:10.040487 (+743210us) raft_consensus.cc:1685] finished
0709 12:28:10.040509 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.040654 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8615,"raft.queue_time_us":90496,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":39526,"thread_start_us":228318,"threads_started":2,"wal-append.queue_time_us":124702}
I20260709 12:28:10.128669  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.129343  6302 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:10.052141  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:10.055510 14147 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.131434 14217 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:10.131531  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79474.9us
I20260709 12:28:10.131925 14217 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.132287 14217 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.057382 14216 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.973840 14080 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:09.977407 14145 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:09.959774  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:10.002683  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:10.072670  6274 leader_election.cc:403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:10.072082 14218 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.075524  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:10.072324 14210 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.180662 14210 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:10.182180 13629 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:10.082715  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:10.186275  6308 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:10.187562  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.188241  6308 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:10.059209 14170 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.804s	user 0.008s	sys 0.004s
I20260709 12:28:10.082943  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:10.058313 14136 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.003s
I20260709 12:28:10.090082  6307 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.190388 14170 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.103787 13982 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.001415 13766 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:28:10.097286  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:10.084843 13845 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.103888  6348 leader_election.cc:403] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:10.112589 14205 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:28:10.108646  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79454.9us
I20260709 12:28:10.203555  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:10.096549 14183 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:10.146438  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.209501  6452 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.209903  6451 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
W20260709 12:28:10.154417  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 157290us
W20260709 12:28:10.154691  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.211690  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.154992 14213 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:10.157612  6385 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:28:10.158587  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1138) took 1276 ms. Trace:
I20260709 12:28:10.139266 13790 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:10.215538  7317 rpcz_store.cc:276] 0709 12:28:08.863380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:08.863523 (+   143us) service_pool.cc:224] Handling call
0709 12:28:10.139466 (+1275943us) spinlock_profiling.cc:243] Waited 83 ms on lock 0x7b54005dacd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:10.139608 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190996736}
I20260709 12:28:10.158919 14216 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.162456  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.162822  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.221702  6455 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:10.165611 14080 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.139475 14137 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:28:10.169704 14220 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:10.169850 13713 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:10.171330 14142 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:10.173203 14166 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:10.236418  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:10.092754  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67900.9us
I20260709 12:28:10.246678  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:10.192799  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.247818  6456 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:10.193358  6381 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:28:10.102376 14090 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:28:10.201597  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68506us
I20260709 12:28:10.253150  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:10.206388  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 165465us
I20260709 12:28:10.254287  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:10.206694  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109524us
I20260709 12:28:10.205132  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:10.207670  6302 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:10.205930 13845 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:10.102782 13916 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:10.282987  6382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.018s	sys 0.037s
I20260709 12:28:10.213488  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:10.283489  6382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.018s	sys 0.037s
I20260709 12:28:10.214632  6300 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 15: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:10.091991 14168 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.218034  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:10.163235 14145 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.098918 14230 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.228245  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:10.206755 14205 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:10.208734 13791 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:10.213524 14047 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:10.297928  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:10.192633 14170 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.301772  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.244844  6376 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: [27.19-28.20]
I20260709 12:28:10.214661 14004 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.219600 13790 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:10.308317 13790 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.208868 14150 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:28:10.312676  7316 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.258100  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:10.314357  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56349.2us
I20260709 12:28:10.296356 13900 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:10.264736 14142 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:10.267007 14147 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:10.236429 13772 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:10.272225 14137 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:10.245162 14166 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:10.250149 14090 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:10.280061  6456 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:10.281342  6452 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:10.347333  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:10.348551  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.310307 13791 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:10.285660  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:10.288491  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75111.2us
I20260709 12:28:10.293656  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:10.237188 14220 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.287204  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:10.284631 13849 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:10.262588  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.375334  6384 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:10.306977 13916 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:28:10.318699  7331 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:10.321661  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:10.382717  6450 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:10.295544 13713 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.325361  6455 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:10.348388  7317 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:10.355576  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:10.356563  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.359258  6383 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:10.361492  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:10.302523 14012 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:28:10.368687  7319 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 12:28:10.364646  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 77530.1us
I20260709 12:28:10.281605  6307 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:10.373648  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:10.398530  6454 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:10.399063  6310 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:10.297284 14195 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:10.304258 14174 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.823s	user 0.014s	sys 0.000s
I20260709 12:28:10.335865 14142 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.409219  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:10.378659  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:10.296588 14183 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.346509 14137 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.411859  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:10.420358  6456 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:28:10.420979  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 215928us
I20260709 12:28:10.448182  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:10.355741 14186 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.462760  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.465687  7332 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:10.336675 13772 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.379837 13849 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.506006  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:10.404449 13991 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.517755  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.496769  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.520426  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.497766  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:28:10.495003  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85866.1us
I20260709 12:28:10.383848 14216 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:10.364363 13900 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:10.501652  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:10.324049 14004 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.511426  7319 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:28:10.396986 14073 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:10.522265  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:10.397575 14169 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:10.473825 14142 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:10.536157 14216 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:10.542553  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:10.504252 14137 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:10.476327 14012 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:10.563351 14236 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:28:10.563778 13655 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:10.505764 13772 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:10.515379 13982 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:10.543546 14169 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.559069 14216 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:10.560784 14233 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:28:10.511831 13849 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:10.509672 14135 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:10.508239 14111 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:10.516126 14186 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:10.595247  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 343308us
I20260709 12:28:10.617823  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:10.593638  6308 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:10.415122 13713 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:10.537694 14229 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:28:10.358307 14205 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.525000 14150 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:10.554885 14226 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:28:10.465123 14174 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.546815 14114 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:10.507519 14212 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.067s
I20260709 12:28:10.564399 13655 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:10.624223  6450 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:28:10.624735  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 427500us
I20260709 12:28:10.629508  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.629575  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1200) took 1087 ms. Trace:
I20260709 12:28:10.645833  6305 rpcz_store.cc:276] 0709 12:28:09.542202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.543757 (+  1555us) service_pool.cc:224] Handling call
0709 12:28:09.593265 (+ 49508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:09.833806 (+240541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.833853 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:09.833912 (+    59us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:09.833957 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:10.629328 (+795371us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54002de6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:10.629376 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:10.629520 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552567680}
W20260709 12:28:10.631469  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.631695  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 179607us
I20260709 12:28:10.541483 13991 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:10.615162 14228 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.354s	user 0.000s	sys 0.001s
I20260709 12:28:10.567023 14131 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:28:10.503705 14166 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:10.647565  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.648147  6308 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:10.565325 14073 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.564188 14012 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.651937  6302 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:10.647671  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:10.663257 14251 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:10.669763  6301 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:28:10.639714 14111 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:10.544793 14090 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.682791  6308 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:10.654973 14168 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:10.761341  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 60505.6us
I20260709 12:28:10.741446  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:10.763898  6379 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:10.735579  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:10.640453 14150 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.658238 14145 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:10.635830 14246 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
W20260709 12:28:10.767571  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:10.767030  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:10.609930 13791 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.778546  6308 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:10.675872 13629 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:28:10.728174 14184 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.753s	user 0.007s	sys 0.004s
I20260709 12:28:10.743232 14220 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:10.722285 14234 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:28:10.771605 14166 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:10.771052 13983 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.807s	user 0.013s	sys 0.000s
I20260709 12:28:10.779690  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:10.803540  6308 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:10.735750 14169 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.721369 14251 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:10.737133 14186 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.677161 14230 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.769924  6303 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:10.720613 13655 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.812153  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:10.676088 13900 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.794914 14184 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:10.799412 14002 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.675s	user 0.013s	sys 0.000s
W20260709 12:28:10.816532  7316 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.000s
I20260709 12:28:10.845820  7316 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:10.786454 13629 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:10.721769 14073 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.772550 13982 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:28:10.794309 14171 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.842408  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 220570us
I20260709 12:28:10.771237 14142 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.865854  6380 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:10.729825 13772 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.885761  6380 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:10.845280 14168 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.849081 13629 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:10.849380 14002 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:10.809002 13983 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:10.897068  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:10.853103 13916 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.858566 14210 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.677s	user 0.013s	sys 0.000s
W20260709 12:28:10.934516  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:10.981474  6383 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.003s
I20260709 12:28:10.981990  6383 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:10.881263 14145 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:10.831772 14111 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:10.990670  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 93735.9us
I20260709 12:28:10.989889  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:10.992087  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:10.992247  6380 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:10.947324 14004 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:11.011426  7286 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.004s	sys 0.020s
W20260709 12:28:11.011801  7286 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.004s	sys 0.020s
W20260709 12:28:10.986474 13888 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.889s	user 0.004s	sys 0.009s
I20260709 12:28:10.935281 14249 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.993523  7288 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.665s (SENT) [suppressed 29 similar messages]
W20260709 12:28:10.913457 14217 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.781s	user 0.013s	sys 0.000s
I20260709 12:28:10.947779 13713 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.984493 14254 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:11.006837 14220 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:10.880592 14171 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.983428 14131 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.911039 14168 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:10.854063 14251 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.009073 14169 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:11.029908  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.023581 14230 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.010025 14114 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:10.978368 14210 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.048507  6454 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 25 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:11.008170 14123 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.023494 13991 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.052600 14171 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:10.983659 13916 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:11.052985 13790 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.744s	user 0.010s	sys 0.001s
W20260709 12:28:11.059350 14213 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.840s	user 0.012s	sys 0.000s
I20260709 12:28:11.047293 14131 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.065550 14168 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.067914 14184 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.073326 14169 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.074668 14150 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.034953 14254 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:11.029201 14012 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.007684 14145 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.035475 14217 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.084738  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1199) took 1545 ms. Trace:
I20260709 12:28:11.086033  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:11.090111  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.157961  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:11.159379  6300 rpcz_store.cc:276] 0709 12:28:09.539528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.539714 (+   186us) service_pool.cc:224] Handling call
0709 12:28:11.084515 (+1544801us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540025add4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.084672 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443744640}
I20260709 12:28:11.121165  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:11.164534  6458 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:11.176733  6454 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:28:11.144706 13849 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.515s	user 0.009s	sys 0.004s
W20260709 12:28:11.180747  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 187240us
I20260709 12:28:11.184777  6454 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:28:11.047326 13713 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.197036 13849 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.199386 13849 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:11.200733 14257 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:28:11.201556  6451 raft_consensus.cc:2368] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:28:11.201953  6308 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:11.202207  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:11.204172  6452 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:11.206904  6300 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:11.172106 14213 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22.
I20260709 12:28:11.210053  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3349) took 1126 ms. Trace:
I20260709 12:28:11.213191  6300 rpcz_store.cc:276] 0709 12:28:10.083748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.173588 (+1089840us) service_pool.cc:224] Handling call
0709 12:28:11.209955 (+ 36367us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.219082  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:11.219378  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.220165  7335 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:11.181805 14012 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:11.172736 13790 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.224443  7320 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:11.226593  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 933586us
W20260709 12:28:11.210757 13845 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.912s	user 0.009s	sys 0.005s
I20260709 12:28:11.223722  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.175199 14123 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.106102 14230 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.235793  6309 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:11.175061 14169 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.036345 14249 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.238761  6458 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:11.244284  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1210) took 1101 ms. Trace:
I20260709 12:28:11.244411  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:11.245675  6300 rpcz_store.cc:276] 0709 12:28:10.142835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.214089 (+1071254us) service_pool.cc:224] Handling call
0709 12:28:11.244212 (+ 30123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.254081  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:11.257051  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.215258 13713 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.237383 13655 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.259413  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3026) took 1377 ms. Trace:
I20260709 12:28:11.262812  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:11.263484  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.267933  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.268615  7319 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:28:11.244252 14142 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.276964  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.278384  6458 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:28:11.270685  6300 rpcz_store.cc:276] 0709 12:28:09.881541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.253716 (+1372175us) service_pool.cc:224] Handling call
0709 12:28:11.259328 (+  5612us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.279870  6459 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:11.260306 14186 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.309073 14186 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.310519  6458 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:11.311174 14186 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.311923 14186 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:11.284401  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.313040 14186 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:11.314715 14168 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:11.315286 14168 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.315963  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.316680  7318 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:11.239082 13845 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.320380 13845 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:11.287277 14147 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.002s
I20260709 12:28:11.323110  6382 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:11.323313 14261 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:11.323668 14251 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:11.288208  6454 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:11.214354 14114 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.325461  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:11.326628 14142 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:11.327163 14142 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:11.328068 14142 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.277418 14150 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:11.329552 14150 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.331372 14217 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.332233 14150 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.291767 14149 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:11.332870 14268 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:11.333132 14149 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:11.334576 14220 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:11.334312 14149 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.335897 14149 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.231769 14012 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.299656 14169 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.300071 14249 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.301024 14230 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.301846 14145 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:11.296948  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.267483 14254 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.289541 13790 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.269807 13791 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.303663 13713 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.306391 14263 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.338820 13791 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:11.306728 14166 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.002s
I20260709 12:28:11.307080 14123 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.308514 14210 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.278106 13655 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:11.312342 14137 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.750s	user 0.011s	sys 0.003s
I20260709 12:28:11.315603 14212 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:11.338243 14254 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:11.339075 14263 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.305747 14248 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:11.360108 14248 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:11.360739 13752 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:11.361176 13752 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:11.361559 14263 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.308018 14131 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.361442 14248 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.361980 13752 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.304200 13629 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.363144 13752 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:11.362732 14212 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.343088  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3344) took 1372 ms. Trace:
I20260709 12:28:11.364156  6309 rpcz_store.cc:276] 0709 12:28:09.970825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.517419 (+546594us) service_pool.cc:224] Handling call
0709 12:28:11.342819 (+825400us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b54001b21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.342996 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1618392960}
I20260709 12:28:11.365193  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1203) took 1773 ms. Trace:
I20260709 12:28:11.365301  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:11.366101  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3354) took 1074 ms. Trace:
I20260709 12:28:11.366429  6309 rpcz_store.cc:276] 0709 12:28:10.292295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.365031 (+1072736us) service_pool.cc:224] Handling call
0709 12:28:11.366036 (+  1005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.367457  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.368153  6309 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:11.368530  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3024) took 1487 ms. Trace:
I20260709 12:28:11.368959 14090 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:11.368803  6309 rpcz_store.cc:276] 0709 12:28:09.881159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.367201 (+1486042us) service_pool.cc:224] Handling call
0709 12:28:11.368481 (+  1280us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.371894  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.372860  6309 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 16.
I20260709 12:28:11.374145  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.374876  6309 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:11.317615 14168 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.275327 13849 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:11.352262  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.355434  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:11.348681 14274 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.054s	user 0.003s	sys 0.000s
I20260709 12:28:11.377334  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:11.384658  6300 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:11.375941  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:11.348488 14277 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.003s
I20260709 12:28:11.379125  6458 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:11.377110 13629 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.389817  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:11.391261 14218 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.341149 14111 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:11.348937 14237 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.812s	user 0.004s	sys 0.000s
W20260709 12:28:11.348836 14265 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.059s	user 0.003s	sys 0.000s
I20260709 12:28:11.365655  6304 rpcz_store.cc:276] 0709 12:28:09.592108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.592353 (+   245us) service_pool.cc:224] Handling call
0709 12:28:09.592754 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:09.740118 (+147364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:09.740148 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:09.755638 (+ 15490us) log.cc:844] Serialized 26 byte log entry
0709 12:28:09.961464 (+205826us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:10.390743 (+429279us) log.cc:844] Serialized 14 byte log entry
0709 12:28:10.391102 (+   359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.262208 (+871106us) spinlock_profiling.cc:243] Waited 6.06 ms on lock 0x7b18003d1560. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419
0709 12:28:11.364907 (+102699us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b54001b3fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:11.364962 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.365006 (+    44us) raft_consensus.cc:1685] finished
0709 12:28:11.365017 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.365139 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131606,"prepare.queue_time_us":533313,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":122596,"raft.run_cpu_time_us":349,"raft.run_wall_time_us":61167,"spinlock_wait_cycles":72173056,"thread_start_us":503048,"threads_started":2,"wal-append.queue_time_us":365184}
I20260709 12:28:11.386009  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3347) took 1392 ms. Trace:
I20260709 12:28:11.401238  6309 rpcz_store.cc:276] 0709 12:28:09.993963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.375667 (+1381704us) service_pool.cc:224] Handling call
0709 12:28:11.385939 (+ 10272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.405923  6309 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:11.406836  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3340) took 1521 ms. Trace:
I20260709 12:28:11.407306  6309 rpcz_store.cc:276] 0709 12:28:09.885895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.402759 (+1516864us) service_pool.cc:224] Handling call
0709 12:28:11.403155 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.406459 (+  3304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.406768 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.313925 14186 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.410573  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3342) took 1524 ms. Trace:
W20260709 12:28:11.414896  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 139836us
I20260709 12:28:11.416512  6309 rpcz_store.cc:276] 0709 12:28:09.886599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.408838 (+1522239us) service_pool.cc:224] Handling call
0709 12:28:11.409709 (+   871us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.410259 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.410285 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:11.410332 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:11.410365 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.410415 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.410524 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.421170  6309 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: [20.13-22.14]
I20260709 12:28:11.370741 14137 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.425348  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:11.427608  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3027) took 1527 ms. Trace:
I20260709 12:28:11.428583 14218 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:28:11.430845  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:11.441812  6309 rpcz_store.cc:276] 0709 12:28:09.897095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.420435 (+1523340us) service_pool.cc:224] Handling call
0709 12:28:11.420858 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:11.422879 (+  2021us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.423130 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.430994  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.432343 13655 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.433521  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1385) took 1091 ms. Trace:
I20260709 12:28:11.409549 14111 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.435774 14137 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.436359  6456 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:11.436836 13791 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:11.437538 14262 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:28:11.388391 13849 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.437978 13916 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:11.439882  6310 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3339) took 1555 ms (client timeout 1814 ms). Trace:
I20260709 12:28:11.440392  6304 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 17. Current term is 18. Ops: []
W20260709 12:28:11.443137  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.441323 14212 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:11.518805  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:11.602679  6310 rpcz_store.cc:269] 0709 12:28:09.884078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.227898 (+343820us) service_pool.cc:224] Handling call
0709 12:28:11.439717 (+1211819us) spinlock_profiling.cc:243] Waited 26.5 ms on lock 0x7b540010fc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.439830 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":60930816}
W20260709 12:28:11.462503  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.451457  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:11.491562  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.448485 14123 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.489775  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.494067  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.494756  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.490275  7314 rpcz_store.cc:276] 0709 12:28:10.342061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.342225 (+   164us) service_pool.cc:224] Handling call
0709 12:28:10.342839 (+   614us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:10.612496 (+269657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.612534 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:10.652080 (+ 39546us) log.cc:844] Serialized 52 byte log entry
0709 12:28:11.006234 (+354154us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:11.006290 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.006382 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.433270 (+426888us) raft_consensus.cc:1685] finished
0709 12:28:11.433292 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.433442 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62,"raft.queue_time_us":318108,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":213,"spinlock_wait_cycles":2432,"thread_start_us":539122,"threads_started":2,"wal-append.queue_time_us":568320}
I20260709 12:28:11.495361  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.606936  7335 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:11.609086  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.611512  6380 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:11.612731  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.607276  6383 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:11.442606 14249 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:11.606866  6309 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:11.493676 14192 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:11.617601  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 166230us
I20260709 12:28:11.628695  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:11.629850 14218 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:11.630398  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:11.618302  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.622504 14279 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:11.622709 14216 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:11.528031 14280 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:11.615437 14123 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.633558 14280 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:11.634323 14280 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:11.634894  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1380) took 1305 ms. Trace:
I20260709 12:28:11.635392 14123 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.622931 14249 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.637547  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3458) took 1419 ms. Trace:
I20260709 12:28:11.638069  6384 rpcz_store.cc:276] 0709 12:28:10.218064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.218230 (+   166us) service_pool.cc:224] Handling call
0709 12:28:11.637307 (+1419077us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b5400064854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.637440 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194116992}
I20260709 12:28:11.638530  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.638922 13849 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.640352  6377 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:11.635419  7329 rpcz_store.cc:276] 0709 12:28:10.329428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.346705 (+ 17277us) service_pool.cc:224] Handling call
0709 12:28:10.371681 (+ 24976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:10.606196 (+234515us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.606242 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:10.613053 (+  6811us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.009004 (+395951us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:11.009053 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.009129 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:11.634587 (+625458us) raft_consensus.cc:1685] finished
0709 12:28:11.634620 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.634802 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":289710,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"thread_start_us":589625,"threads_started":2,"wal-append.queue_time_us":591026}
I20260709 12:28:11.492612 14012 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.632841 14216 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.642786 14115 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:11.643316 14115 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:11.630436 14218 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:11.649937  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:11.656848  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:11.527675 13655 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:11.652189  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.657975  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.606305 14284 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:11.637944 14249 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.654799 14192 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.666417  6383 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:11.667085 14047 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:11.670197 14178 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:11.670222 14047 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:11.671101 14192 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.671098 14047 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:11.663180 14012 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:11.675931 14183 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:11.644179 14216 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.667053 14123 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:28:11.679633 14047 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.682073 14047 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.682732  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.651599 14218 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:11.695219  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3452) took 1767 ms. Trace:
I20260709 12:28:11.695863  6381 rpcz_store.cc:276] 0709 12:28:09.927441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.929400 (+  1959us) service_pool.cc:224] Handling call
0709 12:28:09.938893 (+  9493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:10.248791 (+309898us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.694812 (+1446021us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b5400194bd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:11.695125 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584557312}
I20260709 12:28:11.651374 14268 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.703253  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3167) took 1741 ms. Trace:
I20260709 12:28:11.711781  6385 rpcz_store.cc:276] 0709 12:28:09.961661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.013803 (+ 52142us) service_pool.cc:224] Handling call
0709 12:28:10.014127 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:10.215238 (+201111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.702820 (+1487582us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54000af5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:11.703146 (+   326us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329388288}
I20260709 12:28:11.720330 14249 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.720917 14249 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.723038  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3439) took 1317 ms. Trace:
I20260709 12:28:11.723637 14249 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:11.664651 13900 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:28:11.723485  6457 rpcz_store.cc:276] 0709 12:28:10.405585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.644788 (+239203us) service_pool.cc:224] Handling call
0709 12:28:10.667924 (+ 23136us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.787417 (+119493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.787451 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:10.787497 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:10.787529 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.722758 (+935229us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b54001626d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:11.722805 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:11.722947 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":274229632}
I20260709 12:28:11.732203 14216 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:11.732796 14216 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.733868 14147 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.735558  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3023) took 1854 ms. Trace:
I20260709 12:28:11.736052  6304 rpcz_store.cc:276] 0709 12:28:09.880892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.401559 (+1520667us) service_pool.cc:224] Handling call
0709 12:28:11.440233 (+ 38674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.605330 (+165097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.735383 (+130053us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b540010fc50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.735502 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87956352}
I20260709 12:28:11.740042  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:11.740887  6379 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:11.755167 13849 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:11.756275  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.763222  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.763979  6377 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:11.777230  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:11.777475  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.778620  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.779418  6455 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:11.779347 13791 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:11.780519  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.781080 14249 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:11.781456 14249 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:11.734408 14147 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.782305 14249 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.786239  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3337) took 1964 ms. Trace:
I20260709 12:28:11.788210  6301 rpcz_store.cc:276] 0709 12:28:09.822246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.822388 (+   142us) service_pool.cc:224] Handling call
0709 12:28:09.850816 (+ 28428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:10.674061 (+823245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.785887 (+1111826us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b540010f754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:11.786156 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1882607872}
I20260709 12:28:11.833362  6379 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:11.825665  6300 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
W20260709 12:28:11.838356  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.838639  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.839602  6301 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:11.839550  6458 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
W20260709 12:28:11.840847  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3031) took 1760 ms (client timeout 2165 ms). Trace:
W20260709 12:28:11.841334  6300 rpcz_store.cc:269] 0709 12:28:10.087033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.344416 (+1257383us) service_pool.cc:224] Handling call
0709 12:28:11.840684 (+496268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.786626 13752 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:11.705344 13655 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.827265 14249 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:11.662478 14115 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.711230 14284 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.730119 14268 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.859014  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:11.867834  6455 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:11.836442 13900 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:11.851826  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:11.873345  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3021) took 2035 ms (client timeout 1978 ms). Trace:
W20260709 12:28:11.875314  6302 rpcz_store.cc:269] 0709 12:28:09.831234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.464769 (+633535us) service_pool.cc:224] Handling call
0709 12:28:11.865875 (+1401106us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001b35d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.866074 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333937536}
I20260709 12:28:11.860361 14216 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.878213  6302 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:11.880784  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3030) took 1811 ms. Trace:
I20260709 12:28:11.842823  6300 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:28:11.884145  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.886453  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3029) took 1817 ms. Trace:
I20260709 12:28:11.887995  6379 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:11.891853  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3432) took 1850 ms (client timeout 2126 ms). Trace:
W20260709 12:28:11.898694  6451 rpcz_store.cc:269] 0709 12:28:10.041905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.301420 (+259515us) service_pool.cc:224] Handling call
0709 12:28:11.891562 (+1590142us) spinlock_profiling.cc:243] Waited 899 ms on lock 0x7b540033b154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.891771 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066974464}
I20260709 12:28:11.892093  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3463) took 1671 ms. Trace:
I20260709 12:28:11.881248  6302 rpcz_store.cc:276] 0709 12:28:10.069186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.876472 (+1807286us) service_pool.cc:224] Handling call
0709 12:28:11.877942 (+  1470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.880407 (+  2465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.880705 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.861529 13752 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.905864 13752 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:11.888501  6300 rpcz_store.cc:276] 0709 12:28:10.068816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.842112 (+1773296us) service_pool.cc:224] Handling call
0709 12:28:11.842438 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.883748 (+ 41310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.886319 (+  2571us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:11.905922  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 245050us
I20260709 12:28:11.907310  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:11.901094  6382 rpcz_store.cc:276] 0709 12:28:10.220442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.289896 (+ 69454us) service_pool.cc:224] Handling call
0709 12:28:11.891866 (+1601970us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54003e7e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:11.892015 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2356168064}
W20260709 12:28:11.915992  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.903172  6302 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:11.884001 14115 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:11.926249  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3032) took 1837 ms. Trace:
W20260709 12:28:11.927031  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.927124  6302 rpcz_store.cc:276] 0709 12:28:10.089321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.902181 (+1812860us) service_pool.cc:224] Handling call
0709 12:28:11.902690 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.925870 (+ 23180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.926177 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.928777  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.929636  6300 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:11.908592  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:11.930277  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.930527  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3033) took 1834 ms. Trace:
I20260709 12:28:11.931041  6379 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:11.931044  6300 rpcz_store.cc:276] 0709 12:28:10.096029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.907483 (+1811454us) service_pool.cc:224] Handling call
0709 12:28:11.929204 (+ 21721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.930195 (+   991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.930443 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.932104  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.932521  6379 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:11.933310  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.934477  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:11.935497  6379 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:11.877619 14280 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:11.879381 13900 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:11.938982  6302 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:11.941175  6300 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:11.940066  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:11.942061  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.941220  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3351) took 1723 ms. Trace:
I20260709 12:28:11.943259  6302 rpcz_store.cc:276] 0709 12:28:10.218102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.928219 (+1710117us) service_pool.cc:224] Handling call
0709 12:28:11.938773 (+ 10554us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.940869 (+  2096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.941146 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.945451 14284 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:11.930008 14012 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:11.946655  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3352) took 1720 ms. Trace:
I20260709 12:28:11.947178  6300 rpcz_store.cc:276] 0709 12:28:10.224259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.932343 (+1708084us) service_pool.cc:224] Handling call
0709 12:28:11.940985 (+  8642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.941896 (+   911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.944393 (+  2497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:11.943074  6379 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:11.949259  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.949790  6379 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:11.951514  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:11.952327  6382 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:11.959779  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:11.963060  6380 raft_consensus.cc:2368] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18.
I20260709 12:28:11.963994  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:11.966022  6459 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:11.965606  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.968964  7319 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:11.973114  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:11.973603  6300 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:11.974783  6379 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:11.943912 14291 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:11.971537  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:11.984573  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:11.980356  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:11.982196  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:11.988343  6302 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:12.010567  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.011202  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.011524  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.013175  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:12.014317  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.016860  7329 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:11.954037  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:12.018184  6451 raft_consensus.cc:606] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:12.020761  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:11.969475 14142 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:12.020890  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.013821  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:11.967278 14012 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:12.029877  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.034022  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:12.035046  6302 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:11.950428 13752 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:28:12.033846  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:11.975935 14212 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:11.983724 14291 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:12.051115  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 98098us
I20260709 12:28:12.019907 14131 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:28:12.053326  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70242us
I20260709 12:28:12.019641 13900 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.054044  6300 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:12.020248 14263 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:12.059342  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:12.044389 14254 consensus_meta.cc:287] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.684s	user 0.008s	sys 0.005s
I20260709 12:28:12.030210 13849 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:12.084856  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:12.088951  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:12.052260 14285 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.224s	user 0.001s	sys 0.001s
I20260709 12:28:12.039376 14137 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:12.093302  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:12.100785  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.101616  7314 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:12.101966  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:12.093532  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1201) took 2508 ms (client timeout 1665 ms). Trace:
I20260709 12:28:12.094971  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:12.108670  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48296us
I20260709 12:28:12.112244  6307 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:12.112985  7319 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:12.113554  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:12.114396  6451 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 26
W20260709 12:28:12.107260  6303 rpcz_store.cc:269] 0709 12:28:09.584777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.585581 (+   804us) service_pool.cc:224] Handling call
0709 12:28:12.093296 (+2507715us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002d19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:12.093410 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2395605760}
I20260709 12:28:12.116552  6303 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:12.117301  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3034) took 1862 ms. Trace:
I20260709 12:28:12.117734  6303 rpcz_store.cc:276] 0709 12:28:10.255488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.115709 (+1860221us) service_pool.cc:224] Handling call
0709 12:28:12.116106 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.117008 (+   902us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.117232 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:12.046409 14133 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:12.124933  7335 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:12.053964 14291 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:12.125918  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.059794 14216 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:12.125828  6300 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:12.127269  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.129860  7329 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:12.125953  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:12.135282  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.135906  6303 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:28:12.087980 14263 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:12.136637  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3035) took 1880 ms. Trace:
I20260709 12:28:12.091527 13849 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.142174  6303 rpcz_store.cc:276] 0709 12:28:10.255831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.119763 (+1863932us) service_pool.cc:224] Handling call
0709 12:28:12.135619 (+ 15856us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:12.136400 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.136579 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:12.100469 13655 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.111167 14142 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:12.148313  6348 leader_election.cc:400] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:28:12.140187  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:12.112125 14137 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.060686 14284 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:28:12.188951  6348 leader_election.cc:403] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:28:12.124960 14136 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.189855  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 146104us
W20260709 12:28:12.168843  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.125290 14133 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.125691 14254 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.191654  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52621.1us
I20260709 12:28:12.059762 14212 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:12.195210  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47338.5us
W20260709 12:28:12.145402 14286 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
W20260709 12:28:12.153074 14289 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.273s	user 0.001s	sys 0.001s
I20260709 12:28:12.143685 13828 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:12.203105  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:12.125027 13791 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:28:12.151239 13849 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:12.204330  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3028) took 2269 ms. Trace:
I20260709 12:28:12.218737  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.194331 14137 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:12.192351 14268 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:12.235085  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:12.278976  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62292.7us
W20260709 12:28:12.311882  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 108878us
I20260709 12:28:12.348338  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:12.344480  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.302007 13828 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.216624  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.301571  6304 rpcz_store.cc:276] 0709 12:28:09.924011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.755219 (+1831208us) service_pool.cc:224] Handling call
0709 12:28:11.755506 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.755687 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:11.903995 (+148308us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.192716 (+288721us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.193061 (+   345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:12.193182 (+   121us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:12.193270 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.193393 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.193596 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":178844,"prepare.queue_time_us":225537,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":111,"raft.queue_time_us":278338,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":11110,"thread_start_us":394805,"threads_started":3,"wal-append.queue_time_us":236425}
I20260709 12:28:12.201802 14133 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:12.146663 14263 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:12.355108  6302 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:28:12.252724  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.361873  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:12.362517  6453 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:12.380633  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.204572 14136 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:12.316916 14090 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.948s	user 0.014s	sys 0.001s
I20260709 12:28:12.375802  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.185428 14012 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.381304  6453 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:12.385844  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.385913  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.386439  6453 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:12.387887  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.388934  6453 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:12.390084  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.390861  6453 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 17: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:12.196810 13655 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:12.145938 14216 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.363034 13828 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:12.403568  6382 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:12.397471  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.406342  6451 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:12.316220 14186 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20.
I20260709 12:28:12.399734  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:12.362946 14131 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:12.410290 14149 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.074s	user 0.013s	sys 0.000s
I20260709 12:28:12.411275 13900 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:12.411785 13900 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:12.412652 13900 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:12.416949 14218 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.002s
I20260709 12:28:12.417379 14218 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:12.418035 14299 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
W20260709 12:28:12.419252 14298 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:28:12.421911 14313 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:12.422472 14313 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.424614 14313 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:12.400063 14114 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.426669 14297 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:28:12.401232 13655 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:12.427379 13655 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:12.426720 14114 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:12.432410 14300 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:28:12.433358 14114 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.434410 14131 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.316433 14303 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:12.402938  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.210332 14254 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.197013 14142 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.441267 14212 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.442348 14218 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:12.442452 14284 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.444743  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3430) took 2416 ms. Trace:
I20260709 12:28:12.445354 14136 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:12.448312  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:12.471203  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.468612  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.468420  6456 rpcz_store.cc:276] 0709 12:28:10.028181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.542191 (+514010us) service_pool.cc:224] Handling call
0709 12:28:12.444537 (+1902346us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b5400344c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:12.444676 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2031858560}
I20260709 12:28:12.476442  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:12.504921  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1169) took 1884 ms. Trace:
I20260709 12:28:12.507419  6456 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
W20260709 12:28:12.458418 14249 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.549s	user 0.003s	sys 0.006s
I20260709 12:28:12.505404  6378 rpcz_store.cc:276] 0709 12:28:10.620449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.621461 (+  1012us) service_pool.cc:224] Handling call
0709 12:28:10.621765 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:10.744033 (+122268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:10.744065 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:10.744112 (+    47us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:10.744149 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.504624 (+1760475us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003e94d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:12.504691 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.504856 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280628992}
I20260709 12:28:12.511648  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.512755  7329 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:12.512866  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.514158  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.514730  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.515317  7314 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:12.515359  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:12.515928  6378 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:12.516422  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:12.517907  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:12.518956  6425 connection.cc:256] RPC call timeout handler was delayed by -1.4365797009995731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 12:28:12.519543  6459 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
W20260709 12:28:12.519636  6425 connection.cc:256] RPC call timeout handler was delayed by -1.0963797009999325s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2356s for a response
I20260709 12:28:12.523049  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:12.531307  7335 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:12.531569  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:12.521363  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:12.536288  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.537326  6451 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:12.536305  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:12.541718  7314 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:28:12.437631 13849 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.548472  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.549408  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.551059 14216 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.530619 14115 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:12.552595 14115 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.555042 14214 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:12.555588 14214 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.555984 14214 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:12.562805 14305 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:12.563499 14316 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:12.563796 14305 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:12.563951 14316 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:12.564227 14305 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:12.564359 14316 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:12.565054 14305 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.534435 14136 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:12.471957 14303 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:12.566829 14303 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:12.568517 14218 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:12.568933 14218 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:12.569290 14218 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:12.511129 14284 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.571604 14142 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:12.573092 14284 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.576476  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3467) took 1131 ms. Trace:
I20260709 12:28:12.577081  6380 rpcz_store.cc:276] 0709 12:28:11.445414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.612452 (+167038us) service_pool.cc:224] Handling call
0709 12:28:12.576156 (+963704us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b540023f5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:12.576393 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":803992320}
I20260709 12:28:12.579615 14254 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:12.546334 14249 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:28:12.535177 14080 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:12.583932 14080 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:12.584371 14114 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:12.584617 14080 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:12.585039 14114 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:12.545313 14304 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:12.586026 14304 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.577518 13849 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.601217  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.566386  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1148) took 2214 ms. Trace:
I20260709 12:28:12.594915  6378 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:12.603508  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.604192  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.532480 14308 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.604305  6451 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:12.566272 14305 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:12.541294 14149 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:12.606279  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.597847  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:12.605739  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.583848 14249 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:12.546977 14295 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.468s	user 0.001s	sys 0.000s
I20260709 12:28:12.611594  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.607482  7317 rpcz_store.cc:276] 0709 12:28:10.351619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.396007 (+ 44388us) service_pool.cc:224] Handling call
0709 12:28:10.396353 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:10.679862 (+283509us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:10.993150 (+313288us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.253862 (+260712us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.253996 (+   134us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:11.282737 (+ 28741us) log.cc:844] Serialized 52 byte log entry
0709 12:28:11.282888 (+   151us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:11.283166 (+   278us) log.cc:844] Serialized 14 byte log entry
0709 12:28:11.283451 (+   285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.283496 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:12.564587 (+1281091us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b24002eee78. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f8fb0 00007f6fa2e6f488 00007f6fa2e6fbe8 00007f6fa2e47398 00007f6fa2e4276a 00007f6fa2e41bb6
0709 12:28:12.566112 (+  1525us) raft_consensus.cc:1685] finished
0709 12:28:12.566135 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.566301 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":207948,"prepare.run_cpu_time_us":187,"prepare.run_wall_time_us":8437,"raft.queue_time_us":276575,"raft.run_cpu_time_us":443,"raft.run_wall_time_us":19650,"spinlock_wait_cycles":26130688,"thread_start_us":744809,"threads_started":3,"wal-append.queue_time_us":323300}
W20260709 12:28:12.603516  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.512423 14212 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.093s
I20260709 12:28:12.642692  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:12.586414 14304 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:28:12.618750  6349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.013s	sys 0.023s
I20260709 12:28:12.634155  7329 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:12.619270  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.639977  7314 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
W20260709 12:28:12.639171  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.617718  6378 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:12.647624  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:12.651332  6349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.017s	sys 0.031s
I20260709 12:28:12.655161  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:12.651131  7335 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:12.618561  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:12.612226 14178 consensus_meta.cc:287] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.941s	user 0.009s	sys 0.004s
I20260709 12:28:12.663542  6451 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:12.566849 14136 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:12.651085  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.648967  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.745791  6378 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:12.732007  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:12.750619  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.586201 14114 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.756822  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3346) took 2785 ms. Trace:
I20260709 12:28:12.769805  6301 rpcz_store.cc:276] 0709 12:28:09.971689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.837926 (+1866237us) service_pool.cc:224] Handling call
0709 12:28:11.839408 (+  1482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:11.850220 (+ 10812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:12.756500 (+906280us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b54002062d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:12.756730 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364672}
I20260709 12:28:12.759207  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.771818  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:12.654842 14249 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:12.779605  6309 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:12.616070 14308 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:12.781252  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 231902us
I20260709 12:28:12.781517  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.786696  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:12.767747  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:12.788173  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.633899 13791 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.133s
I20260709 12:28:12.791889  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3447) took 1044 ms. Trace:
I20260709 12:28:12.618841 14291 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:12.799114  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 262889us
I20260709 12:28:12.804265  6274 leader_election.cc:400] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18.
I20260709 12:28:12.804754  6274 leader_election.cc:403] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:12.809157  6451 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:12.814662  6456 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:12.790966 14284 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:12.798563  6457 rpcz_store.cc:276] 0709 12:28:11.747351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.777268 (+ 29917us) service_pool.cc:224] Handling call
0709 12:28:11.777643 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.782825 (+  5182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.782859 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.783376 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.902112 (+118736us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:11.902159 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.902265 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:12.791621 (+889356us) raft_consensus.cc:1685] finished
0709 12:28:12.791647 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:12.791811 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4775,"raft.queue_time_us":5103,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":118230,"threads_started":1,"wal-append.queue_time_us":186134}
I20260709 12:28:12.782249 14223 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.820858  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 172015us
I20260709 12:28:12.815534  6451 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:12.665020 14312 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.152s
I20260709 12:28:12.665094 14314 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.159s
I20260709 12:28:12.856424  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:12.858223  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:12.857939 14223 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.915146  6304 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:12.791190 14115 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:12.862885  6451 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:12.801249 14114 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:12.871094 14249 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.878796 14136 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.855966 14131 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:12.801453 14186 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
I20260709 12:28:12.899544 13655 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:12.901922 13900 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:12.934712  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 167119us
I20260709 12:28:12.935719  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:12.905297 13772 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:12.906169 14310 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:12.911020  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:12.917276  6380 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:12.802299 14319 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:28:12.927567  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 106163us
I20260709 12:28:12.930176 14131 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.957939  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.959410  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.938772  6301 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:28:12.965070  6383 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:12.978081  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 322416us
W20260709 12:28:12.979568  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:12.931056 14284 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:12.983546  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124550us
I20260709 12:28:12.985101  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:12.964731 14319 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:12.989182  7332 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:12.989686  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:12.968287 14328 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:12.988406 14310 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:12.988399 14186 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:12.985994 14322 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.963148 13900 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:12.989559 14012 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:13.016623  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.020567 14080 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:13.022831 14183 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.346s	user 0.008s	sys 0.009s
I20260709 12:28:13.023788 13655 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:12.997476 14115 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:12.998391 13849 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:12.993189 14319 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:13.030290 13900 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.003183 14284 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.045186  6383 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:13.066576  6310 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:28:13.004015 14174 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:13.083599  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:13.069062  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 133441us
I20260709 12:28:13.044968 14012 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:13.030143 14310 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:13.013871 14137 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.003s
I20260709 12:28:13.007714 14318 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.088379  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.063731 14186 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:13.067659 14334 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:28:13.090132 14214 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.003s
I20260709 12:28:13.097478 14214 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.094161 14309 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.105845 14309 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.110111  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.117473  6383 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:13.062090 14322 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.122550 14321 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:13.122790 14339 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.123756 14312 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.124390 14186 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.127938 14284 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.132146 14308 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:13.116382  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.123497 14339 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.075258 14115 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.079571 13849 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.093902 14334 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:13.140399  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3193) took 1181 ms. Trace:
I20260709 12:28:13.140615  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.142185  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.145367  6383 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:13.144951  6382 rpcz_store.cc:276] 0709 12:28:11.959245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.445692 (+486447us) service_pool.cc:224] Handling call
0709 12:28:12.445982 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.446203 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:12.665005 (+218802us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.016350 (+351345us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.140051 (+123701us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:13.140114 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:13.140157 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.140218 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.140347 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51438,"prepare.queue_time_us":471953,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":71,"raft.queue_time_us":147864,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":6833,"spinlock_wait_cycles":33027968,"thread_start_us":582421,"threads_started":4,"wal-append.queue_time_us":366562}
I20260709 12:28:13.108057 14291 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.146833  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:13.148871  7319 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:13.142246 14316 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.575s	user 0.008s	sys 0.005s
I20260709 12:28:13.094594 14137 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:13.141328 14303 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 12:28:13.134374 14328 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:13.173517 14345 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.173933  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.174156 14345 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.174964  7319 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:13.175431 14328 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.175983 14328 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.096836 14318 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:13.163172 14263 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.000s
W20260709 12:28:13.164252 13828 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.735s	user 0.007s	sys 0.004s
I20260709 12:28:13.183547 14344 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.183823 13828 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.184300 14344 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.186087 13828 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.165156 14137 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:13.186461  6378 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:13.165798 14303 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.187664 14263 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.188458 14303 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.188741  6458 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:13.189720 14263 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:13.166381  6383 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:28:13.191452  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:13.191619 14131 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.191480 14115 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.147423 14334 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.193820  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.176841 14316 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.196229 14280 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.197005 14316 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.197136  6376 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:13.166905  6455 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:13.176506 14345 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:13.198611 14304 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.544s	user 0.006s	sys 0.005s
I20260709 12:28:13.149519 14335 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.168695 13849 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.169812  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.199762 14335 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.200173 14335 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.170451 14339 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:13.142128 14214 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.141443 14012 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.203104  6384 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:13.203730  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.172384  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3362) took 1721 ms. Trace:
W20260709 12:28:13.150658 14218 consensus_meta.cc:287] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.002s
W20260709 12:28:13.155697  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:13.142107 14133 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.760s	user 0.002s	sys 0.010s
W20260709 12:28:13.150100 14305 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.534s	user 0.009s	sys 0.003s
I20260709 12:28:13.179965  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:13.195079 14296 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.830s	user 0.002s	sys 0.000s
I20260709 12:28:13.124353 14312 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:13.170730 14288 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.482s	user 0.005s	sys 0.000s
I20260709 12:28:13.199044  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.201604 14318 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:13.203014 14336 consensus_queue.cc:1175] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 21 status { last_received { term: 21 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:28:13.207687  7316 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1054) took 3220 ms (client timeout 1596 ms). Trace:
W20260709 12:28:13.217783  7316 rpcz_store.cc:269] 0709 12:28:09.987250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.987396 (+   146us) service_pool.cc:224] Handling call
0709 12:28:13.207487 (+3220091us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.207620 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":396242176}
I20260709 12:28:13.218271  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1354) took 1880 ms. Trace:
I20260709 12:28:13.208202  7331 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:28:13.218703  6454 rpcz_store.cc:276] 0709 12:28:11.338109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.340711 (+  2602us) service_pool.cc:224] Handling call
0709 12:28:11.347413 (+  6702us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.347819 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:11.347856 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.348275 (+   419us) log.cc:844] Serialized 26 byte log entry
0709 12:28:11.494747 (+146472us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:11.494796 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:11.494871 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.170026 (+1675155us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:13.217952 (+ 47926us) raft_consensus.cc:1685] finished
0709 12:28:13.217980 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.218209 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":341,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":15119,"spinlock_wait_cycles":36138880,"thread_start_us":107374,"threads_started":1,"wal-append.queue_time_us":248496}
I20260709 12:28:13.221304 14334 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.221835  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3176) took 1483 ms. Trace:
I20260709 12:28:13.208634  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:13.208953  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.214823  7314 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:13.204495  6309 rpcz_store.cc:276] 0709 12:28:11.451119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.674416 (+223297us) service_pool.cc:224] Handling call
0709 12:28:13.172197 (+1497781us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400140d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.172314 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2519843072}
I20260709 12:28:13.203078 14080 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.199536 14304 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.222222  6385 rpcz_store.cc:276] 0709 12:28:11.738698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.738906 (+   208us) service_pool.cc:224] Handling call
0709 12:28:11.739548 (+   642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:11.820215 (+ 80667us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:11.821399 (+  1184us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:11.850032 (+ 28633us) log.cc:844] Serialized 26 byte log entry
0709 12:28:12.061430 (+211398us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:12.061479 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.176729 (+1115250us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b54003d1154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:13.176779 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.194381 (+ 17602us) spinlock_profiling.cc:243] Waited 4.47 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:13.221617 (+ 27236us) raft_consensus.cc:1685] finished
0709 12:28:13.221642 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.221774 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1322,"raft.queue_time_us":490,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":6092,"spinlock_wait_cycles":119271040,"thread_start_us":180672,"threads_started":1,"wal-append.queue_time_us":344532}
I20260709 12:28:13.225750 14304 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.206218 14218 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.229475 14111 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.232586 14345 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:13.234139  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3364) took 1287 ms. Trace:
I20260709 12:28:13.239102  6307 rpcz_store.cc:276] 0709 12:28:11.946595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.012785 (+ 66190us) service_pool.cc:224] Handling call
0709 12:28:13.233946 (+1221161us) spinlock_profiling.cc:243] Waited 98.2 ms on lock 0x7b5400093dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.234072 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":225772928}
I20260709 12:28:13.238433  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:39690 (request call id 6) took 3130 ms. Trace:
I20260709 12:28:13.240234  6450 rpcz_store.cc:276] 0709 12:28:10.108011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.643947 (+535936us) service_pool.cc:224] Handling call
0709 12:28:13.238174 (+2594227us) spinlock_profiling.cc:243] Waited 946 ms on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ad735 00007f6fa826dd29 00007f6fa17ec1d4 00007f6fa17ec162 00007f6fa17ec091 00007f6fa17ec00c 00007f6fa17eb2c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.238340 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2176739712}
W20260709 12:28:13.241979  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet 906fbea0458f4a28bfad3b9dde8c6b35: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:13.242120  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.244043  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:13.245038  7316 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:13.244839 14335 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.246701  6377 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:28:13.247753 14344 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:13.248818  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.248579 13828 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:13.211969 14133 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.201442 14328 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.251657  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.251768 14319 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.252959  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3355) took 2954 ms. Trace:
W20260709 12:28:13.253790  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.254036 14319 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:13.254623  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.255350 14178 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:13.255429  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 451331us
I20260709 12:28:13.255985 14178 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.256306  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:13.255517  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 342446us
I20260709 12:28:13.258147 14178 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.258708 14183 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:28:13.256254 14327 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.259562 14327 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.252948 14328 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:13.261461 14328 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:13.261704 14327 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.262470 14328 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.263525 14327 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:13.265208  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.266762  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.263540 14335 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.266258  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.268117  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.266889 14254 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.278090  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.278223 14254 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.278815  6307 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:13.268802 14133 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.279647  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.280251  6307 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:13.280937 14254 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.275233  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3745) took 1262 ms. Trace:
I20260709 12:28:13.271975  6301 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:13.283536  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.284027  6301 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:13.274297  6310 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3360) took 1823 ms (client timeout 1727 ms). Trace:
I20260709 12:28:13.281574 14254 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:13.285100  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.288556  7332 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:13.289485 13752 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:13.289934 13752 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:13.271245  6304 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 12:28:13.284781  6310 rpcz_store.cc:269] 0709 12:28:11.450552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.618598 (+168046us) service_pool.cc:224] Handling call
0709 12:28:13.274004 (+1655406us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54002ddcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.274210 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3282002304}
I20260709 12:28:13.290815 13752 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.291464  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.291793 13752 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:13.292168  6304 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:13.291939  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.292806  6310 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:13.293085  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.284973  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.295029  7335 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:13.282409  6455 rpcz_store.cc:276] 0709 12:28:12.012597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.385572 (+372975us) service_pool.cc:224] Handling call
0709 12:28:13.274981 (+889409us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b54001e14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.275157 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794265600}
I20260709 12:28:13.285379 14205 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:13.270221  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.253429  6303 rpcz_store.cc:276] 0709 12:28:10.298574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.144956 (+1846382us) service_pool.cc:224] Handling call
0709 12:28:12.194732 (+ 49776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.194941 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:12.274342 (+ 79401us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.590146 (+315804us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.233325 (+643179us) spinlock_profiling.cc:243] Waited 3.79 ms on lock 0x7b18004e7ba0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:13.252440 (+ 19115us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b18004e7ba0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:13.252607 (+   167us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:13.252660 (+    53us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:13.252710 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.252778 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.252897 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":648837,"prepare.queue_time_us":108369,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":89,"raft.queue_time_us":664616,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":4754,"spinlock_wait_cycles":17575168,"thread_start_us":312104,"threads_started":1,"wal-append.queue_time_us":513086}
I20260709 12:28:13.298007  6302 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:13.287225  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:13.299168  7316 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:13.298833  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.299964  7314 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:13.300882  6275 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.302232  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3455) took 1090 ms. Trace:
I20260709 12:28:13.320605  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1149) took 2924 ms. Trace:
W20260709 12:28:13.324231  6308 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3345) took 3353 ms (client timeout 2225 ms). Trace:
I20260709 12:28:13.313318  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1176) took 1866 ms. Trace:
W20260709 12:28:13.311012  6452 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3732) took 3120 ms (client timeout 2183 ms). Trace:
I20260709 12:28:13.310456  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:13.328550  6452 rpcz_store.cc:269] 0709 12:28:10.191034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.348149 (+157115us) service_pool.cc:224] Handling call
0709 12:28:13.310755 (+2962606us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54001635d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.310906 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965119872}
I20260709 12:28:13.329497 14128 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:13.327879  6308 rpcz_store.cc:269] 0709 12:28:09.971093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.811851 (+840758us) service_pool.cc:224] Handling call
0709 12:28:13.323992 (+2512141us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400205dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.324149 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894580096}
I20260709 12:28:13.318529 14337 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 12:28:13.331641 13983 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:13.332118 14337 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.328121 14147 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.332990 14337 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.304703 14322 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.335623 14322 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:13.335988 14322 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:13.337231  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3452) took 1357 ms (client timeout 1771 ms). Trace:
I20260709 12:28:13.337107 14322 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:13.337790  6458 rpcz_store.cc:269] 0709 12:28:11.979550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.010948 (+ 31398us) service_pool.cc:224] Handling call
0709 12:28:13.337002 (+1326054us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000c12d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.337155 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2706350976}
I20260709 12:28:13.338713 14322 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.340289 14312 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:13.340708 14290 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.412s	user 0.003s	sys 0.003s
I20260709 12:28:13.342816 14248 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:13.328383  6384 rpcz_store.cc:276] 0709 12:28:11.447238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.639338 (+192100us) service_pool.cc:224] Handling call
0709 12:28:11.639768 (+   430us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.203703 (+1563935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.313057 (+109354us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540023f0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:13.313248 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3594843008}
I20260709 12:28:13.354979 14218 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.323823  7331 rpcz_store.cc:276] 0709 12:28:10.396784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:10.399492 (+  2708us) service_pool.cc:224] Handling call
0709 12:28:13.207957 (+2808465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.219036 (+ 11079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.320362 (+101326us) spinlock_profiling.cc:243] Waited 2.73 s on lock 0x7b540048a150. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.320536 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6288124288}
W20260709 12:28:13.368707 14307 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.938s	user 0.000s	sys 0.022s
I20260709 12:28:13.323627  6456 rpcz_store.cc:276] 0709 12:28:12.211742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.509776 (+298034us) service_pool.cc:224] Handling call
0709 12:28:12.510117 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.815206 (+305089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.301956 (+486750us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54000c0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:13.302159 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":699208448}
W20260709 12:28:13.377522  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.268180  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.382289  6309 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:13.398252  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3194) took 1422 ms. Trace:
I20260709 12:28:13.302327  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:13.400673  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 98419.8us
I20260709 12:28:13.271402 14223 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.302878 14178 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.403223 14178 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.403710 14178 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.407675  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:13.408555  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:13.409143  7331 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.409392  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.409960  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:13.410677 13655 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:13.410984  7319 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:13.411310 13655 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.411940 13655 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:13.289567 14254 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.415515 14254 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:13.417665 14327 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:13.431947  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.432972  6301 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
W20260709 12:28:13.434042  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 123702us
W20260709 12:28:13.434966  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.440653  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.440984  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:13.434597  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 168395us
I20260709 12:28:13.442699  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.443342  6451 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:13.436547  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.438028  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.444815  6455 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:13.445257  6451 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:13.446933  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.437889  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.436043  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:13.453495  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:13.454280  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71424.3us
I20260709 12:28:13.398947  6379 rpcz_store.cc:276] 0709 12:28:11.976264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.444791 (+468527us) service_pool.cc:224] Handling call
0709 12:28:12.445102 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:12.445288 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:12.776811 (+331523us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.985970 (+209159us) log.cc:844] Serialized 14 byte log entry
0709 12:28:12.991373 (+  5403us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.297665 (+306292us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b48001f0158. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:13.366249 (+ 68584us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b180042a360. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:13.397516 (+ 31267us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b180042a360. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287
0709 12:28:13.397689 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:13.397779 (+    90us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:13.397861 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.397963 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.398157 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34907,"prepare.queue_time_us":466939,"prepare.run_cpu_time_us":204,"prepare.run_wall_time_us":5461,"raft.queue_time_us":263312,"raft.run_cpu_time_us":2869,"raft.run_wall_time_us":248580,"spinlock_wait_cycles":29598592,"thread_start_us":601031,"threads_started":3,"wal-append.queue_time_us":239882}
I20260709 12:28:13.458858  6456 raft_consensus.cc:2410] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:13.460572  7316 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:13.460852  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.461115  6310 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:13.456743  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.450302  6451 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 18 index: 8. (term mismatch)
I20260709 12:28:13.465121  6451 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 7
I20260709 12:28:13.465466  6451 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 17.8
I20260709 12:28:13.465746  6451 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:13.468174  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.469120  6455 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 17.
I20260709 12:28:13.469368  6307 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:13.473265  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.473685  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:13.476013  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.478391  6450 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:13.478672  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.478641  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.476920  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.482069  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.485016  6457 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:13.472012  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.486182  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.457041 14136 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:13.464968 14133 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.464751 14349 raft_consensus.cc:2764] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:28:13.463969 14047 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.466203 14314 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:13.465839 14218 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:13.507103  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.470314 14012 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:13.496598  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.476289 13828 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.496737  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.488696  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.499559  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.490923 13766 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.011s
I20260709 12:28:13.500566  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.499327 14047 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:13.502540 14314 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:13.508694  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.510732  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.512081  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.488519 14318 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:13.485416 14336 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:13.488045 14333 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:13.490899 14346 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:13.495775 14136 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.530925  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.525303  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.480201 14313 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:28:13.473954 14304 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:28:13.517741  6457 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:13.532176  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.513118  6379 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:28:13.518311  6455 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:13.519569  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.520635  6385 raft_consensus.cc:2410] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:13.495610 14133 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.522228  6380 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:13.523355  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.523548  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.523978  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.536264  6381 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:13.512766 14218 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.537216  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:13.538019  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49435.1us
I20260709 12:28:13.483566 14192 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:28:13.419083 14327 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.526986  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.518023 13828 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.545615 13828 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.550423  7331 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:13.554422  7318 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:13.556624 14254 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.558717 14131 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:13.559156 14131 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.559537 14131 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:13.538160  6456 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
W20260709 12:28:13.525132  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 266249us
I20260709 12:28:13.533026  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.560849 14279 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:13.561511 14279 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:13.562762 14279 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.563705  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.534235  6378 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:13.542083  6377 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:13.538930 14133 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.543848  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.544382 14335 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:13.524312 14047 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.572309 14212 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.572889 14212 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.522230 14318 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.567492  6310 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 12:28:13.523782 13983 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:13.576527  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.586470  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.526965 14128 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:13.542620  6451 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:13.523522 13752 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:13.578925  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.607347  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.608318  6380 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:13.608882  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.577214  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.587126  6379 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.587790  6307 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:28:13.589123  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 50266.9us
I20260709 12:28:13.613353  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:13.615110  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.559566 14254 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.536513 14136 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.624367 14345 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:13.625288  6381 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:13.594099 14128 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.576870 14218 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.611351  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.617174  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.539512 14337 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:13.607432 14279 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:28:13.592950 14333 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:28:13.549806 14178 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.620287  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.581758 14147 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:13.630267  6377 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:13.630553 14123 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.590349 13983 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:13.620707 14310 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.627193 14128 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:13.631693 14310 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.632678 14216 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:13.531152 14346 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:13.633621 14346 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.586150 14318 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:13.631211 13983 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.564836 14334 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.632768 14303 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:13.636199 14314 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.639370  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3366) took 1692 ms. Trace:
I20260709 12:28:13.639581 14254 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.640197 14254 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.593341  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.639947  6305 rpcz_store.cc:276] 0709 12:28:11.947094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:12.209969 (+262875us) service_pool.cc:224] Handling call
0709 12:28:12.309942 (+ 99973us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:12.347259 (+ 37317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:12.347305 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:12.359117 (+ 11812us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.323177 (+964060us) spinlock_profiling.cc:243] Waited 5.51 ms on lock 0x7b18004e7360. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:13.510086 (+186909us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:13.510142 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.510214 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:13.638993 (+128779us) raft_consensus.cc:1685] finished
0709 12:28:13.639022 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:13.639286 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44268,"raft.queue_time_us":52800,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":19636,"spinlock_wait_cycles":12670208,"thread_start_us":186621,"threads_started":1,"wal-append.queue_time_us":1151708}
I20260709 12:28:13.641959 14304 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:28:13.541361 14309 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.643919  6385 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:13.645993  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.672825  6385 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:13.646833 14147 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:13.651831 14333 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:28:13.654719  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61503us
W20260709 12:28:13.655398  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44154.6us
I20260709 12:28:13.661963  6378 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:13.674314  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.674352  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.681299  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.697153  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.531421 14336 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.683426  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:13.689978  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.701546  6385 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.692018 14333 consensus_queue.cc:1243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:13.688831  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:13.714129  6310 pending_rounds.cc:85] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 11
I20260709 12:28:13.714568  6310 pending_rounds.cc:107] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:28:13.714881  6310 raft_consensus.cc:2889] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:13.718094 14338 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:28:13.711163 14350 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:13.720090  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.720935 13655 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:13.724570  6206 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 144) took 72545 ms (client timeout 15000 ms). Trace:
I20260709 12:28:13.725069 14047 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.717852 14147 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.726784  6376 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:13.727558 14183 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18.
W20260709 12:28:13.728385  7318 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1391) took 2297 ms (client timeout 1662 ms). Trace:
W20260709 12:28:13.728745  7318 rpcz_store.cc:269] 0709 12:28:11.430528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.430701 (+   173us) service_pool.cc:224] Handling call
0709 12:28:13.728231 (+2297530us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b54003bcc54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:13.728318 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4745051904}
I20260709 12:28:13.720959  6305 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:13.732249 13655 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.734254 14133 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.735536  6300 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:13.738962 14212 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.691310 14136 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.711436 14254 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.742183 14136 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.742587 14218 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.744831 14133 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.745796  6459 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:28:13.747051 13828 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:13.749534  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.749773 13828 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:13.752749  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55697us
I20260709 12:28:13.754027  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.658561 14137 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:13.754958 14349 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:13.755968 14349 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:13.754915 14137 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:13.757383 14336 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:13.617975 14263 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:13.758728  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:13.759253 14047 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.747480 14178 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:13.763058  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.764793  6377 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:13.757436 14137 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:13.766554  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52660.9us
I20260709 12:28:13.769330  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.767652  6380 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
W20260709 12:28:13.766716  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 310041us
I20260709 12:28:13.772886  6385 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:13.815336  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52364.2us
I20260709 12:28:13.687381 14304 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:13.816476  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.681149 14342 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:13.814364  6379 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:13.826718  6383 raft_consensus.cc:606] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:28:13.846896  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.769893 14350 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:13.834528  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.805356 14333 raft_consensus.cc:2764] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:28:13.725021  6206 rpcz_store.cc:269] 0709 12:27:01.179483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.179672 (+   189us) service_pool.cc:224] Handling call
0709 12:27:01.200711 (+ 21039us) spinlock_profiling.cc:243] Waited 8.07 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa01830b2 00007f6fa90fc759 00007f6fa9094a76 00007f6fa9065d58 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc
0709 12:28:13.724374 (+72523663us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:07.309524 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:07.446903 (+137379us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:07.446985 (+    82us) write_op.cc:435] Acquired schema lock
0709 12:28:07.447031 (+    46us) tablet.cc:662] Decoding operations
0709 12:28:07.447661 (+   630us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:07.447771 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:07.447820 (+    49us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:28:07.448520 (+   700us) tablet.cc:701] Row locks acquired
0709 12:28:07.448576 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:28:07.448805 (+   229us) write_op.cc:270] Start()
0709 12:28:07.448965 (+   160us) write_op.cc:276] Timestamp: P: 1783600087448729 usec, L: 0
0709 12:28:07.449013 (+    48us) op_driver.cc:348] REPLICATION: starting
0709 12:28:07.450115 (+  1102us) log.cc:844] Serialized 901 byte log entry
0709 12:28:07.647942 (+197827us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b
0709 12:28:08.769545 (+1121603us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:09.053574 (+284029us) op_driver.cc:464] REPLICATION: finished
0709 12:28:09.274901 (+221327us) write_op.cc:301] APPLY: starting
0709 12:28:09.275005 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:09.277256 (+  2251us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:09.277295 (+    39us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:09.281601 (+  4306us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:09.281932 (+   331us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:28:09.282006 (+    74us) write_op.cc:312] APPLY: finished
0709 12:28:09.282316 (+   310us) log.cc:844] Serialized 33 byte log entry
0709 12:28:09.283329 (+  1013us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:09.283416 (+    87us) write_op.cc:454] Released schema lock
0709 12:28:09.283474 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:09.283929 (+   455us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:09.283964 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7131779,"reported_tablets":29,"spinlock_wait_cycles":18607616,"child_traces":[["op",{"apply.queue_time_us":220918,"apply.run_cpu_time_us":5327,"apply.run_wall_time_us":9078,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":64901,"prepare.run_cpu_time_us":19010,"prepare.run_wall_time_us":425413,"raft.queue_time_us":186467,"raft.run_cpu_time_us":29550,"raft.run_wall_time_us":179611,"replication_time_us":1604300,"spinlock_wait_cycles":78813696,"thread_start_us":596593,"threads_started":4,"wal-append.queue_time_us":330105,"wal-append.run_cpu_time_us":50470,"wal-append.run_wall_time_us":2498432}]]}
I20260709 12:28:13.863978  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.877277  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.877767 14212 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:13.823948 14304 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:13.867077  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97875.6us
I20260709 12:28:13.871251  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.879998 14212 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.872810  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.904125  6300 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:13.758359 14263 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.903275  6274 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.907838  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.908723  6305 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.908982  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.828089 14342 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:13.820799 14178 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:13.887410  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:13.873824  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.862619 14350 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:13.913447  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.865774 14136 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:13.898365  6302 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:28:13.929293  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.902875 14212 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:13.902201  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:13.851181 14309 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:28:13.877907  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 405936us
I20260709 12:28:13.920615  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.859808 14080 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.924168  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.939082  6379 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:13.904145 14186 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.926803  6300 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
W20260709 12:28:13.946692  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 494396us
I20260709 12:28:13.935945  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:13.904995 14254 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:13.937211  6348 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:13.897533  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.948031  6348 leader_election.cc:403] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:28:13.948349  6301 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:13.949056  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.900764 13655 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:13.905612 14304 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:13.953874 14254 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:13.949563  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.950681  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.955726  6301 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:13.956063  6300 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:13.956670  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.957158  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.957554  6301 raft_consensus.cc:2410] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:28:13.959074  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.959939  6301 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:28:13.961030  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.923480 14174 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:13.924744 14183 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:13.923643 14349 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:13.934330 14223 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:13.972764  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:13.946219  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.994800  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.995877  6451 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:13.996635  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:13.997362  6455 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:13.998404  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:13.973884 14090 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:13.964569  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:14.001014 14205 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.705s	user 0.015s	sys 0.013s
I20260709 12:28:14.001669 14205 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:14.002065 14205 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:13.940418 14303 raft_consensus.cc:2764] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:28:13.978602 14328 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:14.002951 14059 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.003690 14059 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.004410  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:14.007952 14131 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.010703 14303 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:14.011013 14284 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:14.011633 14303 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:13.907210  6198 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 14 to 18, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 18 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:14.014935 14318 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.016312  6454 raft_consensus.cc:2368] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:14.016768  6198 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 9 to 13, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 13 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.018278 14318 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:14.019006 14284 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:14.019922 14212 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:14.020175 14284 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.023947  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.024672  6454 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:14.026939 14349 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:14.027738  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:14.028033 14349 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.029330  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.030591  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.030665 14223 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:14.031436  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:14.032017 14223 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.036108 14128 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.038542  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:14.038707 14313 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:14.039177  6456 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:28:14.039249 14313 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.039626 14131 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:14.039963  6383 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:14.040629  6383 raft_consensus.cc:606] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:14.016163 13766 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:13.925037 14342 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:13.980192  6300 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.969941  6452 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:13.982350  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46548.9us
I20260709 12:28:14.057621  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:13.962478  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.986080  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:13.931881 14350 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:13.988987  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:13.975162  6379 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:13.972951 13916 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:14.063258  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74351.1us
I20260709 12:28:14.045523  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:13.943928 14322 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.065654  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:14.047326  6457 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:14.060953  6301 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26.
I20260709 12:28:13.979668 14183 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.056418  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.075095  6452 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:28:14.075608  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:14.075958  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:14.076421  6379 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:14.076531  6452 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:28:14.002431 14205 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:14.040391 14131 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:28:14.087980  6300 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
W20260709 12:28:14.088668  6309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.009s	sys 0.021s
I20260709 12:28:14.072270  6456 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:13.984802 14310 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:14.091475  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.099372  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:13.987040 14174 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:14.173761  6377 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:14.086200  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:14.169977  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.101785  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:14.172410  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:14.173861  6380 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:14.174822  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:13.971290 14080 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:13.977196 14186 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.058549 14133 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.144s
I20260709 12:28:14.102346 14312 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.289s
I20260709 12:28:14.390430 14337 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:14.391341 14337 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:14.083385 13916 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:14.391942  6309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.012s	sys 0.039s
I20260709 12:28:14.175642 14047 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:14.392302 14337 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.393522 13916 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:14.375305 14361 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 12:28:14.393586 14337 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:14.047359 14279 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:14.384616  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.395668 14279 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:14.395921  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 294259us
I20260709 12:28:14.395780 14362 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:14.286551  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 481919us
I20260709 12:28:14.396744 14279 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.084846 14309 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:14.066553 14254 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.088519 14214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.019s	sys 0.011s
I20260709 12:28:14.389844  6455 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:14.083516 14328 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:14.239106  6272 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:14.296005  6382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.008s	sys 0.032s
I20260709 12:28:14.400925  6377 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:14.402195  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:14.402138  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.175040  6305 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:14.114212  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.391433  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.262986  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:14.430385  6456 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
W20260709 12:28:14.429095  6382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.008s	sys 0.038s
I20260709 12:28:14.175935  6348 leader_election.cc:400] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:14.288152 14313 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:14.414546  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.175336  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:14.169584 13983 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.006s
I20260709 12:28:14.421072  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:14.442705  6455 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:14.413952 14355 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 12:28:14.269335 14128 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:14.419809 14214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.027s	sys 0.016s
I20260709 12:28:14.457464  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:14.458133  6305 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:14.257354  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 171228us
I20260709 12:28:14.432585  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.436240  6381 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:14.460501  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.460880  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.461530  6303 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:14.461921  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:14.462857  6456 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:14.433293  6348 leader_election.cc:403] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:28:14.440928  6301 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:14.444000  6380 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:14.465579  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:14.465991  7331 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:14.466656  7318 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:14.445305  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:14.467494  7314 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:14.460255  6376 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:14.088871 14183 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:14.336169 14111 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.452344  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:14.452888  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 479178us
W20260709 12:28:14.489364  7319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.012s	sys 0.021s
I20260709 12:28:14.083187 14360 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:14.404064 14303 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:14.455590  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:14.428308 14309 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:14.466580  7331 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:28:14.447067  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.169819 14318 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.284s
I20260709 12:28:14.473991  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.419450 13772 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:28:14.083624 14322 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.515790  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.516017  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:14.516335  6310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.015s	sys 0.014s
I20260709 12:28:14.516506  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
W20260709 12:28:14.517475  6310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.016s	sys 0.015s
I20260709 12:28:14.440034  6452 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:14.481803  7316 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:14.485617  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:14.487082  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 311295us
I20260709 12:28:14.516562  6300 raft_consensus.cc:2376] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22. Re-sending same reply.
I20260709 12:28:14.522502  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.487705  6308 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
W20260709 12:28:14.491014  7319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.012s	sys 0.021s
I20260709 12:28:14.493611  6377 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:14.401562 14047 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.201782 14327 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:14.502019 14305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.021s	sys 0.003s
I20260709 12:28:14.501369  6455 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:28:14.578831  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.450260 14263 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:14.504935  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:14.585543  6456 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:28:14.497884  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.110409 13766 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.363s
I20260709 12:28:14.456884 13983 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.427634 14254 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.536629  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1398) took 1090 ms. Trace:
I20260709 12:28:14.613413  7319 rpcz_store.cc:276] 0709 12:28:13.446121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.457805 (+ 11684us) service_pool.cc:224] Handling call
0709 12:28:13.475889 (+ 18084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.476104 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:13.930812 (+454708us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.534158 (+603346us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.534348 (+   190us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:14.534543 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.534949 (+   406us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.536550 (+  1601us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4696,"prepare.queue_time_us":103907,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":11398,"raft.queue_time_us":40495,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":23548,"thread_start_us":1036336,"threads_started":2}
W20260709 12:28:14.499594  6378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.012s	sys 0.009s
W20260709 12:28:14.505676 14012 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.988s	user 0.012s	sys 0.000s
I20260709 12:28:14.489599 14280 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:14.405563 14174 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.448117 14328 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.579792  6301 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
W20260709 12:28:14.589102  6458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.022s	sys 0.051s
W20260709 12:28:14.629426  6458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.022s	sys 0.051s
I20260709 12:28:14.630218  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1361) took 1196 ms. Trace:
I20260709 12:28:14.502131 14360 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:14.529443  6385 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
W20260709 12:28:14.606005  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:14.610386  6305 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
W20260709 12:28:14.617674  6378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.012s	sys 0.009s
W20260709 12:28:14.577865 14305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.022s	sys 0.003s
I20260709 12:28:14.499495 14111 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.643023  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:14.648625  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:14.613456  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78843.9us
I20260709 12:28:14.532855 13655 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:14.533743 14212 raft_consensus.cc:2764] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 25: won
I20260709 12:28:14.626027  6384 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:14.582976 14137 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:28:14.601344 14344 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:28:14.635735  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:14.680657  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:14.637173  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:14.346973  6198 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 15 to 26. New cstate: current_term: 26 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:14.639292 14360 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:14.633229 14059 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:14.639946 14339 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:28:14.685070  6385 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:14.698722  6456 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:14.584769 14327 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:14.709584  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 66683.8us
I20260709 12:28:14.628129  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:14.723016  6456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:14.723448  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3208) took 1211 ms. Trace:
I20260709 12:28:14.750532  6382 rpcz_store.cc:276] 0709 12:28:13.512258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.539582 (+ 27324us) service_pool.cc:224] Handling call
0709 12:28:13.591290 (+ 51708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.756122 (+164832us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:13.756511 (+   389us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.449423 (+692912us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.454741 (+  5318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.454805 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:14.473285 (+ 18480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.723090 (+249805us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54001950d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:14.723163 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.723359 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8807,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":474,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":15868,"spinlock_wait_cycles":407360256,"thread_start_us":675946,"threads_started":1,"wal-append.queue_time_us":876446}
W20260709 12:28:14.757197  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 129173us
W20260709 12:28:14.724859  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 485838us
I20260709 12:28:14.750425  6456 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:28:14.759765  6451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.019s	sys 0.014s
I20260709 12:28:14.650694 14309 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.759833  6456 raft_consensus.cc:614] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:14.737707 14357 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.359s	user 0.000s	sys 0.001s
I20260709 12:28:14.799485  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:14.750334 14012 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:14.762986  6300 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:14.734637 14318 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:28:14.738667  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:14.812403  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74304.9us
I20260709 12:28:14.813171  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:14.766635  6451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.020s	sys 0.014s
W20260709 12:28:14.744340 14373 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.339s	user 0.000s	sys 0.002s
I20260709 12:28:14.775236 14254 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:14.779501 14304 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.829s	user 0.015s	sys 0.001s
W20260709 12:28:14.780449 14136 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.844s	user 0.009s	sys 0.002s
I20260709 12:28:14.781322 14047 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:14.783656 14328 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:14.787489 14149 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:14.788709 14280 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:14.791081 14369 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:28:14.737058 14133 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:14.791926 14218 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.826432 14314 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:14.806234  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:14.825182 14313 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:14.905753  6381 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:14.771445 14349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.008s	sys 0.025s
W20260709 12:28:14.910463 14349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.008s	sys 0.025s
I20260709 12:28:14.809612  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.847290 14218 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.912081  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.912945  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:14.913348  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:14.913589 14218 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:14.913722  6450 raft_consensus.cc:606] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:14.913865  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3386) took 1626 ms. Trace:
I20260709 12:28:14.914155  6301 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:14.914369  6304 rpcz_store.cc:276] 0709 12:28:13.287457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.293974 (+  6517us) service_pool.cc:224] Handling call
0709 12:28:13.294258 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.294455 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.294480 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.430462 (+135982us) log.cc:844] Serialized 26 byte log entry
0709 12:28:13.825918 (+395456us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:13.826520 (+   602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:13.826622 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.913672 (+1087050us) raft_consensus.cc:1685] finished
0709 12:28:14.913693 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.913813 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":336079,"threads_started":1,"wal-append.queue_time_us":1375667}
I20260709 12:28:14.915988  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.917342  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:14.918386  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.905344 14314 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:14.919140 14314 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:14.919466  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.920287  6304 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:14.910558 14376 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.255s	user 0.003s	sys 0.000s
W20260709 12:28:14.921278 14381 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
I20260709 12:28:14.922032  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.922781  6304 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 18: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:14.923133  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3043) took 1029 ms. Trace:
I20260709 12:28:14.923437  6304 rpcz_store.cc:276] 0709 12:28:13.893925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.920808 (+1026883us) service_pool.cc:224] Handling call
0709 12:28:14.923086 (+  2278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:14.924423  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.925405  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:14.926164  6304 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:14.804041  7331 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:14.811051  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3393) took 1097 ms. Trace:
I20260709 12:28:14.927562  7331 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:14.927517  6310 rpcz_store.cc:276] 0709 12:28:13.713429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.713587 (+   158us) service_pool.cc:224] Handling call
0709 12:28:13.713918 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:13.814809 (+100891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.814982 (+   173us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:13.883322 (+ 68340us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:13.884184 (+   862us) log.cc:844] Serialized 52 byte log entry
0709 12:28:14.518227 (+634043us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:14.518295 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.518526 (+   231us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.793054 (+274528us) raft_consensus.cc:1685] finished
0709 12:28:14.793088 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.810970 (+ 17882us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7541,"raft.queue_time_us":48386,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7039,"spinlock_wait_cycles":12151040,"thread_start_us":579986,"threads_started":1,"wal-append.queue_time_us":822989}
I20260709 12:28:14.767349 14186 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:14.835129 14136 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:28:14.935974 14346 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.302s	user 0.010s	sys 0.005s
I20260709 12:28:14.936571 14346 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:14.941009 14123 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.310s	user 0.012s	sys 0.000s
I20260709 12:28:14.941550 14123 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:14.850870 14280 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:14.941671  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3397) took 1069 ms. Trace:
W20260709 12:28:14.761775  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 326774us
I20260709 12:28:14.957846  6308 rpcz_store.cc:276] 0709 12:28:13.872232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.873516 (+  1284us) service_pool.cc:224] Handling call
0709 12:28:14.941396 (+1067880us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b5400093dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:14.941585 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1206316416}
I20260709 12:28:14.817359  6425 leader_election.cc:400] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26.
I20260709 12:28:14.961941  6425 leader_election.cc:403] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:28:14.820573  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 335097us
I20260709 12:28:14.962025  6308 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:28:14.963704  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3053) took 1011 ms. Trace:
I20260709 12:28:14.965507  6308 rpcz_store.cc:276] 0709 12:28:13.952636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.960126 (+1007490us) service_pool.cc:224] Handling call
0709 12:28:14.960700 (+   574us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.963328 (+  2628us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.963631 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:14.752996 14131 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:14.829643 14310 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:14.999809  6379 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:14.926033 13655 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.831509 14304 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:14.834244 14149 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:14.850571 14328 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:28:14.840018 14047 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:14.943742  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.023448  6305 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:14.975975 14339 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26.
I20260709 12:28:14.768808 14137 consensus_queue.cc:1243] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:14.923148  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3201) took 1414 ms. Trace:
I20260709 12:28:14.765839  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:14.949965  6452 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:14.815626  6382 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:15.056473  6384 rpcz_store.cc:276] 0709 12:28:13.509105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.517554 (+  8449us) service_pool.cc:224] Handling call
0709 12:28:14.922976 (+1405422us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54003e4ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:14.923085 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2516263680}
I20260709 12:28:14.914177  6450 raft_consensus.cc:614] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:14.950282  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3491) took 1372 ms. Trace:
W20260709 12:28:15.003026 14371 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.348s	user 0.001s	sys 0.001s
I20260709 12:28:15.058411  6378 rpcz_store.cc:276] 0709 12:28:13.577380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.709652 (+132272us) service_pool.cc:224] Handling call
0709 12:28:13.710282 (+   630us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.816433 (+106151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:13.816468 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:13.828239 (+ 11771us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.635710 (+807471us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:14.635777 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.635872 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:14.950045 (+314173us) raft_consensus.cc:1685] finished
0709 12:28:14.950066 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.950218 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96455,"raft.queue_time_us":109612,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":25861,"thread_start_us":725685,"threads_started":1,"wal-append.queue_time_us":1080334}
I20260709 12:28:15.060266  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.061015  6378 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:14.814632 14012 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:15.063889  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 299428us
I20260709 12:28:14.972226  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:14.973232  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.064801  7329 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:14.974339  7314 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
W20260709 12:28:14.981377  6453 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.007s	sys 0.031s
W20260709 12:28:15.066788  6453 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.007s	sys 0.031s
I20260709 12:28:15.067829  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3482) took 1029 ms. Trace:
W20260709 12:28:14.960727 14387 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 12:28:15.082213 14388 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.282s	user 0.000s	sys 0.000s
I20260709 12:28:14.967094 13983 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:14.959234 14133 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:14.945170 14322 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:15.043643  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.052947  7331 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:15.043931  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:15.030361 14362 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:28:14.975406 14111 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:14.998739 14310 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:14.775163  6198 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 16 to 21, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 21 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:14.970841  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 153642us
I20260709 12:28:15.064733  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:15.000347 14318 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:15.005482  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:15.057745  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.113061  6382 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:15.112948  7331 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:15.056000 14047 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.072618  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:15.073787 14350 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.902s	user 0.013s	sys 0.000s
I20260709 12:28:15.111660 14401 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:15.074107  6301 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:15.076357  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3483) took 1033 ms. Trace:
W20260709 12:28:15.078579 14379 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.354s	user 0.001s	sys 0.001s
I20260709 12:28:15.078824 14305 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.080142  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.080842  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3486) took 1034 ms. Trace:
I20260709 12:28:15.081425 14304 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.087523 13772 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:15.023051 14385 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:15.091490 14342 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 1.005s	user 0.014s	sys 0.000s
I20260709 12:28:15.043577 14367 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:28:15.092195  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3059) took 1137 ms. Trace:
I20260709 12:28:15.196307  6300 rpcz_store.cc:276] 0709 12:28:13.954583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.522225 (+567642us) service_pool.cc:224] Handling call
0709 12:28:15.091991 (+569766us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54002de6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:15.092123 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551439104}
I20260709 12:28:15.068216  6453 rpcz_store.cc:276] 0709 12:28:14.038172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.038330 (+   158us) service_pool.cc:224] Handling call
0709 12:28:14.038695 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.038914 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:14.062165 (+ 23251us) log.cc:844] Serialized 14 byte log entry
0709 12:28:15.067361 (+1005196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.067439 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:15.067508 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.067603 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.067753 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24040,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"thread_start_us":1004029,"threads_started":1}
I20260709 12:28:15.031778  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:15.198351  6308 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:15.197451  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:15.199617  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:15.199724  6300 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:15.200626  6308 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:14.969504 14358 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.095620  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:15.096318  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:15.006785  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:15.101022  7332 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:15.206876  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 200170us
I20260709 12:28:14.972827 14131 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:15.102355 14337 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.002s
W20260709 12:28:15.111106  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46465.2us
I20260709 12:28:15.215540  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:15.111302  6455 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:15.072074 14365 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 12:28:15.116307  6382 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:15.115406  6451 rpcz_store.cc:276] 0709 12:28:14.042632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.044738 (+  2106us) service_pool.cc:224] Handling call
0709 12:28:14.045046 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.059173 (+ 14127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.059203 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.067434 (+  8231us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.815278 (+747844us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:14.815316 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.815379 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.072537 (+257158us) spinlock_profiling.cc:243] Waited 9.86 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:15.076029 (+  3492us) raft_consensus.cc:1685] finished
0709 12:28:15.076052 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.076297 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82485,"raft.queue_time_us":17529,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":17554,"spinlock_wait_cycles":22677888,"thread_start_us":585115,"threads_started":1,"wal-append.queue_time_us":853114}
W20260709 12:28:15.175647  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61931.3us
I20260709 12:28:15.220737  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:15.117633  6454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.077s	user 0.006s	sys 0.035s
I20260709 12:28:15.221393  6308 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
W20260709 12:28:15.118599  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.187553  7331 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:28:15.188170  6377 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:28:15.227746  6377 pending_rounds.cc:85] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 13
I20260709 12:28:15.228103  6377 pending_rounds.cc:107] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:28:15.228390  6377 raft_consensus.cc:2889] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:15.118275 14284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.008s	sys 0.011s
I20260709 12:28:15.209530  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:15.232925  6300 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:15.234694  6300 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:15.235477  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3055) took 1282 ms. Trace:
I20260709 12:28:15.235872  6300 rpcz_store.cc:276] 0709 12:28:13.953303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.233890 (+1280587us) service_pool.cc:224] Handling call
0709 12:28:15.234250 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.235172 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.235412 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.238510  6300 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:28:15.239331  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3058) took 1285 ms. Trace:
I20260709 12:28:15.239805  6300 rpcz_store.cc:276] 0709 12:28:13.954236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.237624 (+1283388us) service_pool.cc:224] Handling call
0709 12:28:15.238195 (+   571us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.239026 (+   831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.239268 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.212977  7329 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:15.071022 13655 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.221313  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:15.174122  6457 rpcz_store.cc:276] 0709 12:28:14.046383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.500015 (+453632us) service_pool.cc:224] Handling call
0709 12:28:14.500472 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.500934 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.500974 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.501570 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.853902 (+352332us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:28:14.853980 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.854088 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.080613 (+226525us) raft_consensus.cc:1685] finished
0709 12:28:15.080637 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.080773 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33421,"raft.queue_time_us":14465,"raft.run_cpu_time_us":7033,"raft.run_wall_time_us":54067,"thread_start_us":351902,"threads_started":1,"wal-append.queue_time_us":574995}
W20260709 12:28:15.221752  6454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.181s	user 0.006s	sys 0.038s
I20260709 12:28:15.108973 14133 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.114773 14350 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.250617  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:15.248538  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3761) took 1256 ms. Trace:
W20260709 12:28:15.210837  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 451870us
I20260709 12:28:15.251765  6454 rpcz_store.cc:276] 0709 12:28:13.992241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.025530 (+ 33289us) service_pool.cc:224] Handling call
0709 12:28:14.025850 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.026066 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:14.026382 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:28:15.248103 (+1221721us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.248174 (+    71us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:15.248234 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.248326 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.248473 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":371,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"thread_start_us":1206964,"threads_started":1}
I20260709 12:28:15.253448  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:15.118027 14314 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:15.217163 14205 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.041s	user 0.014s	sys 0.000s
I20260709 12:28:15.219296 14111 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:15.261183  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:15.244372  6300 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:28:15.241838 14284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.011s	sys 0.011s
I20260709 12:28:15.117861 14318 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.255069  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:15.197425 14385 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.276276  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3400) took 1286 ms. Trace:
I20260709 12:28:15.259178 14349 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:15.111718  6198 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 15 to 16, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 16 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:15.280793 14047 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.270051 14401 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:28:15.216917 14358 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.186753 13772 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.233343 14304 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:15.283583  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.337913  6382 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
W20260709 12:28:15.251538 14183 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.747s	user 0.010s	sys 0.003s
I20260709 12:28:15.233961 14337 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:15.118046 14149 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.289803  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3387) took 1852 ms. Trace:
I20260709 12:28:15.285974  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:15.205328 14342 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.303542  6300 rpcz_store.cc:276] 0709 12:28:13.989650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.243603 (+1253953us) service_pool.cc:224] Handling call
0709 12:28:15.243977 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.275919 (+ 31942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.276212 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.304630 14322 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:15.286561 14309 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.344951  6309 rpcz_store.cc:276] 0709 12:28:13.437412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.437568 (+   156us) service_pool.cc:224] Handling call
0709 12:28:13.438945 (+  1377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:13.439335 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:13.472822 (+ 33487us) log.cc:844] Serialized 14 byte log entry
0709 12:28:13.568858 (+ 96036us) spinlock_profiling.cc:243] Waited 3.97 ms on lock 0x7b18004e70c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:14.392534 (+823676us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.393046 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.393169 (+   123us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:14.393200 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.395226 (+  2026us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.284146 (+888920us) raft_consensus.cc:1685] finished
0709 12:28:15.289515 (+  5369us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.289722 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4624,"prepare.queue_time_us":36282,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":103624,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":19495,"spinlock_wait_cycles":9121664,"thread_start_us":891701,"threads_started":1,"wal-append.queue_time_us":1275263}
I20260709 12:28:15.350934  6309 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:15.351488  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3402) took 1286 ms. Trace:
I20260709 12:28:15.351790  6309 rpcz_store.cc:276] 0709 12:28:14.065207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.350307 (+1285100us) service_pool.cc:224] Handling call
0709 12:28:15.350623 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.351249 (+   626us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.351435 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.353180  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3403) took 1269 ms. Trace:
I20260709 12:28:15.353495  6309 rpcz_store.cc:276] 0709 12:28:14.084261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.352469 (+1268208us) service_pool.cc:224] Handling call
0709 12:28:15.352720 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.352883 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.352909 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.352949 (+    40us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:15.352981 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.353044 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.353132 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:15.355119 14389 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:28:15.306068 14111 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:15.356137 14111 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.358283 14133 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.358976  6309 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 20. (term mismatch)
I20260709 12:28:15.359373  6309 pending_rounds.cc:85] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 19
I20260709 12:28:15.359583  6309 pending_rounds.cc:107] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 28.20
I20260709 12:28:15.359799  6309 raft_consensus.cc:2889] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:15.360416  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1218) took 1268 ms. Trace:
I20260709 12:28:14.630596  6458 rpcz_store.cc:276] 0709 12:28:13.433665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.434594 (+   929us) service_pool.cc:224] Handling call
0709 12:28:13.477532 (+ 42938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:13.477725 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:13.820334 (+342609us) log.cc:844] Serialized 14 byte log entry
0709 12:28:14.629842 (+809508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:14.629897 (+    55us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:14.629944 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.630027 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:14.630154 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16902,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":133,"thread_start_us":1134650,"threads_started":2}
I20260709 12:28:15.360684  6309 rpcz_store.cc:276] 0709 12:28:14.092208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.354327 (+1262119us) service_pool.cc:224] Handling call
0709 12:28:15.358662 (+  4335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.360195 (+  1533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.360367 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:15.308868  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.269515 14305 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:28:15.287505 14392 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.289s	user 0.000s	sys 0.001s
I20260709 12:28:15.327406  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:15.363214 14314 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:15.363950  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3480) took 1328 ms. Trace:
I20260709 12:28:15.364771 14318 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.367450  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3499) took 1290 ms. Trace:
I20260709 12:28:15.367925 13655 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.369349 14350 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.371955 14328 raft_consensus.cc:993] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: : Instructing follower 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:28:15.372601 14385 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:15.372745 14328 raft_consensus.cc:1081] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Signalling peer 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:28:15.374109  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3766) took 1288 ms. Trace:
I20260709 12:28:15.374626  6452 rpcz_store.cc:276] 0709 12:28:14.085730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.520239 (+434509us) service_pool.cc:224] Handling call
0709 12:28:14.520548 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.056841 (+536293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.373818 (+316977us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:15.374037 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":987069312}
I20260709 12:28:15.375833 14337 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.380034 14080 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:15.383229 14377 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.688s	user 0.001s	sys 0.001s
I20260709 12:28:15.384039  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3051) took 1432 ms. Trace:
I20260709 12:28:15.384618  6301 rpcz_store.cc:276] 0709 12:28:13.951971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.954667 (+1002696us) service_pool.cc:224] Handling call
0709 12:28:14.955084 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.115146 (+160062us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.383705 (+268559us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002ddcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:15.383946 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":250895744}
I20260709 12:28:15.386299 13772 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.389588 14346 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.390954 14327 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.394874  7335 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:15.396358 14358 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.399200 14342 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.399971  6385 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:15.404563 14400 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:15.367921  6379 rpcz_store.cc:276] 0709 12:28:14.077469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.077580 (+   111us) service_pool.cc:224] Handling call
0709 12:28:14.077869 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.000432 (+922563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.367163 (+366731us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54003d0c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:15.367381 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119689472}
I20260709 12:28:15.411173  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.413587  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.414580  6457 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:15.419328  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.283659 14178 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:15.325922  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:15.262022 14362 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:28:15.262125 14399 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:28:15.346441  6300 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:15.277529  7332 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:15.436888  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:15.348575  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62718.9us
I20260709 12:28:15.437827  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3401) took 1373 ms. Trace:
I20260709 12:28:15.438263  6300 rpcz_store.cc:276] 0709 12:28:14.064700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.345840 (+1281140us) service_pool.cc:224] Handling call
0709 12:28:15.346129 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.437523 (+ 91394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.437758 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.348730 14047 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:15.349080 14339 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:15.340879 14304 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:15.423439  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:15.364300  6459 rpcz_store.cc:276] 0709 12:28:14.035117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.035288 (+   171us) service_pool.cc:224] Handling call
0709 12:28:14.035686 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.058650 (+ 22964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.058684 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.067096 (+  8412us) log.cc:844] Serialized 26 byte log entry
0709 12:28:14.397590 (+330494us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b48000cccd8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:14.628351 (+230761us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:14.628402 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:14.631152 (+  2750us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:15.363691 (+732539us) raft_consensus.cc:1685] finished
0709 12:28:15.363711 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.363896 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26952,"raft.queue_time_us":26648,"raft.run_cpu_time_us":12513,"raft.run_wall_time_us":331349,"spinlock_wait_cycles":16413952,"thread_start_us":398656,"threads_started":1,"wal-append.queue_time_us":665077}
I20260709 12:28:15.455672  6450 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:15.424510 14346 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.284934 14205 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.428663  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.471189  6379 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:15.427066  7335 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:15.431883  6453 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:15.474046  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.427105  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1224) took 1333 ms. Trace:
W20260709 12:28:15.330168  7288 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.665s (SENT)
I20260709 12:28:15.260478 14012 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:15.436703  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 110921us
W20260709 12:28:15.412245 14303 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.013s	sys 0.017s
W20260709 12:28:15.382369 14360 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.000s
I20260709 12:28:15.445588  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.506012  6198 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 22 to 23, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 23 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:15.506932 14360 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.446620  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.455031  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.509402  6456 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:15.458315  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:15.464521  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 137229us
I20260709 12:28:15.475095  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:15.474927  6301 rpcz_store.cc:276] 0709 12:28:14.093918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.386108 (+1292190us) service_pool.cc:224] Handling call
0709 12:28:15.421024 (+ 34916us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.426503 (+  5479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.426669 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.426744 (+    75us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:15.426801 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.426888 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.427036 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.485409  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:15.488162  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.290424 14131 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:15.519567  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 189433us
I20260709 12:28:15.436908 14322 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:15.444382 14314 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:15.530105  6303 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:15.455204 14339 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.514569  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:15.519088  7319 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:15.508791  6457 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:15.443118 14399 consensus_queue.cc:1243] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:28:15.520531  7288 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.734s (SENT)
I20260709 12:28:15.541882  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:15.433545 14337 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.436025 14178 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:15.552739 14178 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:15.553929 14322 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.503446  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:15.559186  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55852.6us
W20260709 12:28:15.559969  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.560390  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.561340 14318 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:15.561818 14318 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.345294 14183 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.464557 14304 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:15.422158 14280 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.527637 14133 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:28:15.510700 14303 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.014s	sys 0.018s
I20260709 12:28:15.477618 14149 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:15.462396 14047 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.546525 14131 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:15.538766 14333 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.534344 14362 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:15.550755  6275 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:15.602341  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51666.7us
I20260709 12:28:15.551193 14254 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:15.550618  6198 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 19 to 20, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 20 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:15.499655 14313 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:15.567232  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46756.4us
W20260709 12:28:15.570379  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3215) took 1398 ms (client timeout 1695 ms). Trace:
W20260709 12:28:15.612481  6376 rpcz_store.cc:269] 0709 12:28:14.171799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.175474 (+  3675us) service_pool.cc:224] Handling call
0709 12:28:15.570158 (+1394684us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54000af5d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:15.570307 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39837568}
W20260709 12:28:15.533740 14407 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:28:15.566066  6459 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:15.480127 14350 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:15.577198  6450 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:15.572228 13772 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:15.572165 14400 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.532449 13916 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.620621 14400 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.532752 14137 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:15.621989 14137 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.623001 14178 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.624219 14137 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.481232 14346 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.627807 14346 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.562259 14318 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:15.592527  6307 raft_consensus.cc:2368] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 29. Current term is 30.
I20260709 12:28:15.581642 14133 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:15.619973  6300 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:15.620687 13916 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.537110 14339 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:15.581171 14303 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:15.611726 14305 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:28:15.652613  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 138174us
I20260709 12:28:15.662263  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:15.639631 14133 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:15.689013 14366 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.102s	user 0.013s	sys 0.004s
I20260709 12:28:15.692341  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1225) took 1598 ms. Trace:
I20260709 12:28:15.692910  6300 rpcz_store.cc:276] 0709 12:28:14.094187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.477940 (+1383753us) service_pool.cc:224] Handling call
0709 12:28:15.478319 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.639971 (+161652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.691987 (+ 52016us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54001b3fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:15.692238 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325358464}
I20260709 12:28:15.702425 14400 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.707226  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1185) took 1176 ms. Trace:
I20260709 12:28:15.706401 14214 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:15.708168 14214 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:15.707808  6385 rpcz_store.cc:276] 0709 12:28:14.530939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.727876 (+196937us) service_pool.cc:224] Handling call
0709 12:28:14.737553 (+  9677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.400368 (+662815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.706893 (+306525us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b54001f4fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:15.707150 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522973952}
I20260709 12:28:15.709616 14214 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.710836 14305 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.714395 14305 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.715939 14047 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.711452  6300 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:15.617642 14327 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 12:28:15.712327 14214 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:15.673362 14344 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:15.730731 14344 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:15.732025 14344 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.732749 13772 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.733363 14385 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.733884 14327 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:15.733348 13772 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.734373 14327 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:15.734295 14385 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.735517 14327 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.736568 13772 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:15.671713 14319 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.631318 14313 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.736701 14385 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.736944 14327 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:15.739712  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.741364  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:15.755587  7335 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:15.674355 14346 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:15.727023  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3404) took 1123 ms. Trace:
I20260709 12:28:15.796083  6300 rpcz_store.cc:276] 0709 12:28:14.603626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.694025 (+1090399us) service_pool.cc:224] Handling call
0709 12:28:15.694378 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.719261 (+ 24883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.726948 (+  7687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:15.609216 14333 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.787330  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:15.757767  6307 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1215) took 1666 ms (client timeout 1665 ms). Trace:
W20260709 12:28:15.827000  6307 rpcz_store.cc:269] 0709 12:28:14.091277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.401608 (+310331us) service_pool.cc:224] Handling call
0709 12:28:15.757565 (+1355957us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002d19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:15.757691 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2673854336}
I20260709 12:28:15.828359  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:15.758383  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:15.759531  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3399) took 1833 ms. Trace:
I20260709 12:28:15.830431  6310 rpcz_store.cc:276] 0709 12:28:13.925792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.929871 (+1004079us) service_pool.cc:224] Handling call
0709 12:28:14.930246 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:14.953589 (+ 23343us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:15.304515 (+350926us) log.cc:844] Serialized 14 byte log entry
0709 12:28:15.759240 (+454725us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.759284 (+    44us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:15.759330 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.759390 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.759482 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62639,"prepare.queue_time_us":509607,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":79468,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"spinlock_wait_cycles":57870208,"thread_start_us":703549,"threads_started":3,"wal-append.queue_time_us":376908}
I20260709 12:28:15.740139  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.833787  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:15.763619  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:15.770706  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:15.771464  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.774722  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.779974  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:15.781944  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.784283  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.733283 14344 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:15.742343 14137 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:15.789013  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.760726  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.788527  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.857421  6454 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:28:15.790503  6457 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
W20260709 12:28:15.790717  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 178104us
W20260709 12:28:15.858831  7288 leader_election.cc:336] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.875s (SENT)
W20260709 12:28:15.795328  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.839723  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76202.8us
I20260709 12:28:15.824182  6453 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:15.788520  7320 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:15.841941  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:15.866568  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:15.842860  6379 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:15.866359  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:15.845408  7329 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:15.851961  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:15.864585  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:15.877758  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 90557.5us
I20260709 12:28:15.879875  6385 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:15.889652  6198 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 7 to 13, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 13 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:15.893041  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.878480  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:15.896996  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:15.900470  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:15.900682  7331 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:15.901273  6300 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:15.835398 14400 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.909789  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:15.837337 14205 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:15.861454 14403 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:28:15.908412  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:15.737811 14413 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.142s
I20260709 12:28:15.884754 14395 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:15.905706  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1222) took 1812 ms. Trace:
I20260709 12:28:15.855495 14178 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.859210 14080 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:15.917140  6198 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 10 to 17, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 17 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:15.923507 14415 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
W20260709 12:28:15.924177 14422 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:28:15.924304  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:15.924937 14430 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 12:28:15.825917 14303 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.926702 14280 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:15.927132 14419 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
W20260709 12:28:15.927714 14409 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.287s	user 0.000s	sys 0.000s
I20260709 12:28:15.761224 14319 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:15.929150  6300 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
W20260709 12:28:15.902455 14370 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.157s	user 0.003s	sys 0.003s
W20260709 12:28:15.926270 14391 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:28:15.961629 14080 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:15.995209  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:15.930608  7335 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:15.861656 14305 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:15.885031 14111 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:15.931257  7331 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:15.911904  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:15.946856  6459 raft_consensus.cc:2410] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:28:15.899834 14123 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:15.949651 14279 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:28:16.024180  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:15.983611 14136 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:15.906617 14399 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:16.032991 14399 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.033455 14399 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:15.899636 14362 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:15.950320  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:15.913079 14397 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:28:15.913475 14393 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:28:15.953266  6309 rpcz_store.cc:276] 0709 12:28:14.093336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.361385 (+1268049us) service_pool.cc:224] Handling call
0709 12:28:15.361633 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.361805 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:15.637441 (+275636us) log.cc:844] Serialized 14 byte log entry
0709 12:28:15.905329 (+267888us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:15.905391 (+    62us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:15.905442 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.905510 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:15.905644 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142942,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":26565,"thread_start_us":540802,"threads_started":2}
W20260709 12:28:16.036293  6305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3047) took 2102 ms (client timeout 2137 ms). Trace:
I20260709 12:28:15.904881 14349 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.036677  6305 rpcz_store.cc:269] 0709 12:28:13.933642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.636891 (+703249us) service_pool.cc:224] Handling call
0709 12:28:16.036129 (+1399238us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b54001b35d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.036235 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615478016}
I20260709 12:28:16.037300  6382 raft_consensus.cc:1217] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Deduplicated request from leader. Original: 19.12->[21.13-21.13]   Dedup: 21.13->[]
I20260709 12:28:16.037362 14349 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.038096  6382 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:28:15.877534 14342 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.038842 14342 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.039887 14349 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.040918 14342 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.041188 14174 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:16.041636 14174 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:15.923558 14360 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.902700 14254 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:28:16.043077 14254 consensus_queue.cc:1243] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:15.949369 14414 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:15.949560 14192 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:15.962069 13655 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:28:16.005342 14080 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.050696 13916 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.046710 14136 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.056283 14313 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:16.057582 14136 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.058828  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3398) took 2133 ms. Trace:
I20260709 12:28:16.058992 14333 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.059206  6304 rpcz_store.cc:276] 0709 12:28:13.925381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.926765 (+1001384us) service_pool.cc:224] Handling call
0709 12:28:14.927034 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:14.927379 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.927404 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:14.927815 (+   411us) log.cc:844] Serialized 26 byte log entry
0709 12:28:15.318733 (+390918us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:15.318800 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.318946 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.058623 (+739677us) raft_consensus.cc:1685] finished
0709 12:28:16.058645 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.058770 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8391,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":121,"spinlock_wait_cycles":71552,"thread_start_us":390489,"threads_started":1,"wal-append.queue_time_us":659713}
I20260709 12:28:16.061391  6304 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:16.062158  6304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:57670
I20260709 12:28:15.953194 14395 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.062727 14395 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:16.024493 14111 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:15.929749 14400 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.017561 14123 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.047523 14414 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:16.066812 14123 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:16.067028 14414 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:16.068143 14400 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:15.975668 14183 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.014027  7314 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:16.066211 14111 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.018805  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.088003  6382 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:28:16.018669  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.026695  6198 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 13 to 18, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 18 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:16.028137  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.029744  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3221) took 1416 ms. Trace:
W20260709 12:28:15.934007  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 271844us
I20260709 12:28:16.094226  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:16.095189  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.032042  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.008100 14435 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.049984 14192 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:16.069149  6304 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:16.069196  6300 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:16.030663  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3054) took 2077 ms. Trace:
I20260709 12:28:16.104979  6304 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:16.105443  6308 rpcz_store.cc:276] 0709 12:28:13.952979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.225280 (+1272301us) service_pool.cc:224] Handling call
0709 12:28:15.225736 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.226208 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.226237 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:15.242432 (+ 16195us) log.cc:844] Serialized 26 byte log entry
0709 12:28:15.444326 (+201894us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:15.444382 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.444459 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.030366 (+585907us) raft_consensus.cc:1685] finished
0709 12:28:16.030399 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.030591 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6913,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":12239,"thread_start_us":201458,"threads_started":1,"wal-append.queue_time_us":408022}
I20260709 12:28:16.094772  6378 rpcz_store.cc:276] 0709 12:28:14.613875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.061997 (+448122us) service_pool.cc:224] Handling call
0709 12:28:15.062408 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.109605 (+ 47197us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:15.109638 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:15.174662 (+ 65024us) log.cc:844] Serialized 26 byte log entry
0709 12:28:15.455653 (+280991us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:15.455698 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:15.455759 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.029545 (+573786us) raft_consensus.cc:1685] finished
0709 12:28:16.029567 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.029685 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15255,"raft.queue_time_us":19759,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":58593,"thread_start_us":246483,"threads_started":1,"wal-append.queue_time_us":501625}
I20260709 12:28:16.034258 14304 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.098390  7331 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:16.109020  6378 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:16.113188  6382 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:16.115481  6376 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 16: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:16.116458  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.109413  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.119302  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.126060  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.107317 14192 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:16.134773  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.135138 14337 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:16.135457 14080 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.138309  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.045473 14174 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.138815  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.139328 14174 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:16.139516  7331 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:16.139568  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.014565 14424 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.132s
I20260709 12:28:16.140692  7316 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:16.141680  7316 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:16.135425  7335 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:16.141723  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.144102  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.144022  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.118983 14149 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:16.086086 13772 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.150895  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.152132  6301 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:16.153549  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.176649  7335 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:28:16.122447 14313 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:16.176208  6351 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21.
I20260709 12:28:16.115080 14183 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:16.154170  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.190551 14183 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:16.154229 14080 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.188850  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.194685  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:16.194315  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.816s (SENT) [suppressed 24 similar messages]
I20260709 12:28:16.164500 14350 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.215646  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:16.142704 14322 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:16.218109  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.242630  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.243011  6382 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.120826 14435 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.226984  7331 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:16.194239 14344 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.163420 14337 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.053s
W20260709 12:28:16.241766  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47609.6us
W20260709 12:28:16.199807 14131 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.004s
I20260709 12:28:16.252594  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:16.245334  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.246927  6380 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:28:16.250620  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.254906  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:16.255029  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:16.245389  6376 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:16.191993 13772 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.221717 13655 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:28:16.260030  6378 raft_consensus.cc:1217] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Deduplicated request from leader. Original: 20.11->[22.12-26.14]   Dedup: 22.12->[24.13-26.14]
I20260709 12:28:16.262112  6378 pending_rounds.cc:85] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 12
I20260709 12:28:16.262355  6378 pending_rounds.cc:107] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 23.13
W20260709 12:28:16.250691 14318 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.616s	user 0.011s	sys 0.001s
I20260709 12:28:16.209314 14398 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:28:16.228830 14183 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:16.225710 14147 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.270020 14453 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:16.272682 14452 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:28:16.273347  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.266024 14313 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:16.249377 14350 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.228274 14136 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:28:16.302539 14304 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.303375 14310 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.304738 14310 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:16.304948 14350 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:16.275036 14450 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:28:16.227936 14254 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:28:16.307235  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3220) took 1700 ms. Trace:
I20260709 12:28:16.307402 14254 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:16.308022 14254 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:16.300261 14443 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:28:16.309309 14254 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.258330 14444 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:28:16.258512 14344 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.251674 14268 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.040s
I20260709 12:28:16.266083 14319 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.268154 14318 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.291767 14395 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.310425  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.341289  6310 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:16.342245  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.342955 14385 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:16.343403 14385 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:16.343716  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.344138 14254 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:16.345717  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:16.346953 14400 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:16.347827 14424 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:16.348205 14424 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.348023 14400 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:16.348562 14424 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:16.349336 14111 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:16.349752 14111 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:16.349859 14400 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:16.350323 14400 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:16.350700 14111 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.351461 14400 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.351984 14111 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:16.352617 14400 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:16.348507  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.354928  6452 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:16.355444  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.356058  6459 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:16.356317  6456 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:16.356528  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.251554 14328 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:16.357331 14328 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.358142  6451 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:28:16.278725  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:16.359751  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81105.9us
I20260709 12:28:16.360364  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.361162  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:16.361485 14178 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:16.361914 14178 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:16.362282 14178 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:16.363382  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:16.363929 14444 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:16.284621  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.312928 14435 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.313289  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.365203  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.313900 14358 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:16.365806 14358 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:16.320461 14344 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.366279  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:16.323602 14442 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:28:16.367216  6308 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:16.323727 14447 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:28:16.368220  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.324154  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:16.368887  6308 raft_consensus.cc:2376] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20. Re-sending same reply.
W20260709 12:28:16.286913 14339 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 12:28:16.354727 14186 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:16.363646 14123 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:16.276923 14418 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.600s	user 0.002s	sys 0.000s
I20260709 12:28:16.284600  6382 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:28:16.570825  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.586115  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.262928  6378 raft_consensus.cc:2889] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.364682 14444 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.151s
W20260709 12:28:16.593408  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.331364 14327 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
I20260709 12:28:16.338929 14313 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:16.334122 14414 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.297075 14147 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:16.369673  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45611us
I20260709 12:28:16.616717  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:16.334379  7319 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1167) took 1375 ms (client timeout 1816 ms). Trace:
W20260709 12:28:16.618294  7319 rpcz_store.cc:269] 0709 12:28:14.958757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.962342 (+  3585us) service_pool.cc:224] Handling call
0709 12:28:16.334164 (+1371822us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b5400a41fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.334298 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":910717184}
W20260709 12:28:16.333253  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3042) took 2439 ms (client timeout 1514 ms). Trace:
I20260709 12:28:16.340159  7329 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:28:16.619552  6302 rpcz_store.cc:269] 0709 12:28:13.893624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.928961 (+ 35337us) service_pool.cc:224] Handling call
0709 12:28:16.333022 (+2404061us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54002e7a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.333176 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3535991168}
I20260709 12:28:16.620671  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.326428  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1226) took 2232 ms. Trace:
W20260709 12:28:16.332139 14428 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.429s	user 0.000s	sys 0.000s
I20260709 12:28:16.307659  6381 rpcz_store.cc:276] 0709 12:28:14.607362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.907124 (+299762us) service_pool.cc:224] Handling call
0709 12:28:14.952227 (+ 45103us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:14.952799 (+   572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:14.952846 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:15.788592 (+835746us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:15.835487 (+ 46895us) log.cc:844] Serialized 52 byte log entry
0709 12:28:16.010949 (+175462us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:16.010999 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.011092 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:16.306861 (+295769us) raft_consensus.cc:1685] finished
0709 12:28:16.306884 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:16.307163 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6317,"raft.queue_time_us":423,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":9288,"spinlock_wait_cycles":82278784,"thread_start_us":82081,"threads_started":1,"wal-append.queue_time_us":242589}
I20260709 12:28:16.327770  6379 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:28:16.512619  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:16.337452  6303 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3066) took 2370 ms (client timeout 1914 ms). Trace:
I20260709 12:28:16.614900 14214 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
W20260709 12:28:16.630221  6303 rpcz_store.cc:269] 0709 12:28:13.966639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.472738 (+506099us) service_pool.cc:224] Handling call
0709 12:28:16.337246 (+1864508us) spinlock_profiling.cc:243] Waited 943 ms on lock 0x7b5400140fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.337372 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2169375616}
I20260709 12:28:16.631494  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.632270  6303 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:16.568132 14435 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.592701 14136 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:16.633080  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.633592 14136 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.374460  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.634974 14313 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.366722 14358 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.636368 14358 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:16.540551  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:16.638888  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 98396.1us
I20260709 12:28:16.565928 14339 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:16.335253 14425 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.642s	user 0.002s	sys 0.000s
I20260709 12:28:16.640039 14414 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.544495  6456 pending_rounds.cc:85] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 8
I20260709 12:28:16.640687  6456 pending_rounds.cc:107] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:28:16.640812 14339 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:16.641029  6456 raft_consensus.cc:2889] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:16.611049 14452 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:28:16.641667 14123 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.568384 14328 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.545109  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:16.534113  6459 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:16.336335 14318 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:16.644681  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:16.531934  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:16.336643  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.645869  6458 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:16.539000  6424 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:16.554088  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.340533  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:16.647524  7317 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:16.551918 14344 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:28:16.488915 14395 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.567591  6451 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:28:16.656641  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 117733us
I20260709 12:28:16.656951  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.541177 14350 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:16.558374 14133 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.825s	user 0.013s	sys 0.000s
I20260709 12:28:16.344369 14385 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.661409  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:16.661782  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.541347 14268 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:16.567976  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:16.565161 14399 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.531s	user 0.013s	sys 0.000s
I20260709 12:28:16.598807  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:16.675249  6300 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:16.570382 14424 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.611799  6307 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:16.313014  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1184) took 1782 ms. Trace:
I20260709 12:28:16.621488  6301 rpcz_store.cc:276] 0709 12:28:14.094517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.511444 (+1416927us) service_pool.cc:224] Handling call
0709 12:28:15.511731 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:16.175673 (+663942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.326189 (+150516us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b54000a9e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:16.326359 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1472209792}
I20260709 12:28:16.623569  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.632759 14435 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.371474  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.580406 14192 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.645292  6459 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:16.634258  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:16.685477  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86754.3us
I20260709 12:28:16.546015  6304 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.547058  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.706179  6305 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:16.566663  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.706956  6308 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20. Re-sending same reply.
W20260709 12:28:16.648636  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 263638us
W20260709 12:28:16.657189  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.657467  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:16.560871 14303 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.551362 14349 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.721213 14349 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.721937  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.722508  6379 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:28:16.723253 14349 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.723781 14303 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.724970 14435 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.645385 13916 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:16.725755 13916 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:16.615702 14183 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.726193 13916 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:16.704807  6309 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:16.727106  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.728104  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.728857 14136 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.729983  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.635478  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.730829  6457 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:16.731528  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.725749 14303 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.734721  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.735419 14183 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:16.736294  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.737557  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:16.737800  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.738914  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:16.715391  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58018.3us
I20260709 12:28:16.710131  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:16.690572  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:16.708504  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:16.742458 14452 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:16.743870 14452 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:16.744220 14452 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:16.744534 14452 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:16.720214  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.752183  7316 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:16.733223  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.720703 14447 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:16.700487 14218 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:16.739637  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:16.745040 14349 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:28:16.746057 14459 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:28:16.651595 14214 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:16.808696 14218 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.814333 14214 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:16.700758 14444 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:16.686329 13766 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:28:16.747841 14192 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.748072  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3495) took 1369 ms. Trace:
I20260709 12:28:16.750751 13772 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:16.731630  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 118375us
I20260709 12:28:16.843184  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.844482  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.753247  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.688714  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.846684  6303 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 15.
I20260709 12:28:16.846968  6301 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:16.716020  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:16.756850  6381 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:16.768392 14337 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.717360  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 170391us
I20260709 12:28:16.842046  6458 rpcz_store.cc:276] 0709 12:28:15.409687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.409835 (+   148us) service_pool.cc:224] Handling call
0709 12:28:16.747884 (+1338049us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002c9554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.748012 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2610965248}
I20260709 12:28:16.850356  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.851094  6301 raft_consensus.cc:2376] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19. Re-sending same reply.
I20260709 12:28:16.709463  6377 rpcz_store.cc:276] 0709 12:28:14.530131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:14.530431 (+   300us) service_pool.cc:224] Handling call
0709 12:28:14.924154 (+393723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:15.228962 (+304808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.311714 (+1082752us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54003e4ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:16.312779 (+  1065us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54003e4ed0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.312948 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510764032}
I20260709 12:28:16.758072  6309 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:16.852929  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:16.769156  6300 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:16.771296  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:16.771343  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 80921.1us
I20260709 12:28:16.773569  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:16.670516 14318 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:16.858973  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 87783.3us
W20260709 12:28:16.859732  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47181us
I20260709 12:28:16.859942  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:16.860893  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.809751 14435 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:16.863834 14350 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:16.702690 14399 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.817464 14131 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.768872 14268 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:16.819525 14436 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.821455 14310 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.003s
I20260709 12:28:16.816077 14328 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:16.823540  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3418) took 1040 ms. Trace:
I20260709 12:28:16.906764  6307 rpcz_store.cc:276] 0709 12:28:15.783101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.828034 (+ 44933us) service_pool.cc:224] Handling call
0709 12:28:16.823372 (+995338us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:16.823481 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1799926144}
I20260709 12:28:16.907593  6451 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:28:16.862263 14192 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:16.825673 14136 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:16.826386  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.828670  6379 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:16.815227 14214 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:16.816504 14451 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:28:16.731688  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.853026  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.855273  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:16.945542  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90358.9us
I20260709 12:28:16.946409  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:16.943542  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:16.855508  7319 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:16.852073  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.856352  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.669675 14133 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.770442 14344 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:16.777920  6304 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.842351 14447 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:16.876379  6385 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.013s	sys 0.045s
W20260709 12:28:16.955703  6385 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.013s	sys 0.046s
I20260709 12:28:16.868161 14183 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:28:16.877967  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.809958  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:16.960394  6309 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:28:16.961685  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.962527  6459 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
I20260709 12:28:16.880079  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.964186  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.880731  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:16.846222 14424 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.967425  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:57670 (request call id 3417) took 1176 ms. Trace:
I20260709 12:28:16.967869  6304 rpcz_store.cc:276] 0709 12:28:15.778754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.062051 (+283297us) service_pool.cc:224] Handling call
0709 12:28:16.955317 (+893266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:16.893615  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.908267  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:16.970472  6307 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:16.961650  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.971766  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.973692  6307 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:16.868188 14358 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:16.923342  6305 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:16.924214  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.868860 14458 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:28:16.949072 14174 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.784s	user 0.012s	sys 0.000s
I20260709 12:28:16.985543  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:16.867919 14337 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:16.890816 14131 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.965353  6459 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 27 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:17.000531  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:16.965721  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:16.968969  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.003015  6304 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:16.816151 14218 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:16.972332  6309 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:16.977047  6458 raft_consensus.cc:2410] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 20 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:28:16.976806  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:17.017794  6301 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:16.948673 14111 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 12:28:16.941948  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:16.937924 14333 raft_consensus.cc:2764] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:28:16.950839  6377 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:17.092483  6459 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:28:16.986449  6457 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:16.986802  6303 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:16.992447  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:16.992671  6452 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:16.877177  6376 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:16.941221 14192 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.106972  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:16.889379 14455 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.104s
I20260709 12:28:17.016520  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.110859  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:28:17.111986  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:17.112846  6304 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:17.020865 14399 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.022231  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.113492  6459 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:17.113982  6384 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 16.
W20260709 12:28:17.058156 14254 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.713s	user 0.009s	sys 0.000s
I20260709 12:28:16.944521 14136 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.968293 14447 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.968858 14183 consensus_queue.cc:1243] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:28:16.904906 14436 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:16.976387 13655 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:16.976858 14284 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:16.998775 14147 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.107431  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.143066  6376 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:17.143815  6377 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:17.107986  6381 raft_consensus.cc:2368] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26.
I20260709 12:28:17.145938  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:17.017432 14123 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 12:28:17.109356  6451 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
I20260709 12:28:17.161818  6451 pending_rounds.cc:85] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 13
I20260709 12:28:17.162087  6451 pending_rounds.cc:107] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:28:17.162312  6451 raft_consensus.cc:2889] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:17.049011  6301 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:17.117930  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1372) took 1383 ms. Trace:
I20260709 12:28:17.165411  6450 rpcz_store.cc:276] 0709 12:28:15.734945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:15.789173 (+ 54228us) service_pool.cc:224] Handling call
0709 12:28:15.789503 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:15.789919 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:15.789947 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:15.790434 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.173096 (+382662us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:16.173160 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.173253 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:17.117725 (+944472us) raft_consensus.cc:1685] finished
0709 12:28:17.117750 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:17.117873 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58264,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":24955,"spinlock_wait_cycles":8064,"thread_start_us":264367,"threads_started":1,"wal-append.queue_time_us":688839}
I20260709 12:28:16.957226 13916 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.183275  6455 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.187125  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.141165 13655 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.194115  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:17.014249 14458 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:17.125317  6310 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:17.093708 14339 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:17.128641  6309 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:17.197321  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:17.197676  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.199357  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.129845  6305 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.202131  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.203593  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.108991  7319 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:17.140293  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.058499 14414 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.070s
I20260709 12:28:17.172055 14192 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.173537 14183 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.175148 14436 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:17.175794 14131 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:17.093413 14268 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:28:17.176751 14447 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:17.177152 14284 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:17.177817 14449 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.345s	user 0.004s	sys 0.000s
I20260709 12:28:17.062223 14452 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.163430  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:17.050019 14178 consensus_meta.cc:287] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.001s
I20260709 12:28:17.194867  6301 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:17.195853  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:17.200137  6309 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.070700 14337 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.196878 14339 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:17.208757  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.002s	sys 0.006s
W20260709 12:28:17.209707  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 499615us
I20260709 12:28:17.193256 13655 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:17.230198  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:17.287479  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57404.8us
I20260709 12:28:17.289470  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:17.242456  6379 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:17.290932  6379 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:17.251960  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3075) took 1096 ms. Trace:
I20260709 12:28:17.259672  6308 raft_consensus.cc:2368] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21.
I20260709 12:28:17.294029  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1230) took 1141 ms. Trace:
I20260709 12:28:17.294435  6308 rpcz_store.cc:276] 0709 12:28:16.152166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.190101 (+1037935us) service_pool.cc:224] Handling call
0709 12:28:17.293954 (+103853us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:17.295709  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.296437  6308 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.260571  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:17.298230  6309 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:17.299823  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:17.300642  6308 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.261813  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:17.307925  7316 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:17.309836  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:17.310559  7317 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.312749  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:17.270402  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3519) took 1046 ms. Trace:
I20260709 12:28:17.314711  6385 rpcz_store.cc:276] 0709 12:28:16.223518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.316564 (+ 93046us) service_pool.cc:224] Handling call
0709 12:28:16.316928 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.317187 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.317212 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.317703 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:28:16.956277 (+638574us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:16.956345 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.956455 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:17.270096 (+313641us) raft_consensus.cc:1685] finished
0709 12:28:17.270127 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:17.270322 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":638036,"threads_started":1,"wal-append.queue_time_us":940710}
W20260709 12:28:17.266685  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.003s	sys 0.006s
W20260709 12:28:17.317104  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 691336us
I20260709 12:28:17.221520  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:17.273033  6302 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:17.322304  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:17.334515  6309 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:17.249691 14284 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.291808  6379 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.218475  6304 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
W20260709 12:28:17.249745  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.362246 14318 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:17.293287  6310 rpcz_store.cc:276] 0709 12:28:16.155180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.512179 (+356999us) service_pool.cc:224] Handling call
0709 12:28:17.251708 (+739529us) spinlock_profiling.cc:243] Waited 495 ms on lock 0x7b54002d14d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:17.251877 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1138807552}
I20260709 12:28:17.387249  6310 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:17.255133  6303 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:17.388000 13916 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.388494  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3098) took 1160 ms. Trace:
I20260709 12:28:17.389034  6303 rpcz_store.cc:276] 0709 12:28:16.228141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.163176 (+935035us) service_pool.cc:224] Handling call
0709 12:28:17.388420 (+225244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:17.259153  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1236) took 1105 ms. Trace:
I20260709 12:28:17.390590  6301 rpcz_store.cc:276] 0709 12:28:16.153674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.188360 (+1034686us) service_pool.cc:224] Handling call
0709 12:28:17.259051 (+ 70691us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:17.392161  6301 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:17.392917  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3079) took 1226 ms. Trace:
I20260709 12:28:17.393357  6301 rpcz_store.cc:276] 0709 12:28:16.166323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.391559 (+1225236us) service_pool.cc:224] Handling call
0709 12:28:17.391931 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.392623 (+   692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.392851 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:17.255596  6300 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:17.355399  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:17.388026  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3072) took 1233 ms. Trace:
I20260709 12:28:17.396600  6310 rpcz_store.cc:276] 0709 12:28:16.154434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.386323 (+1231889us) service_pool.cc:224] Handling call
0709 12:28:17.386762 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.387698 (+   936us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.387958 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:17.261544 14339 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.400326 14435 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:17.293293  6274 leader_election.cc:400] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26.
I20260709 12:28:17.402842  6274 leader_election.cc:403] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:28:17.350550 14349 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:28:17.358553  6424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.044s	sys 0.110s
I20260709 12:28:17.348412 14268 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:17.254091 14452 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.240484 14183 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.261134 14178 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.248754 14136 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.321642 14444 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:17.326766 14254 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:17.374286 13655 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.442711  6382 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:28:17.379168  6309 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [23.13-25.14]
I20260709 12:28:17.403919  6301 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:28:17.398319  6310 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:28:17.278412  6307 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:17.409981  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3509) took 1039 ms. Trace:
I20260709 12:28:17.410454 14350 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:17.412034  6424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.045s	sys 0.110s
W20260709 12:28:17.451256  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 626825us
I20260709 12:28:17.446272  6456 rpcz_store.cc:276] 0709 12:28:16.370634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.385503 (+ 14869us) service_pool.cc:224] Handling call
0709 12:28:16.543639 (+158136us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:16.641876 (+ 98237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.641916 (+    40us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:16.642696 (+   780us) log.cc:844] Serialized 78 byte log entry
0709 12:28:16.918369 (+275673us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:16.918435 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:16.918557 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:17.409722 (+491165us) raft_consensus.cc:1685] finished
0709 12:28:17.409749 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:17.409916 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3296,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":3314,"spinlock_wait_cycles":27008,"thread_start_us":275173,"threads_started":1,"wal-append.queue_time_us":764577}
W20260709 12:28:17.455709  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.457624  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:17.459057  6385 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:17.358240 14322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.009s	sys 0.004s
I20260709 12:28:17.348065 14453 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
I20260709 12:28:17.371284 14414 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:17.478253  6303 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:17.421303  6378 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.005s	sys 0.009s
W20260709 12:28:17.420739  6422 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.009s	sys 0.005s
I20260709 12:28:17.384567 14358 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:17.422259  7317 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
W20260709 12:28:17.438036  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.447012  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3071) took 1292 ms. Trace:
I20260709 12:28:17.484284  6309 rpcz_store.cc:276] 0709 12:28:16.154194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.378221 (+1224027us) service_pool.cc:224] Handling call
0709 12:28:17.378729 (+   508us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:17.446638 (+ 67909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.446936 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:17.485128  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:17.414124 14314 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.057s	sys 0.118s
I20260709 12:28:17.461256  6384 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:17.417047 14349 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:17.424163 14452 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:17.480640  6422 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.010s	sys 0.005s
I20260709 12:28:17.489645  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.497536  6385 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:17.498418  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.418982 14268 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:17.491953  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:17.479601  6378 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.005s	sys 0.009s
I20260709 12:28:17.441542 14136 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26.
I20260709 12:28:17.469911 14337 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:17.470847 14322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.009s	sys 0.004s
I20260709 12:28:17.450732 14254 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.471225  7319 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
W20260709 12:28:17.490850 14314 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.058s	sys 0.119s
I20260709 12:28:17.370100 14458 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.492543 13655 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.483208 14350 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:17.486637 14447 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.500921  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:17.577664  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 76810.9us
I20260709 12:28:17.502734  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:17.513773  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:17.580590  6454 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:17.580761  6450 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:17.582336  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:17.583699  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:17.488442 14414 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.531685 14358 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:17.532282 14254 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:17.533735 14178 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.503751  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.503571 14436 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:17.502154  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:17.523298  6457 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:17.527964  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:17.490954 14131 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.524032  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:17.525571  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:17.496006 14452 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:17.585338  6450 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.534142 14443 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:28:17.506443 14444 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.602980 13655 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.495064 14344 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:28:17.628335 13655 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.506132 14313 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.116s
W20260709 12:28:17.647104  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 119224us
W20260709 12:28:17.640425  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3080) took 1473 ms (client timeout 1925 ms). Trace:
W20260709 12:28:17.650192  6302 rpcz_store.cc:269] 0709 12:28:16.166586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.620370 (+453784us) service_pool.cc:224] Handling call
0709 12:28:17.640202 (+1019832us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b54000b4d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:17.640351 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":865027968}
I20260709 12:28:17.594628 14218 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:17.639340  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3074) took 1484 ms. Trace:
I20260709 12:28:17.655846  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:17.656044  6303 rpcz_store.cc:276] 0709 12:28:16.154936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.389885 (+1234949us) service_pool.cc:224] Handling call
0709 12:28:17.390248 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.478769 (+ 88521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.639007 (+160238us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b54002062d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:17.639263 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183229696}
I20260709 12:28:17.657146  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:17.526341 14337 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:17.695519  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:17.564333 14318 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.074s
W20260709 12:28:17.741724  6377 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.003s	sys 0.016s
W20260709 12:28:17.626325 14454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.005s	sys 0.012s
W20260709 12:28:17.743026  6377 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.003s	sys 0.016s
I20260709 12:28:17.604408 14358 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:17.780068 14337 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:17.791680  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:17.596765 14447 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:17.799279 14174 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.006s	sys 0.017s
I20260709 12:28:17.655846 14218 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:17.816177 14218 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:17.604285 14268 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:17.821918 14469 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:28:17.656476 14436 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:17.629829 14131 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.775022 14471 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.107s
I20260709 12:28:17.824389 14131 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:17.742851 14303 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:17.825006 14303 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.825415 14303 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:17.639355 14192 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.150s
I20260709 12:28:17.800231 14452 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.638391 14349 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.815109 14358 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.774760 14047 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:17.743211 14314 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:17.639691 14444 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:17.796872 14454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.005s	sys 0.013s
W20260709 12:28:17.834352  7329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.001s	sys 0.010s
I20260709 12:28:17.794234 14318 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:17.846208  6458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.002s	sys 0.024s
W20260709 12:28:17.850265  6458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.003s	sys 0.024s
W20260709 12:28:17.846748  7329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.002s	sys 0.010s
W20260709 12:28:17.838613 14174 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.006s	sys 0.017s
I20260709 12:28:17.787042 14398 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:28:17.838147 14047 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.840883 14314 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:17.876210  6454 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:17.877509  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:17.841004 14133 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:17.886591 14476 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:28:17.887903 14472 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:28:17.891852 14183 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.892144 14218 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.878213  6454 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:17.905438 14385 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:17.907744 14474 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:17.908394 14474 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.908886 13655 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.915166 14339 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:17.933812 14477 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:28:17.970711  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.019s	sys 0.039s
I20260709 12:28:17.938661 14218 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.946625 14447 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:17.803326 14350 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:17.949337 14414 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:17.951798 14254 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:17.871999 14047 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:17.866001 14318 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:17.969843 14470 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.160s	user 0.000s	sys 0.003s
W20260709 12:28:17.938011 14475 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:28:17.875126 14314 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:17.971072 14477 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:17.984611 13766 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:17.867022 14358 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:17.993850 14310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.005s	sys 0.024s
W20260709 12:28:17.998275 14214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.011s	sys 0.021s
I20260709 12:28:17.962075 14436 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:18.011755 14310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.005s	sys 0.025s
I20260709 12:28:17.957098  6453 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:17.900307  6450 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:28:18.123387  6450 pending_rounds.cc:85] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 10
W20260709 12:28:18.123894  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.124156  6450 pending_rounds.cc:107] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:28:18.124993  6450 raft_consensus.cc:2889] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:18.009831 13766 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:18.104883  6272 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.002s	sys 0.024s
W20260709 12:28:18.126042  6272 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.002s	sys 0.024s
W20260709 12:28:18.128163  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:17.975602  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.020s	sys 0.040s
W20260709 12:28:18.128870  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 907407us
W20260709 12:28:18.129261  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 811276us
W20260709 12:28:18.105228  6274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.002s	sys 0.024s
W20260709 12:28:18.131134  6274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.003s	sys 0.024s
W20260709 12:28:18.131448  6459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.007s	sys 0.020s
W20260709 12:28:18.132165  6459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.007s	sys 0.020s
W20260709 12:28:18.109191  6305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.003s	sys 0.021s
W20260709 12:28:18.133337  6305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.004s	sys 0.021s
I20260709 12:28:18.136876  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1231) took 1984 ms. Trace:
W20260709 12:28:18.109499  6308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.006s	sys 0.019s
I20260709 12:28:18.115996  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3233) took 1465 ms. Trace:
W20260709 12:28:18.139043  6308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.006s	sys 0.019s
I20260709 12:28:18.139969 14254 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:18.131916  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 838827us
I20260709 12:28:18.131148  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3788) took 1566 ms. Trace:
W20260709 12:28:18.122423  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.116451  6376 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.011s	sys 0.001s
W20260709 12:28:18.105509  6275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.014s	sys 0.013s
W20260709 12:28:18.117781  6451 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.017s	sys 0.022s
W20260709 12:28:18.122777  6452 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.017s	sys 0.041s
W20260709 12:28:18.120707  6383 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.007s	sys 0.027s
W20260709 12:28:18.117107  6425 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.265s	user 0.005s	sys 0.032s
W20260709 12:28:18.110612  6351 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.011s	sys 0.014s
W20260709 12:28:18.109824  6309 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.012s	sys 0.011s
W20260709 12:28:18.211752  6452 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.041s	user 0.018s	sys 0.041s
W20260709 12:28:18.212284  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70501.6us
I20260709 12:28:18.120396  6379 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:18.213778  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:18.021884 14474 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:18.217342  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 86866.8us
I20260709 12:28:18.221892 14318 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:18.225136  6351 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.011s	sys 0.015s
W20260709 12:28:18.226004  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 866223us
I20260709 12:28:18.130545 13766 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:18.137542  6305 rpcz_store.cc:276] 0709 12:28:16.152459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.204561 (+1052102us) service_pool.cc:224] Handling call
0709 12:28:17.205001 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:17.205496 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.205537 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:17.206316 (+   779us) log.cc:844] Serialized 52 byte log entry
0709 12:28:18.134299 (+927983us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:18.135101 (+   802us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.135799 (+   698us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.136062 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.136278 (+   216us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.136354 (+    76us) raft_consensus.cc:1685] finished
0709 12:28:18.136377 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.136668 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14401,"raft.run_cpu_time_us":449,"raft.run_wall_time_us":9343,"thread_start_us":896242,"threads_started":1,"wal-append.queue_time_us":841270}
I20260709 12:28:18.232250  6382 pending_rounds.cc:85] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 16
W20260709 12:28:18.232647  6275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.015s	sys 0.013s
I20260709 12:28:18.232842  6382 pending_rounds.cc:107] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 24.17
W20260709 12:28:18.233259  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 895825us
I20260709 12:28:18.233697  6382 raft_consensus.cc:2889] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:18.014393 14214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.012s	sys 0.021s
W20260709 12:28:18.239871  6383 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.007s	sys 0.028s
I20260709 12:28:18.241602  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3521) took 1705 ms. Trace:
I20260709 12:28:18.071208 14454 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:18.243220 14454 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:18.244099 14398 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:18.245117 14398 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.244858 14454 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.246879 13655 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:18.230643  6309 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.013s	sys 0.011s
I20260709 12:28:18.233827  6305 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
W20260709 12:28:18.250725  6425 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.399s	user 0.005s	sys 0.032s
W20260709 12:28:18.249881  6376 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.011s	sys 0.002s
I20260709 12:28:18.247768 13655 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:18.253544  6451 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.018s	sys 0.022s
I20260709 12:28:18.242044  6383 rpcz_store.cc:276] 0709 12:28:16.536753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.536929 (+   176us) service_pool.cc:224] Handling call
0709 12:28:16.561621 (+ 24692us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:16.562040 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.562074 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:16.562758 (+   684us) log.cc:844] Serialized 52 byte log entry
0709 12:28:17.146555 (+583797us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:18.240577 (+1094022us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.241098 (+   521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.241230 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.241317 (+    87us) raft_consensus.cc:1685] finished
0709 12:28:18.241329 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.241547 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":893877,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":93,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":19191,"thread_start_us":1568265,"threads_started":2,"wal-append.queue_time_us":982122}
I20260709 12:28:18.275071  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3436) took 1509 ms. Trace:
I20260709 12:28:18.275671  6305 rpcz_store.cc:276] 0709 12:28:16.759279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.232565 (+1473286us) service_pool.cc:224] Handling call
0709 12:28:18.233253 (+   688us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:18.253232 (+ 19979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.268468 (+ 15236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:18.278185  6305 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [18.12-20.13]
W20260709 12:28:18.278273  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 1.42661e+06us
I20260709 12:28:18.279655  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.281272  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.192375  6458 rpcz_store.cc:276] 0709 12:28:16.564469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.092232 (+527763us) service_pool.cc:224] Handling call
0709 12:28:17.124532 (+ 32300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.124781 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.124819 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.136253 (+ 11434us) log.cc:844] Serialized 26 byte log entry
0709 12:28:17.850854 (+714601us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:17.850924 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.851018 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.130422 (+279404us) raft_consensus.cc:1685] finished
0709 12:28:18.130463 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.130861 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":657163,"threads_started":1,"wal-append.queue_time_us":858444}
I20260709 12:28:18.324138  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3103) took 1357 ms. Trace:
I20260709 12:28:18.139164  6380 rpcz_store.cc:276] 0709 12:28:16.650400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.650564 (+   164us) service_pool.cc:224] Handling call
0709 12:28:16.650954 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:16.651211 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.651246 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:16.722124 (+ 70878us) spinlock_profiling.cc:243] Waited 986 us on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:16.722717 (+   593us) log.cc:844] Serialized 26 byte log entry
0709 12:28:17.199899 (+477182us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:17.199945 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.200017 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:17.968623 (+768606us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:18.115083 (+146460us) raft_consensus.cc:1685] finished
0709 12:28:18.115157 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.115647 (+   490us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12716544,"thread_start_us":476365,"threads_started":1,"wal-append.queue_time_us":900983}
I20260709 12:28:18.339840  6379 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:18.338130  6305 rpcz_store.cc:276] 0709 12:28:16.966748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.276943 (+1310195us) service_pool.cc:224] Handling call
0709 12:28:18.277649 (+   706us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:18.285017 (+  7368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.324036 (+ 39019us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:18.357944  6379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3530) took 1462 ms (client timeout 1694 ms). Trace:
I20260709 12:28:18.049400 14444 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:18.373296  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91149.1us
W20260709 12:28:18.372076  6379 rpcz_store.cc:269] 0709 12:28:16.885046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.923858 (+ 38812us) service_pool.cc:224] Handling call
0709 12:28:17.242324 (+318466us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54000436a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa321edc0 00007f6fa32aae18 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:18.118415 (+876091us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b58006e60d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa329dfe7 00007f6fa329f7de 00007f6fa32ab27c 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c
0709 12:28:18.346706 (+228291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010598400}
W20260709 12:28:18.351756 14443 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.006s	sys 0.012s
W20260709 12:28:18.391623  6457 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.009s	sys 0.013s
W20260709 12:28:18.395704 14443 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.007s	sys 0.013s
W20260709 12:28:18.400974  6457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.009s	sys 0.013s
I20260709 12:28:18.372571 13766 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:18.424232  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.438020  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.430883  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:18.453537 14268 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.011s	sys 0.001s
I20260709 12:28:18.453848  6383 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
W20260709 12:28:18.454111  6377 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.011s	sys 0.007s
W20260709 12:28:18.454470 14192 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.007s	sys 0.012s
W20260709 12:28:18.454948 14460 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.026s	sys 0.060s
W20260709 12:28:18.455530 14460 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.027s	sys 0.061s
I20260709 12:28:18.455659  6383 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
W20260709 12:28:18.457062 14322 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.006s	sys 0.012s
W20260709 12:28:18.457862 14322 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.006s	sys 0.012s
W20260709 12:28:18.458062  6381 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.011s	sys 0.006s
W20260709 12:28:18.458595  6381 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.011s	sys 0.007s
W20260709 12:28:18.458896  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.459836  6378 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.007s	sys 0.010s
W20260709 12:28:18.460348  6378 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.007s	sys 0.011s
I20260709 12:28:18.460902  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:18.461971  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3515) took 2246 ms. Trace:
W20260709 12:28:18.464835 14111 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 23 current term: 23 is higher.
W20260709 12:28:18.465680 14400 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.014s
W20260709 12:28:18.466254 14400 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.005s	sys 0.014s
W20260709 12:28:18.453896  6302 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.011s	sys 0.012s
W20260709 12:28:18.467942  6302 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.011s	sys 0.012s
W20260709 12:28:18.469161  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 255706us
I20260709 12:28:18.457796  6383 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:18.470415  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:18.478636  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 281265us
I20260709 12:28:18.478705 14512 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:18.480221  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:18.454602  6377 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.012s	sys 0.007s
I20260709 12:28:18.460139 14454 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:18.457130 14284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.005s	sys 0.006s
I20260709 12:28:18.482455  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:18.485286  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3792) took 1765 ms. Trace:
I20260709 12:28:18.485294  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1197) took 1832 ms. Trace:
W20260709 12:28:18.509739 14284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.006s	sys 0.006s
W20260709 12:28:18.510242 14469 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
W20260709 12:28:18.483404  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1235) took 2329 ms (client timeout 2179 ms). Trace:
W20260709 12:28:18.511004 14268 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.011s	sys 0.001s
W20260709 12:28:18.511205  6300 rpcz_store.cc:269] 0709 12:28:16.153429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.858421 (+704992us) service_pool.cc:224] Handling call
0709 12:28:18.482775 (+1624354us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54000b3954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:18.483099 (+   324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":677844736}
I20260709 12:28:18.512380  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:18.513310  6453 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:18.515458  6307 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:28:18.462388  6378 rpcz_store.cc:276] 0709 12:28:16.215521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.259235 (+ 43714us) service_pool.cc:224] Handling call
0709 12:28:16.259822 (+   587us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:16.658824 (+399002us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:16.658848 (+    24us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:16.659321 (+   473us) log.cc:844] Serialized 52 byte log entry
0709 12:28:17.504585 (+845264us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:18.460933 (+956348us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.461417 (+   484us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.461565 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.461656 (+    91us) raft_consensus.cc:1685] finished
0709 12:28:18.461675 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.461888 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54648,"prepare.queue_time_us":539435,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":51970,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":9056,"thread_start_us":1666996,"threads_started":2,"wal-append.queue_time_us":941572}
I20260709 12:28:18.518699  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:18.519697  6310 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.782s	user 0.004s	sys 0.019s
I20260709 12:28:18.519857  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:18.520182  6310 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.004s	sys 0.020s
I20260709 12:28:18.023366 14477 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.523516 14477 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:18.457779  6301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.880s	user 0.003s	sys 0.017s
W20260709 12:28:18.527138  6301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.011s	user 0.003s	sys 0.017s
I20260709 12:28:18.528456  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:18.528851 14475 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.001s
I20260709 12:28:18.488476  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:18.529616  6304 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:18.510385 13655 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:18.485860 14192 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.008s	sys 0.012s
I20260709 12:28:18.485599 14400 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:18.446745 14303 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.504s	user 0.008s	sys 0.004s
W20260709 12:28:18.547713  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.453617 14333 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:18.550144  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.484489 14318 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:18.557778  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1182) took 1245 ms. Trace:
I20260709 12:28:18.558909  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.487998 14398 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:18.507454  6377 rpcz_store.cc:276] 0709 12:28:16.652877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.164646 (+511769us) service_pool.cc:224] Handling call
0709 12:28:17.164899 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.165102 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:17.797275 (+632173us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.482124 (+684849us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.482402 (+   278us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.482468 (+    66us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:18.484881 (+  2413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.484964 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.485222 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5137,"prepare.queue_time_us":879328,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":10194,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":206,"thread_start_us":1310276,"threads_started":2,"wal-append.queue_time_us":657844}
I20260709 12:28:18.565676  7335 rpcz_store.cc:276] 0709 12:28:17.312193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.312334 (+   141us) service_pool.cc:224] Handling call
0709 12:28:18.557282 (+1244948us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54006c16d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:18.557506 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2570260736}
I20260709 12:28:18.532546  6451 rpcz_store.cc:276] 0709 12:28:16.719917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.163767 (+443850us) service_pool.cc:224] Handling call
0709 12:28:17.164086 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:17.164462 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.164490 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:17.165013 (+   523us) log.cc:844] Serialized 52 byte log entry
0709 12:28:18.484554 (+1319541us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:18.484693 (+   139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.484864 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.484914 (+    50us) raft_consensus.cc:1685] finished
0709 12:28:18.484929 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.485191 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5943,"raft.run_cpu_time_us":64,"raft.run_wall_time_us":64,"thread_start_us":1046371,"threads_started":1,"wal-append.queue_time_us":868318}
I20260709 12:28:18.581286  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:18.581931 14318 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.585127 14318 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:18.587062  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3241) took 1125 ms. Trace:
I20260709 12:28:18.586756 14303 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.587560  6376 rpcz_store.cc:276] 0709 12:28:17.461944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.462143 (+   199us) service_pool.cc:224] Handling call
0709 12:28:17.487083 (+ 24940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.487331 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:18.279032 (+791701us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.565750 (+286718us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.575651 (+  9901us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:18.586654 (+ 11003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.586788 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.586997 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1132,"prepare.queue_time_us":556497,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":11623,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":9185,"thread_start_us":1011924,"threads_started":2}
I20260709 12:28:18.589619 14303 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:18.592314 14474 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:18.592844 14474 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:18.593286 14474 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:18.598562 14452 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:18.600265  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3424) took 1939 ms. Trace:
I20260709 12:28:18.600778  6307 rpcz_store.cc:276] 0709 12:28:16.660560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.496726 (+836166us) service_pool.cc:224] Handling call
0709 12:28:18.514786 (+1018060us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:18.516007 (+  1221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.599986 (+ 83979us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54000b3950. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:18.600183 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280270080}
W20260709 12:28:18.602368 14505 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.129s	user 0.000s	sys 0.002s
I20260709 12:28:18.604387  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3428) took 1902 ms. Trace:
I20260709 12:28:18.582624  6377 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:18.613267  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:18.614264 14471 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.615033 14471 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:18.616983 14047 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.623s	user 0.012s	sys 0.004s
I20260709 12:28:18.618119 14047 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.586783 14333 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.619724 14218 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:18.620329 14218 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:18.620664 14333 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:18.620680 14047 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:18.621544 14218 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:18.623309  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.618670  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 136436us
I20260709 12:28:18.629129 14313 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:18.629721 14313 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:18.630237  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:18.631173 14313 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.632550 14313 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:18.562227 14400 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.604921  6310 rpcz_store.cc:276] 0709 12:28:16.701990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.641488 (+939498us) service_pool.cc:224] Handling call
0709 12:28:17.641957 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.642449 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.642513 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.643167 (+   654us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.520975 (+877808us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:18.521129 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.521261 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.598352 (+ 77091us) spinlock_profiling.cc:243] Waited 85.3 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:18.603869 (+  5517us) raft_consensus.cc:1685] finished
0709 12:28:18.603897 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.604319 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12074,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":56858,"spinlock_wait_cycles":196167552,"thread_start_us":877163,"threads_started":1,"wal-append.queue_time_us":829093}
I20260709 12:28:18.637008  6307 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:28:18.637095  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 109015us
I20260709 12:28:18.640915  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3443) took 1410 ms. Trace:
I20260709 12:28:18.641305  6307 rpcz_store.cc:276] 0709 12:28:17.230129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.602001 (+1371872us) service_pool.cc:224] Handling call
0709 12:28:18.602800 (+   799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:18.638737 (+ 35937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.640865 (+  2128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:18.634469 14471 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:18.649606 14186 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:18.650404 14186 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.651015 14186 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:18.651464 14398 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:18.651969 14398 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.652525 14398 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:18.655174  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3535) took 1122 ms. Trace:
I20260709 12:28:18.655552  6454 rpcz_store.cc:276] 0709 12:28:17.532579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.901382 (+368803us) service_pool.cc:224] Handling call
0709 12:28:17.901681 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.901995 (+   314us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.902019 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:18.246797 (+344778us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b2c0012cca8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:18.247737 (+   940us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.565448 (+317711us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:18.570154 (+  4706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.586654 (+ 16500us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.641437 (+ 54783us) raft_consensus.cc:1685] finished
0709 12:28:18.641474 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.654977 (+ 13503us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10297,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":121,"spinlock_wait_cycles":503982720,"thread_start_us":312105,"threads_started":1,"wal-append.queue_time_us":383482}
I20260709 12:28:18.661365  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:18.659432  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:18.662030  6453 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:28:18.641382 14400 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:18.664212  6377 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:18.666131 13655 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:18.696943  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:18.697945  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.690804  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3444) took 1459 ms. Trace:
I20260709 12:28:18.699760  6310 rpcz_store.cc:276] 0709 12:28:17.231211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.640320 (+1409109us) service_pool.cc:224] Handling call
0709 12:28:18.646007 (+  5687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:18.684904 (+ 38897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:18.684954 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.690418 (+  5464us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:18.690467 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.690558 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.690724 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:18.701709  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 241078us
I20260709 12:28:18.704068  7335 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:28:18.705542 13655 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.706149  7319 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:18.707716 13655 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:18.707945  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:18.696920  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:18.720422 14474 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.724229 14474 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:18.725286  6453 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:18.726637  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:18.727754  7316 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:18.726778  6453 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:18.731797  6451 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:18.733979  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:18.734112  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:18.729759  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:18.740433 14303 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:18.740943 14303 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:18.741345 14303 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:18.745905 14523 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.265s	user 0.000s	sys 0.002s
W20260709 12:28:18.747397 14508 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.271s	user 0.000s	sys 0.003s
I20260709 12:28:18.748816  7316 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:28:18.759502  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3431) took 2039 ms. Trace:
I20260709 12:28:18.765473  6450 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:18.766726  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:18.768538  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:18.769033  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 123252us
I20260709 12:28:18.769323  7316 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:18.770744 14414 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:18.771454  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3789) took 2063 ms. Trace:
I20260709 12:28:18.772023  6459 rpcz_store.cc:276] 0709 12:28:16.708499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.115988 (+407489us) service_pool.cc:224] Handling call
0709 12:28:17.116387 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.176012 (+ 59625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.176045 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.176702 (+   657us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.133211 (+956509us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:18.133358 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.770840 (+637482us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b5400265f54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:18.770926 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.770999 (+    73us) raft_consensus.cc:1685] finished
0709 12:28:18.771016 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.771366 (+   350us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":135868800,"thread_start_us":955494,"threads_started":1,"wal-append.queue_time_us":876914}
I20260709 12:28:18.773841  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:18.762178  6302 rpcz_store.cc:276] 0709 12:28:16.719917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.652754 (+932837us) service_pool.cc:224] Handling call
0709 12:28:17.653151 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.653433 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.653471 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.654073 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.468665 (+814592us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:18.468781 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.468924 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.734973 (+266049us) spinlock_profiling.cc:243] Waited 82.4 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:18.749513 (+ 14540us) raft_consensus.cc:1685] finished
0709 12:28:18.749546 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.759410 (+  9864us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":189411200,"thread_start_us":748372,"threads_started":1,"wal-append.queue_time_us":756028}
W20260709 12:28:18.776475 14507 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.304s	user 0.003s	sys 0.002s
I20260709 12:28:18.777855  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:18.778848 14518 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.301s	user 0.004s	sys 0.000s
I20260709 12:28:18.785670  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3531) took 1271 ms. Trace:
W20260709 12:28:18.787447  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53546.5us
I20260709 12:28:18.787266  6457 rpcz_store.cc:276] 0709 12:28:17.514317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.657216 (+142899us) service_pool.cc:224] Handling call
0709 12:28:17.716543 (+ 59327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.716849 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.716885 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.717642 (+   757us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.464521 (+746879us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:18.464608 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.464734 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.775533 (+310799us) spinlock_profiling.cc:243] Waited 94.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:18.785336 (+  9803us) raft_consensus.cc:1685] finished
0709 12:28:18.785357 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.785589 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216419072,"thread_start_us":695949,"threads_started":1,"wal-append.queue_time_us":753510}
I20260709 12:28:18.789337  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:18.791124  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:18.792130 14218 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:18.792752 14218 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:18.792789  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:18.793345 14398 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:18.778041 14358 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.765s	user 0.013s	sys 0.007s
I20260709 12:28:18.794639 14436 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:18.794838 14414 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:18.794195 14218 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.795689 14474 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:18.796535  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:18.797587 14123 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:18.795516 14414 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:18.799549 14414 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.801608  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:18.795396 13655 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:18.802549 14123 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:18.803104 14314 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.008s
I20260709 12:28:18.803318 14123 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:18.803752  7288 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 3.228s (SENT)
I20260709 12:28:18.804361  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:18.804714 14123 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:18.807389  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:18.807168 14385 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.901s	user 0.007s	sys 0.012s
I20260709 12:28:18.808432 14453 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:18.808758 14509 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.324s	user 0.004s	sys 0.000s
I20260709 12:28:18.808897 14453 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.810360  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3420) took 2163 ms. Trace:
I20260709 12:28:18.810776  6304 rpcz_store.cc:276] 0709 12:28:16.646534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.410282 (+763748us) service_pool.cc:224] Handling call
0709 12:28:17.410637 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:18.563125 (+1152488us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.810063 (+246938us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400093dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:18.810287 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2572132096}
W20260709 12:28:18.813011 14537 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
W20260709 12:28:18.813282 14531 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.257s	user 0.001s	sys 0.001s
I20260709 12:28:18.814594 14192 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:18.815169  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:18.814702  6454 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:18.818840 14521 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.340s	user 0.005s	sys 0.002s
I20260709 12:28:18.824323  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3242) took 1361 ms. Trace:
W20260709 12:28:18.825089  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51458.8us
I20260709 12:28:18.827594  6385 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:18.829797  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:18.830547  7316 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:18.833082 14178 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.833738 14178 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.834472  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.835512  6385 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:18.836036 14178 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:18.836660 14178 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:18.727922 14186 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.840572  6454 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:18.797731  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:18.803732 14314 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.850170 14344 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:18.850664 14344 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:18.857342  7316 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
W20260709 12:28:18.861639  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57964.1us
I20260709 12:28:18.863221  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3423) took 2211 ms. Trace:
I20260709 12:28:18.863765  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1232) took 2711 ms. Trace:
W20260709 12:28:18.852981 14526 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.372s	user 0.001s	sys 0.002s
I20260709 12:28:18.864626 14350 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.865265 14350 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:18.866302 14510 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.410s	user 0.001s	sys 0.002s
W20260709 12:28:18.864106 14534 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.165s	user 0.004s	sys 0.000s
I20260709 12:28:18.844411 14192 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:18.869374  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:18.851420 14314 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:18.824694  6381 rpcz_store.cc:276] 0709 12:28:17.462666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.463469 (+   803us) service_pool.cc:224] Handling call
0709 12:28:17.470291 (+  6822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.630627 (+160336us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:18.459206 (+828579us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.615227 (+156021us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.615429 (+   202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.615489 (+    60us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:18.615586 (+    97us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.824086 (+208500us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400038754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:18.824129 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.824269 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31742,"prepare.queue_time_us":413803,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":44371,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":3597,"spinlock_wait_cycles":377754752,"thread_start_us":847378,"threads_started":2,"wal-append.queue_time_us":188983}
I20260709 12:28:18.884924 14495 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.863636  6309 rpcz_store.cc:276] 0709 12:28:16.651525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.485642 (+834117us) service_pool.cc:224] Handling call
0709 12:28:17.486205 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:17.486839 (+   634us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:18.279747 (+792908us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.812506 (+532759us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:18.862278 (+ 49772us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.862346 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:18.862404 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.862511 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.863155 (+   644us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":401447,"prepare.queue_time_us":556251,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"spinlock_wait_cycles":5916032,"thread_start_us":915751,"threads_started":2,"wal-append.queue_time_us":276056}
I20260709 12:28:18.896261  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3104) took 1929 ms. Trace:
I20260709 12:28:18.867849 14350 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:18.905735 14495 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:18.906400 14502 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.431s	user 0.000s	sys 0.004s
I20260709 12:28:18.910669  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1183) took 1541 ms. Trace:
W20260709 12:28:18.921943  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61078.3us
I20260709 12:28:18.911386 14499 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:18.898658  6305 rpcz_store.cc:276] 0709 12:28:16.967104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.360462 (+1393358us) service_pool.cc:224] Handling call
0709 12:28:18.423529 (+ 63067us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:18.482800 (+ 59271us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:18.550146 (+ 67346us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.895781 (+345635us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:18.895848 (+    67us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:18.895913 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.895994 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.896161 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":237592,"prepare.queue_time_us":73197,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"thread_start_us":139769,"threads_started":1,"wal-append.queue_time_us":184515}
I20260709 12:28:18.851687 14344 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:18.912726 14178 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:18.911417 14322 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:18.876231 14385 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:18.864212  6308 rpcz_store.cc:276] 0709 12:28:16.152689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.301803 (+1149114us) service_pool.cc:224] Handling call
0709 12:28:17.318792 (+ 16989us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:18.139829 (+821037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:18.139879 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:18.140741 (+   862us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.445725 (+304984us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:18.510754 (+ 65029us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.863356 (+352602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.863469 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.863520 (+    51us) raft_consensus.cc:1685] finished
0709 12:28:18.863531 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.863689 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":368593,"prepare.queue_time_us":104147,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":91004,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":1105516,"threads_started":2,"wal-append.queue_time_us":455371}
I20260709 12:28:18.916061  7335 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:28:18.933602  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:18.918614 14192 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:18.923812  7331 rpcz_store.cc:276] 0709 12:28:17.335995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.336161 (+   166us) service_pool.cc:224] Handling call
0709 12:28:17.347498 (+ 11337us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:17.847189 (+499691us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.847227 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:17.871544 (+ 24317us) log.cc:844] Serialized 52 byte log entry
0709 12:28:18.663506 (+791962us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:18.782917 (+119411us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.843206 (+ 60289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.854597 (+ 11391us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.854663 (+    66us) raft_consensus.cc:1685] finished
0709 12:28:18.854688 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:18.877529 (+ 22841us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9823,"prepare.queue_time_us":172297,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":217143,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":18241,"thread_start_us":710230,"threads_started":3,"wal-append.queue_time_us":354492}
I20260709 12:28:18.950840  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:18.960808  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:18.964545  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 554351us
W20260709 12:28:18.971284 14491 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.510s	user 0.000s	sys 0.004s
W20260709 12:28:18.972540  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 174964us
I20260709 12:28:18.980312 14344 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:18.978394 14477 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:18.982307 14477 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:18.982790 14477 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:18.983108 14178 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:18.983641 14174 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:18.986398 14047 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:18.987445  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:18.988804  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:18.989856 14550 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:28:18.994817 14454 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:18.996445 14454 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.003352  7329 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1181) took 1688 ms. Trace:
I20260709 12:28:19.003554 14047 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:19.003693  7319 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:19.004420 14444 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:19.005343 13655 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:18.996902 14549 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
W20260709 12:28:19.009318  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:18.978843 14186 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.018692 13655 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.013314 14444 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 12:28:19.021433 13655 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.025030 14495 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.000860 14398 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.006994  7329 rpcz_store.cc:276] 0709 12:28:17.311833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.312002 (+   169us) service_pool.cc:224] Handling call
0709 12:28:17.312383 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.327396 (+ 15013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.327444 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.327955 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:28:17.851492 (+523537us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:17.851549 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:17.851636 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:17.920524 (+ 68888us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b48000cd758. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:18.941498 (+1020974us) spinlock_profiling.cc:243] Waited 31.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:18.974777 (+ 33279us) raft_consensus.cc:1685] finished
0709 12:28:18.974808 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.000279 (+ 25471us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":73023,"raft.queue_time_us":76113,"raft.run_cpu_time_us":11841,"raft.run_wall_time_us":531822,"spinlock_wait_cycles":107312896,"thread_start_us":523178,"threads_started":1,"wal-append.queue_time_us":1116261}
W20260709 12:28:19.035506 14544 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:28:19.038880 14547 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:28:19.023914 14111 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:28:19.040957 14511 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.476s	user 0.002s	sys 0.000s
I20260709 12:28:19.041582 14192 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.042727 14174 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:19.043452 14174 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:19.004765 14047 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:19.063315  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 193392us
I20260709 12:28:19.063396  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:19.064309 14400 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.065212  6384 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:28:19.066448  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3425) took 2397 ms. Trace:
I20260709 12:28:19.067056  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:19.066887  6301 rpcz_store.cc:276] 0709 12:28:16.669568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.513931 (+844363us) service_pool.cc:224] Handling call
0709 12:28:17.514245 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.514739 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:28:17.514786 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.515410 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.527958 (+1012548us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:18.528106 (+   148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.528254 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.875374 (+347120us) spinlock_profiling.cc:243] Waited 83.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:19.045983 (+170609us) raft_consensus.cc:1685] finished
0709 12:28:19.046016 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.066375 (+ 20359us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":20701,"spinlock_wait_cycles":191192960,"thread_start_us":1011895,"threads_started":1,"wal-append.queue_time_us":959308}
I20260709 12:28:19.071516 14358 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:19.073119 14444 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:19.073693 14444 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:19.075119 14444 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:19.076810  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50199.6us
I20260709 12:28:19.044759 14174 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.068809  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:19.101488  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 104737us
W20260709 12:28:19.104020  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.106365  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51308.2us
I20260709 12:28:19.099692 14452 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:19.109543 14444 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:19.140767 13772 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:19.142129 14268 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:19.146999  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3433) took 2307 ms. Trace:
I20260709 12:28:19.147315  7285 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:19.126857 14398 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:28:19.147517  6303 rpcz_store.cc:276] 0709 12:28:16.720438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.658116 (+937678us) service_pool.cc:224] Handling call
0709 12:28:17.658422 (+   306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:18.115260 (+456838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:18.115371 (+   111us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:18.116936 (+  1565us) log.cc:844] Serialized 52 byte log entry
0709 12:28:18.444198 (+327262us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:18.444259 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.808668 (+364409us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b18004e8680. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa717b58f 00007f6fa01ba571 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd
0709 12:28:18.852124 (+ 43456us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:19.011299 (+159175us) spinlock_profiling.cc:243] Waited 451 ms on lock 0x7b54000b3454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:19.011359 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:19.011442 (+    83us) raft_consensus.cc:1685] finished
0709 12:28:19.011464 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.027646 (+ 16182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7103,"raft.queue_time_us":24542,"raft.run_cpu_time_us":359,"raft.run_wall_time_us":358,"spinlock_wait_cycles":1179681152,"thread_start_us":252330,"threads_started":1,"wal-append.queue_time_us":357055}
I20260709 12:28:19.149050  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:19.149708 14214 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:19.150286 14214 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.151794 14512 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:19.152268 14512 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.152671 14512 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:19.154736 14554 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.155251 14499 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:19.155632 14471 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:19.156087 14499 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.156772 14471 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:19.157444 14471 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:19.159075  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:19.160243 14350 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:19.160976 14183 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:19.160846 14350 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.162777 13655 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:19.099470 14284 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 12:28:19.166209  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.172593 14554 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.173661 14350 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.174414  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:19.162709 14499 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.099191 14414 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:28:19.190932  6384 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:19.138011 14400 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.199988  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:19.218233 14551 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:28:19.220307 14303 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.222709 14385 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.223420 14385 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.225993 14385 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.227202 14453 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.228132  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:19.245709  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 113704us
I20260709 12:28:19.255501 14358 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:19.257043 14453 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:28:19.258236  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 160350us
W20260709 12:28:19.274703  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 125775us
I20260709 12:28:19.279874  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.280862  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.281648  6303 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:19.193823 14452 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.282760  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.297536  6301 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:19.301739  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.257556 14453 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:19.300406 13655 consensus_queue.cc:1243] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:28:19.263955 14254 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.198897 14471 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.326100  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.308960  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:19.320155 14303 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.369693  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:19.365593 14314 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:19.374218  6308 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.379035  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:19.378031  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 80245.6us
I20260709 12:28:19.369835 14178 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.378806  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.391975  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.353973 14254 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.395349  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.380563 14314 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:19.397109  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:19.403415  7319 raft_consensus.cc:2410] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:19.404445  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.405277  7316 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:19.405598  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:19.406594  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:19.407196 14111 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 11) took 3171 ms. Trace:
I20260709 12:28:19.408630  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.409700 14214 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.409986  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.410915  6301 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 30: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:19.402755  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.407660 14111 rpcz_store.cc:276] 0709 12:28:16.214325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:16.214469 (+   144us) service_pool.cc:224] Handling call
0709 12:28:16.374459 (+159990us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b4c002ae700. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1573a20 00007f6fa1584a29 00007f6fa154a644 00007f6fa1571f2d 00007f6fa1571b93 00007f6fa32001af 00007f6fa31fff3e 00007f6fa3285888 00007f6fa32a275c 00007f6fa32a376a 00007f6fa32a291b
0709 12:28:16.602909 (+228450us) log.cc:844] Serialized 485 byte log entry
0709 12:28:16.649250 (+ 46341us) spinlock_profiling.cc:243] Waited 8.29 ms on lock 0x7b18003a7a40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa3200b0b 00007f6fa3285b63 00007f6fa32ad752 00007f6fa826dd29
0709 12:28:16.817921 (+168671us) spinlock_profiling.cc:243] Waited 27.9 ms on lock 0x7b54003a5820. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3227b26 00007f6fa322149a 00007f6fa3220714 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634
0709 12:28:16.821564 (+  3643us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b18003a7a40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:16.822235 (+   671us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b18003a7a40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:19.378417 (+2556182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31760,"raft.queue_time_us":82364,"raft.run_cpu_time_us":4495,"raft.run_wall_time_us":303735,"spinlock_wait_cycles":247148544}
I20260709 12:28:19.416589  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:19.416937 14558 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:28:19.417662  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.417980  6301 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:19.419212 14556 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:28:19.420037 14522 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.946s	user 0.004s	sys 0.002s
W20260709 12:28:19.422034 14557 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
W20260709 12:28:19.422216  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.422643  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 118900us
I20260709 12:28:19.423454  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3442) took 2193 ms. Trace:
I20260709 12:28:19.423918  6300 rpcz_store.cc:276] 0709 12:28:17.229776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.512430 (+1282654us) service_pool.cc:224] Handling call
0709 12:28:18.512915 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:18.940208 (+427293us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:18.940258 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:18.977093 (+ 36835us) log.cc:844] Serialized 26 byte log entry
0709 12:28:19.115052 (+137959us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:19.115112 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:19.115205 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:19.423230 (+308025us) raft_consensus.cc:1685] finished
0709 12:28:19.423251 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.423389 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349896,"raft.queue_time_us":397531,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":158621,"threads_started":2,"wal-append.queue_time_us":261346}
I20260709 12:28:19.425631  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:19.429464  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3532) took 1914 ms. Trace:
I20260709 12:28:19.431135 14453 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.432775 14453 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:19.434970 14560 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:28:19.437980  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.439306 14452 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.440202  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.440974  6455 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:19.442705 14554 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.443642  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:19.444293  7317 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.389467 14178 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.447739 13772 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:19.448351 13772 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:19.449082 14178 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.450637 14268 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.451120 14254 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.451727  6305 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:19.412113 14484 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.953s	user 0.000s	sys 0.006s
I20260709 12:28:19.401930 14314 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:19.460074  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 136943us
I20260709 12:28:19.462651  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.429946  6456 rpcz_store.cc:276] 0709 12:28:17.521655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.897254 (+375599us) service_pool.cc:224] Handling call
0709 12:28:17.897755 (+   501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.898276 (+   521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.898320 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:18.131298 (+232978us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.429660 (+298362us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:18.443305 (+ 13645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.453735 (+ 10430us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:19.412211 (+958476us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:19.428947 (+ 16736us) raft_consensus.cc:1685] finished
0709 12:28:19.428969 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.429269 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9264,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":140,"spinlock_wait_cycles":44797824,"thread_start_us":280337,"threads_started":1,"wal-append.queue_time_us":341746}
I20260709 12:28:19.462359 14452 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:19.451239 14268 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.463070  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:19.465528 14313 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:19.468502 14268 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.468757 14577 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.469405 14577 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.449595 13772 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.464286 14452 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.476940  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.478972  7335 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:19.487949 13772 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:19.488592 14136 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.489292 14136 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.491539 14136 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.509299 14400 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:19.509801 14400 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:19.510955  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.511940  6453 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:19.514653 14577 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.518159  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.519160  6451 raft_consensus.cc:2410] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:19.492022  7317 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
W20260709 12:28:19.492966 14517 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.013s	user 0.003s	sys 0.002s
I20260709 12:28:19.488178 14435 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.496515  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:19.499629  6300 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:19.500041  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.500162  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.500901  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:19.501161  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:19.535292  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.503983 14322 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:19.504109 14452 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.519663  6454 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.525568  6308 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.488662  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3790) took 2779 ms. Trace:
W20260709 12:28:19.482767  6351 connection.cc:256] RPC call timeout handler was delayed by -1.4628658800002086s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2758s for a response
I20260709 12:28:19.529018  6305 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
W20260709 12:28:19.540175  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 43771.4us
W20260709 12:28:19.539034  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 56772.6us
I20260709 12:28:19.543388  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.544301  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:19.546000  6351 connection.cc:256] RPC call timeout handler was delayed by -1.1731228509997891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
W20260709 12:28:19.546929  6351 connection.cc:256] RPC call timeout handler was delayed by -1.0911915339997904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
I20260709 12:28:19.551506  6456 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
W20260709 12:28:19.553953  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 111832us
I20260709 12:28:19.555467  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.569455  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:19.537303  6452 rpcz_store.cc:276] 0709 12:28:16.708797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:17.145659 (+436862us) service_pool.cc:224] Handling call
0709 12:28:17.145962 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:17.146292 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:17.146323 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:17.170278 (+ 23955us) log.cc:844] Serialized 26 byte log entry
0709 12:28:18.212962 (+1042684us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:18.213117 (+   155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.213282 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:19.381578 (+1168296us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:19.484847 (+103269us) raft_consensus.cc:1685] finished
0709 12:28:19.484883 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.488577 (+  3694us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23803,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":121,"spinlock_wait_cycles":76656000,"thread_start_us":1041792,"threads_started":1,"wal-append.queue_time_us":1263319}
I20260709 12:28:19.573706 14512 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.580679  6301 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.514055 14400 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.584766  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:19.563786 14528 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.035s	user 0.005s	sys 0.009s
I20260709 12:28:19.588858  6454 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:19.591694  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.592512  6450 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:19.563843 14471 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:19.596004  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.597648  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:19.564352 14123 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:19.597944  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:19.600462  6385 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:19.626463  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 200909us
I20260709 12:28:19.633369  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:19.633879  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:19.594172 14477 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.582s	user 0.011s	sys 0.002s
I20260709 12:28:19.576536 14512 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.571102 14495 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:19.634514  6453 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:19.585773 14573 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.640373 14579 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:19.599416 14444 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.581481 14435 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.644860  6454 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:19.629900  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:19.647418  6452 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.597383 14471 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.646351 14477 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.659420  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.660013  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.660550  6304 raft_consensus.cc:2410] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:28:19.660753  6310 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:19.661861  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:19.662322 14400 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.662807 14495 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.663349  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:19.665175 14495 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:19.666882  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.666957  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.667841  6451 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.669603  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.670264  6453 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:19.671406 14566 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:19.671872 14566 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:19.672907 14566 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.661803  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.642710 14111 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.674837  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:19.675258  6310 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:19.675330 14111 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.677668 14111 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.677734  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:19.667879  6300 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:28:19.653168  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:19.659319  6302 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:19.689172  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:19.690407  6304 raft_consensus.cc:2368] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21.
I20260709 12:28:19.682384  6383 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:19.661952  6457 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:19.686182  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:19.674304 14566 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.593214 14322 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.702611  6452 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.661777 14477 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:19.704986  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3531) took 1475 ms. Trace:
I20260709 12:28:19.684338 14471 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:19.707927  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.747419  6455 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:19.706734  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53656.5us
I20260709 12:28:19.705933  6301 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.740677  6424 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:19.737136  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:19.751681  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.754861  6308 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:19.750878 14214 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:19.762122  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.762313  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:19.763202  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53737.3us
I20260709 12:28:19.764673  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:19.765136  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:19.758509  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:19.766446  6304 raft_consensus.cc:2376] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21. Re-sending same reply.
I20260709 12:28:19.740180 14344 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.777930 14344 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.763778 14495 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:19.778380 14435 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:19.780256 14344 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:19.780097 14314 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.781248  6458 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:19.781473  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:19.762310 14214 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.782697  7332 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:28:19.786353 14494 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:19.787599 14495 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:19.787971  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 199104us
I20260709 12:28:19.772428  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.777364 14580 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:19.783237 14549 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.772053  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.782778  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:19.801631  6453 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.738039  6382 rpcz_store.cc:276] 0709 12:28:18.229947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:18.230755 (+   808us) service_pool.cc:224] Handling call
0709 12:28:18.231575 (+   820us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:18.235849 (+  4274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:18.235890 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:18.236738 (+   848us) log.cc:844] Serialized 52 byte log entry
0709 12:28:18.434959 (+198221us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:18.544446 (+109487us) log.cc:844] Serialized 14 byte log entry
0709 12:28:18.551204 (+  6758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:18.551290 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:18.755550 (+204260us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b1800128620. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db
0709 12:28:19.518963 (+763413us) spinlock_profiling.cc:243] Waited 81.4 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:19.704649 (+185686us) raft_consensus.cc:1685] finished
0709 12:28:19.704684 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:19.704895 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68970,"prepare.queue_time_us":108889,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":67,"raft.queue_time_us":3835,"raft.run_cpu_time_us":523,"raft.run_wall_time_us":524,"spinlock_wait_cycles":255257600,"thread_start_us":195734,"threads_started":1,"wal-append.queue_time_us":247103}
I20260709 12:28:19.803046  6450 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:19.805104  7320 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:19.799196 14580 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:19.809954 14580 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:19.811038  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:19.814786 14593 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:19.815507 14593 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:19.815953 14593 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:19.831259  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:19.832499 14398 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:19.834061 14398 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:19.834648 14549 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.775020 14569 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:19.837644 14569 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:19.838938 14569 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.863467  6452 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:19.705778 14123 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.864990  6348 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:19.798767 14313 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:19.865439  6455 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:19.799367 14477 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:19.843189 14528 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:28:19.733143 14573 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.788686 14495 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.898284  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:19.770079 14444 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.894827  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:19.861966 14174 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:19.924381  6455 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
W20260709 12:28:19.930684  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 190120us
I20260709 12:28:19.902923  6456 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:28:19.895991 14400 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:28:19.917719 14509 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:28:19.939420  6381 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:28:19.939972  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:19.809242 14333 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.122s
I20260709 12:28:19.903688 14452 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:19.926065 14313 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:19.951508  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:19.936614 14398 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:19.959628  6385 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
W20260709 12:28:19.962934  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 309747us
I20260709 12:28:19.964890  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:19.965795 14344 raft_consensus.cc:2764] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election decision vote started in defunct term 20: won
I20260709 12:28:19.967433 14349 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:19.968050 14349 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:19.968302  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:19.969110  6450 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:19.969331 14268 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:19.969997 14268 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:19.944016 14136 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:19.967458  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:19.950634 14400 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:19.972361 14414 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:19.999931  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:19.980028 14398 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:19.979538 14349 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.030905 14313 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.032243 14268 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:19.972879 14452 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:20.067044  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 84841us
I20260709 12:28:20.074251 14549 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.076262 14398 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.065797  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.053517 14566 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:20.057791 14314 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.085023 14512 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:20.099350  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132059us
I20260709 12:28:20.081879 14566 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:20.116487  6351 consensus_peers.cc:597] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1ee2be3ef27a4ec7bd5946b81b8ee8c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:20.118711  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:20.119907  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59813.3us
I20260709 12:28:20.120577  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:20.121390  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:20.128921  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.107995 14566 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:20.133280  6451 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:20.110216 14322 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:20.137900 13772 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.649s	user 0.013s	sys 0.000s
I20260709 12:28:20.132192 14512 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:20.152614  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:20.142084 14595 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:28:20.139382 14123 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.108790 14400 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.142457 14344 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:28:20.142275 14333 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:20.149559 14587 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:20.158466 14512 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:20.163697 13772 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:20.151820 14580 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.174764  7288 leader_election.cc:400] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21.
W20260709 12:28:20.174623  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56026.4us
I20260709 12:28:20.165608 14344 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:20.184244  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69686.6us
I20260709 12:28:20.172442 14512 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:20.191998  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:20.166654 14577 consensus_queue.cc:1059] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
I20260709 12:28:20.205370  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:20.205681  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:20.208689  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:20.190807 14594 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:28:20.184970 14344 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:20.185669 14477 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:20.192655 14592 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:28:20.215263  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.217252  6385 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:20.219233  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.219993  6377 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:28:20.220521  6377 pending_rounds.cc:85] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 15
I20260709 12:28:20.220839  6377 pending_rounds.cc:107] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:28:20.221120  6377 raft_consensus.cc:2889] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:20.197350 14452 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.204736 14549 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:20.231001  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60100.2us
I20260709 12:28:20.231148 14549 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.234855  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:20.213711 14186 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:20.235734 14452 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:20.236624  6303 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:20.237093 14322 raft_consensus.cc:2764] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:28:20.237418 14596 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:20.238840  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.242748 14596 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:20.244547  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:20.236810 14452 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:20.247232 14600 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:28:20.209404 14565 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:20.248783 14565 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:20.249222 14452 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.252141 14596 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:20.252667 14596 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:20.253113 14596 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:20.259459  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.260365 14577 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:28:20.264667 14561 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.911s	user 0.004s	sys 0.003s
I20260709 12:28:20.272562 14322 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:20.274577  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.276089 14444 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.283324  6383 raft_consensus.cc:2410] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 12:28:20.284498 14601 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:28:20.285022  6382 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:20.187481 14573 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.236706 14186 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:20.249301 14565 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:20.299331 14214 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.299603  6380 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:20.313331  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.314661  6377 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:28:20.234423 14549 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.324038  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.326285  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:20.327069  6382 raft_consensus.cc:2376] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23. Re-sending same reply.
I20260709 12:28:20.330658  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.319249 14322 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:20.340312  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.343950 14136 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:20.344659 14136 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:20.351841  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:20.353572  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79137.9us
W20260709 12:28:20.372010  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.379659 14186 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.389042 14565 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.368290  6380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:20.397230  6380 raft_consensus.cc:606] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.399930  6383 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:28:20.400621  6383 pending_rounds.cc:85] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 13
I20260709 12:28:20.400974  6383 pending_rounds.cc:107] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:28:20.401337  6383 raft_consensus.cc:2889] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:20.399288 14593 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.582s	user 0.016s	sys 0.000s
I20260709 12:28:20.429558  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:20.432337  6305 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:20.439419  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.425318 14580 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.441324 14593 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.448734 14344 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.453197  6458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:20.453883  6458 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.454466 14566 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.455502  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:20.403043 14577 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.485208  6309 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:28:20.455834 14313 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.021s
W20260709 12:28:20.476431 14471 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.699s	user 0.009s	sys 0.005s
I20260709 12:28:20.492846 14322 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:20.517300 14597 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.518299 14597 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.521384 14597 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:20.521729 14344 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.521950 14597 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:20.517309 14608 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:28:20.522708  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1102) took 1121 ms. Trace:
I20260709 12:28:20.523106  7332 rpcz_store.cc:276] 0709 12:28:19.400958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.417200 (+ 16242us) service_pool.cc:224] Handling call
0709 12:28:20.522444 (+1105244us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54005c1a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:20.522635 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":837645184}
I20260709 12:28:20.523783 14597 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.526109 14597 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:20.527477 14593 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.528184 14593 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:20.528687 14593 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.535071 14322 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:20.536077 14322 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.510329  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:20.539242  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:20.543424 14495 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.476882 14596 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.484380 14577 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.547582  6380 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:20.549023  6380 raft_consensus.cc:606] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:28:20.553655  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:20.554343  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:20.558724  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:20.563297  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 107892us
I20260709 12:28:20.548688  6307 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:20.540513 14565 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.521953 14454 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.584618  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:20.548696 14435 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:20.505105 14471 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.607455  6453 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:20.608392  6453 raft_consensus.cc:606] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:20.609272 14596 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.598335 14593 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.587024 14444 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:20.611649 14577 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.612169 14444 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:20.612753 14444 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:20.584550 14453 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.556027 14512 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.627002  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3258) took 1037 ms. Trace:
I20260709 12:28:20.627446  6381 rpcz_store.cc:276] 0709 12:28:19.587732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.587855 (+   123us) service_pool.cc:224] Handling call
0709 12:28:19.636174 (+ 48319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:19.941399 (+305225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:20.623962 (+682563us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b5400038c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:20.624861 (+   899us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684731776}
I20260709 12:28:20.629041  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:20.630064  7317 raft_consensus.cc:2368] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:20.636510  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:20.588658 14495 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.640615  7320 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:20.781400  7332 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:28:20.784083  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b776445e68c14533bcde1f784caa703a"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:20.784875  6382 raft_consensus.cc:606] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:20.785223  6382 raft_consensus.cc:614] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:20.793192 14610 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:28:20.795389 14495 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.795500  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:20.803411  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:20.631871 14358 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.166s
I20260709 12:28:20.806874  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:20.808374  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:20.807600 14566 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.807581  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.810189  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:20.811985  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.812376  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.813202  6458 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:20.814389  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.633136 14453 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.816219  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.821058  6456 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:20.817780  6307 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:20.814975  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:20.825270  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:20.827495  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.827754  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.829375  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:20.824188 14565 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.833678  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.831660  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:20.835657  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 274726us
I20260709 12:28:20.836580  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:20.835740  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:20.842655  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:20.805366 14136 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.844913  6453 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
I20260709 12:28:20.853732  6385 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:20.861600  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 277086us
I20260709 12:28:20.860769  6457 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:28:20.865773  6457 pending_rounds.cc:85] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 10
I20260709 12:28:20.866072  6457 pending_rounds.cc:107] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:28:20.866374  6457 raft_consensus.cc:2889] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:20.870239  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.871649  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:20.873024  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:20.873660  6308 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:20.844058 14047 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:20.824952 14314 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:28:20.877370  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:20.880396  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:20.886080  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.831436 14186 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:28:20.838963 14580 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.037s
W20260709 12:28:20.932767  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55475.2us
I20260709 12:28:20.934340  6454 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:20.942970  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.943782  6452 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 27 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
W20260709 12:28:20.940853  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60548.2us
I20260709 12:28:20.893410 14047 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:20.950678  6457 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:28:20.951093 14626 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:28:20.952716 14047 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:20.954339 14435 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:20.959869  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.962750  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.963510  6456 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:20.964542  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:20.925410 14512 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.965229  6456 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:20.965579  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.887476 14214 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:28:20.969341  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.938326 14400 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:20.875541 14565 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:20.977099  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.979318 14136 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:20.979972 14620 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:28:20.980294 14565 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:20.980154  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:20.942132 14573 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:21.030243 14573 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:21.031329 14573 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.032648 14573 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:21.004963  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:21.036777  6452 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.022259  6383 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:21.023406  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.038986  6450 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:21.024497 14627 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:21.025846 14564 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:21.040799  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:21.027443  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.009479  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:21.043977  6453 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:21.044093  6376 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:21.049335  6308 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:21.050457  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:20.979239 14268 raft_consensus.cc:3055] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:21.054610 14254 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:21.056753  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.057926  6453 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:21.059391  6453 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:21.061036  6453 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:20.984047 14453 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:21.076525  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.077668  6454 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:21.086258  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:21.042960 14564 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:21.064268 14358 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:21.062942 14268 raft_consensus.cc:740] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:21.120142  6456 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:21.125499  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:21.131311  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:21.132359  6385 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:21.138319  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:21.141390  6456 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:28:21.062439 14444 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:21.151407  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:21.152659  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:21.148913  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:21.110852 14358 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:21.152607  7332 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:21.131417 14512 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.157671  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:21.158762  7320 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:28:21.159550  6301 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.008s	sys 0.022s
W20260709 12:28:21.159951  6301 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.008s	sys 0.022s
I20260709 12:28:21.205034  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:21.207656  6452 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:21.211377  7316 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:28:21.211560  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.213935  7335 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:21.147006 14454 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:21.155305 14314 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.156184 14358 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:21.237576  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:21.240096  6383 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:21.151533 13772 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 12:28:21.248725  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:21.255589  6385 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:28:21.256809  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:21.257547  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.258236  6456 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:21.258380  6452 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:21.267266  6456 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:21.272622  6383 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:21.271569  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:21.279232  6456 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:21.287740  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:21.271173 14453 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.215638 14512 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.304244  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:21.245630 14314 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:21.308694 14414 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:28:21.327394  7317 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.005s	sys 0.013s
W20260709 12:28:21.328019  7317 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.005s	sys 0.013s
W20260709 12:28:21.325449 14322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.007s	sys 0.011s
I20260709 12:28:21.308008 14453 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:21.351480  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 393431us
I20260709 12:28:21.316401 14186 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:28:21.326840 14414 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:21.342259 14322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.012s
W20260709 12:28:21.367439  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 400582us
I20260709 12:28:20.963183  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:21.366287 14414 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:21.442466 14632 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:28:21.443708 14631 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
W20260709 12:28:21.444890 14630 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:28:21.446604 14634 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:28:21.418617 14444 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:21.377115 14573 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.059s
I20260709 12:28:21.378407 14636 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:28:21.430917 14400 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:21.486270  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 193619us
I20260709 12:28:21.487730  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:21.513865  6376 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
W20260709 12:28:21.516939  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 239298us
W20260709 12:28:21.558176  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 346992us
I20260709 12:28:21.484298 14454 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:28:21.650789  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 163207us
I20260709 12:28:21.530244 14566 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.169s
I20260709 12:28:21.671089 14512 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:21.699352 14047 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:21.717355  6453 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.010s	sys 0.022s
I20260709 12:28:21.719895  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3548) took 1447 ms. Trace:
I20260709 12:28:21.671540 13772 consensus_queue.cc:1243] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:28:21.717900  6453 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.010s	sys 0.022s
W20260709 12:28:21.724462  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 359928us
I20260709 12:28:21.720319  6379 rpcz_store.cc:276] 0709 12:28:20.272265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.293067 (+ 20802us) service_pool.cc:224] Handling call
0709 12:28:20.953522 (+660455us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b58001a3a98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:28:20.953766 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:20.954079 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:20.954444 (+   365us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.719513 (+765069us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.719574 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:21.719629 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.719701 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.719834 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10078,"prepare.queue_time_us":9849,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":379986,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":3759,"spinlock_wait_cycles":5471104,"thread_start_us":754301,"threads_started":2,"wal-append.queue_time_us":574669}
I20260709 12:28:21.709757 14314 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:21.730962  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 470353us
I20260709 12:28:21.716560 14512 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:21.721256 14453 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:21.747777  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:21.731905 13772 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:28:21.755129  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:21.720263 14047 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:21.759653  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.717553 14400 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:21.748766 14646 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:21.793951 14477 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:21.757414 14512 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:21.733327 14454 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.752439 14642 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:28:21.809845 14646 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.794675 14659 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:21.861756  6456 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.016s	sys 0.032s
W20260709 12:28:21.870966  6456 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.017s	sys 0.032s
W20260709 12:28:21.871734  6457 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.007s	sys 0.051s
I20260709 12:28:21.879295 14268 consensus_queue.cc:260] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.875365 14659 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.892015 14047 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:21.871039 14646 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:21.894107 14664 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:21.895102 14644 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:28:21.894888 14664 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:21.897157 14664 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:21.897732 14666 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:21.898312 14666 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.900761 14666 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:21.902367 14471 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:21.903586 14650 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:28:21.904621 14659 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:21.858886 14477 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.907738 14477 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:21.908407 14639 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:28:21.910113 14652 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:21.910797 14652 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.913043 14652 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:21.907758  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3467) took 1450 ms. Trace:
W20260709 12:28:21.816882  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 145645us
W20260709 12:28:21.867286  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112892us
I20260709 12:28:21.813294 14656 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.815526 14512 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:21.925019 14512 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.925127 14477 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.925846 14477 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.918324  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:21.922714 14471 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:21.923489 14596 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:21.813434 14186 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:21.927740  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3462) took 1524 ms. Trace:
I20260709 12:28:21.954432  6301 rpcz_store.cc:276] 0709 12:28:20.403094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.403297 (+   203us) service_pool.cc:224] Handling call
0709 12:28:20.430522 (+ 27225us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:20.430728 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:20.584348 (+153620us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.927274 (+1342926us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.927348 (+    74us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:21.927407 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.927514 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.927667 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":807210,"prepare.queue_time_us":106014,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"thread_start_us":676228,"threads_started":2,"wal-append.queue_time_us":740921}
I20260709 12:28:21.954164  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:21.931738  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:21.932876  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3809) took 1124 ms. Trace:
I20260709 12:28:21.929965  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3817) took 1087 ms. Trace:
I20260709 12:28:21.951519  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3570) took 1135 ms. Trace:
I20260709 12:28:21.963788  6457 rpcz_store.cc:276] 0709 12:28:20.842741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.951703 (+108962us) service_pool.cc:224] Handling call
0709 12:28:20.952042 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:20.952260 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:21.342092 (+389832us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.929501 (+587409us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.929576 (+    75us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:21.929641 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.929743 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.929896 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4983,"prepare.queue_time_us":426218,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":57,"thread_start_us":896799,"threads_started":2}
W20260709 12:28:21.967549  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49324.8us
I20260709 12:28:21.968822  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.968966  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:21.971920  6378 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.007s	sys 0.011s
I20260709 12:28:21.967805  6451 rpcz_store.cc:276] 0709 12:28:20.815956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.816089 (+   133us) service_pool.cc:224] Handling call
0709 12:28:20.816329 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:20.816510 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:21.237424 (+420914us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.951175 (+713751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.951225 (+    50us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:21.951269 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.951333 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.951460 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":549375,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":209811,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":1104152,"threads_started":3,"wal-append.queue_time_us":632137}
W20260709 12:28:21.973677  6378 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.007s	sys 0.011s
I20260709 12:28:21.962015  6455 rpcz_store.cc:276] 0709 12:28:20.810375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.810515 (+   140us) service_pool.cc:224] Handling call
0709 12:28:20.810794 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:20.811024 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:21.103063 (+292039us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.458966 (+355903us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.932497 (+473531us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.932556 (+    59us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:21.932607 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.932676 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.932813 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":544678,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":3795,"raft.queue_time_us":928700,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":158,"thread_start_us":1083247,"threads_started":3,"wal-append.queue_time_us":618183}
I20260709 12:28:21.982656  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:21.983497  6455 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:21.950124 14452 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:21.981253  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:21.989511  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3826) took 1035 ms. Trace:
I20260709 12:28:21.949649 14652 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:21.997309  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 623481us
W20260709 12:28:21.997638  7287 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.016s	sys 0.032s
I20260709 12:28:21.992175  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:21.992863  6458 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:22.004496  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:21.993386  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.006116  6454 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:21.953522 14333 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:21.966425  6302 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Committing config change with OpId 22.14: config changed from index -1 to 14, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:28:22.004408  7287 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.017s	sys 0.032s
I20260709 12:28:21.990124  6453 rpcz_store.cc:276] 0709 12:28:20.954362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.083508 (+129146us) service_pool.cc:224] Handling call
0709 12:28:21.109995 (+ 26487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:21.142350 (+ 32355us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:21.724648 (+582298us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.988804 (+264156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.988897 (+    93us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:21.989013 (+   116us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.989147 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.989400 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80502,"prepare.queue_time_us":286872,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":1976,"thread_start_us":799295,"threads_started":2}
W20260709 12:28:22.006839  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 259109us
I20260709 12:28:21.953815 14186 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:22.009879  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 751828us
I20260709 12:28:22.011068  6459 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:21.979998 14657 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:22.024183 14646 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:21.987176 14566 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:22.004297 14047 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:22.012637 14333 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:21.953748 14471 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:21.977144 14658 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:21.978302 14656 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.016553 14652 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.016685 14186 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:22.037056  6451 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:22.037712  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:21.957249 14596 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.095818 14596 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.008345 14662 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.096900 14646 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.050329  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:22.057031  6455 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:28:22.097908  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 47689.9us
W20260709 12:28:22.017030 14358 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.002s
I20260709 12:28:22.083585  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1110) took 1442 ms. Trace:
I20260709 12:28:22.019573 14657 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.095742  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:21.986740 14642 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:28:22.096783 14662 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:22.113976  6274 connection.cc:256] RPC call timeout handler was delayed by -1.1466597950002324s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1967s for a response
I20260709 12:28:22.073343 14333 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:22.162223 14657 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:22.149536 14655 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:28:22.150272 14358 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.151347  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.151607  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.039074 14495 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.113s
I20260709 12:28:22.082906 14641 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:21.994058 14668 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.113s
I20260709 12:28:22.187403  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.188409  6376 raft_consensus.cc:2410] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 22 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:28:22.164897 14671 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.152550 14665 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:22.189559 14665 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:22.180904 14587 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.190330 14587 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.190659 14665 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.192657 14587 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.166725 14564 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.195420 14322 raft_consensus.cc:993] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:28:22.195933 14322 raft_consensus.cc:1081] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:28:22.171911 14471 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.170722 14596 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:28:22.198094  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:22.098601 14658 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:22.198559 14577 consensus_queue.cc:1175] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 23 status { last_received { term: 22 index: 15 } last_committed_idx: 15 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:22.088891 14579 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.098264 14677 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.080642 14649 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.157079  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.158161  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.212337  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:22.158635  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 121031us
I20260709 12:28:22.212576 14649 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.212898 14579 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.215003 14649 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:22.161192  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100634us
I20260709 12:28:22.215329 14495 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:22.123374 14414 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.004s
I20260709 12:28:22.195884 14564 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:22.189049 14671 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:22.155923 14652 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.154049 14656 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.213356  6383 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:22.214795 14569 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:22.215714 14495 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:22.165884 14662 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.149529 14674 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:22.155866  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.212823  6451 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:22.212261  6385 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
W20260709 12:28:22.231473  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 863842us
W20260709 12:28:22.204200 14667 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.114s	user 0.001s	sys 0.009s
I20260709 12:28:22.162189 14453 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:22.232997  7287 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.591s (SENT) [suppressed 11 similar messages]
I20260709 12:28:22.216121 14414 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.258080  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.258579  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.244141  6454 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:22.259127  6308 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:22.162328 14663 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:22.163831 14659 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:28:22.246513  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 292524us
I20260709 12:28:22.248054  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:22.263521  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:22.252486  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.266077  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.239014 14254 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.276711  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.157939 14673 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.281477  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.238076 13772 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:22.282467  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.283010  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.287369  6304 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:22.288013  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.288353  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.288578  7316 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:22.288041  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.289950  7335 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:22.211601 14454 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:28:22.288666  6384 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:28:22.295689  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.295971  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.296537  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.210929 14512 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.304414  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:22.288117 14254 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.302822 14673 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.289104 14628 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.296460 14645 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.303135  6377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:28:22.273171 14659 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.306557 14682 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:22.319211 14659 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.319435 14682 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:22.320039 14682 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:22.272428 14690 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.322171 14690 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.323172 14565 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:22.324363  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.324615 14690 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.280269 14692 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.325292 14671 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.325559 14692 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.265424 14453 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.318783  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.329831  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.324396  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.311314  6454 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:22.333427  6308 raft_consensus.cc:1240] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 19. Current term is 20. Ops: []
W20260709 12:28:22.311005  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 69453.2us
I20260709 12:28:22.282533 14676 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.165s
I20260709 12:28:22.234692 14677 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.317699  6377 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:28:22.343114  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:22.477306  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.450974  7318 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:28:22.369585  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.454725  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:22.441412  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 177973us
I20260709 12:28:22.415278  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.501482  6458 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:22.441958  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3271) took 1186 ms. Trace:
I20260709 12:28:22.422613  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.505926  6377 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:22.508039  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 165025us
I20260709 12:28:22.290444 14652 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.501467  7320 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:22.503773  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:28:22.531544  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.538539  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.540205  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.509316  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:22.445061 14651 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.653s	user 0.000s	sys 0.004s
W20260709 12:28:22.443483 14686 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 12:28:22.549878  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.550324  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.314126 14254 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:22.327934 14358 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.505044  6378 rpcz_store.cc:276] 0709 12:28:21.255210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.259403 (+  4193us) service_pool.cc:224] Handling call
0709 12:28:21.283829 (+ 24426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:21.984939 (+701110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.984987 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:22.017259 (+ 32272us) log.cc:844] Serialized 26 byte log entry
0709 12:28:22.243002 (+225743us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:22.243042 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.243111 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:22.441686 (+198575us) raft_consensus.cc:1685] finished
0709 12:28:22.441710 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:22.441885 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4393,"raft.queue_time_us":336599,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":58652,"thread_start_us":760861,"threads_started":2,"wal-append.queue_time_us":352073}
I20260709 12:28:22.554517  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.290910 14680 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.314s
I20260709 12:28:22.595496  6301 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 12:28:22.597839  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.468839 14192 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.607795  6309 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:22.602357  6453 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:28:22.307976 13772 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:22.604151  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.439811 14512 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.374217 14565 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:22.343032 14214 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.317173 14673 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.610144  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.611443  6305 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:22.633852  6384 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:22.634008  6454 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:22.612429 14322 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:21.917068  6300 rpcz_store.cc:276] 0709 12:28:20.457811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.457964 (+   153us) service_pool.cc:224] Handling call
0709 12:28:20.474338 (+ 16374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:20.474663 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:20.588398 (+113735us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.085516 (+497118us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.906615 (+821099us) spinlock_profiling.cc:243] Waited 16.9 ms on lock 0x7b18004e8200. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:21.906997 (+   382us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.907174 (+   177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:21.907231 (+    57us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:21.907293 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:21.907400 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:21.907664 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":827071,"prepare.queue_time_us":62757,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":834183,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":21191,"spinlock_wait_cycles":66235392,"thread_start_us":547079,"threads_started":2,"wal-append.queue_time_us":712058}
I20260709 12:28:22.622542  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.617321  6458 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:22.327957 14692 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.374368 14477 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:22.627195  6310 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:22.641713  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.533218 14662 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:22.627750  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.554448 14671 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.622340  7329 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:22.657707  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:22.658280  7329 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:22.637373  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:22.659687  6310 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:22.660240  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.598181 14564 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:22.357126 14333 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.594977 14664 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.316911 14628 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.669709  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:22.635206  6351 connection.cc:256] RPC call timeout handler was delayed by -1.8359874540001329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 12:28:22.661943  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:22.317341 14645 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.664161  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.667415  6304 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:22.318753 14495 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.598716 14690 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
I20260709 12:28:22.656631  6308 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:22.671540  6385 raft_consensus.cc:2376] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23. Re-sending same reply.
I20260709 12:28:22.663494 14564 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.675387  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.675742 14677 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.677937  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.704200  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.678750  6310 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:22.706758  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.683153 14646 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:22.687765  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:22.635505 14565 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.694741 14662 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.701965  6383 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:22.688683  6377 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.704505  6300 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:22.374766 14663 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.275s
I20260709 12:28:22.709079  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:22.561272 14659 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.654762 14322 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.705452 14678 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.705626 14683 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:22.679335 14679 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.093s
I20260709 12:28:22.714725  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.715529  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.677325 14664 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.567617 14652 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.636786 14549 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:28:22.719244  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.721431  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:22.668697 14671 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.726536  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.630230 14192 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.733173 14414 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.707399 14564 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:22.645609 14214 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:22.728802 14625 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.363s	user 0.064s	sys 0.074s
I20260709 12:28:22.707516  6310 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:22.710655  6382 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:22.738696  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:22.705860 14672 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.621495 14186 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.741490  6351 leader_election.cc:400] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:22.730826 14677 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:22.744488  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.744992  6377 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:22.722105 14565 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:22.747629  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.705713 14643 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:22.746556  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:22.756039  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.758143  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.789523  6304 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:28:22.745968 14047 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:22.739363 14646 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:22.760898  7331 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 30: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:22.761475  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:22.766208 14625 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.403s	user 0.064s	sys 0.074s
I20260709 12:28:22.759724 14682 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:22.775030 14705 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:28:22.777766  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.780143  6456 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:28:22.781244  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3582) took 1517 ms. Trace:
I20260709 12:28:22.813519  6300 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:22.103188  7317 rpcz_store.cc:276] 0709 12:28:20.641359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.641509 (+   150us) service_pool.cc:224] Handling call
0709 12:28:20.779591 (+138082us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:21.328501 (+548910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:21.328547 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:21.342779 (+ 14232us) log.cc:844] Serialized 52 byte log entry
0709 12:28:21.761202 (+418423us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:22.040712 (+279510us) log.cc:844] Serialized 14 byte log entry
0709 12:28:22.071780 (+ 31068us) log.cc:844] Serialized 14 byte log entry
0709 12:28:22.083044 (+ 11264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.083145 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:22.083210 (+    65us) raft_consensus.cc:1685] finished
0709 12:28:22.083231 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:22.083510 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76474,"raft.queue_time_us":87388,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":68675,"spinlock_wait_cycles":27086464,"thread_start_us":1082147,"threads_started":3,"wal-append.queue_time_us":506918}
I20260709 12:28:22.758222 14512 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:22.821393 14645 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.801136  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:22.823413 14659 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.824223  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:22.824432 14659 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:22.825203  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:22.825366 14659 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:22.825924 14659 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.743315 13772 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.826673 14577 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:22.828069  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:22.811076  6308 raft_consensus.cc:2410] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 23 because replica has last-logged OpId of term: 22 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:28:22.757994 14641 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:22.829062 14641 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:22.789373 14672 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.830132 14641 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.830772 14653 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:22.831507 14641 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:22.813441  6452 rpcz_store.cc:276] 0709 12:28:21.263650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:21.263820 (+   170us) service_pool.cc:224] Handling call
0709 12:28:21.264229 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:21.709482 (+445253us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:21.738590 (+ 29108us) log.cc:844] Serialized 14 byte log entry
0709 12:28:22.069411 (+330821us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:22.157082 (+ 87671us) log.cc:844] Serialized 26 byte log entry
0709 12:28:22.157282 (+   200us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:22.157318 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.780802 (+623484us) spinlock_profiling.cc:243] Waited 64.4 ms on lock 0x7b54000eb854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:22.780862 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:22.780925 (+    63us) raft_consensus.cc:1685] finished
0709 12:28:22.781032 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:22.781171 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157109,"prepare.queue_time_us":15684,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":39,"raft.queue_time_us":557794,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":29790,"spinlock_wait_cycles":148087040,"thread_start_us":626000,"threads_started":2,"wal-append.queue_time_us":341279}
I20260709 12:28:22.834510 14192 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:22.835484  6452 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:22.836174 14652 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.837973  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:22.758953 14453 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.786429 14565 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.799647 14668 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.017s
I20260709 12:28:22.816424  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.817637 14664 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:22.818996 14682 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.852875  6378 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
W20260709 12:28:22.819175 14700 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:28:22.820852  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:22.821534 14512 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:22.815147  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:22.844174  6303 consensus_meta.cc:287] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.543s	user 0.009s	sys 0.000s
I20260709 12:28:22.863338  6303 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:22.742286 14678 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.864324  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.845777  6310 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:22.848376  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:22.737982 14662 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.855013  6301 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:22.788158 14643 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:22.826359 14659 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:22.796766 14047 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:22.861171  6304 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:22.744572 14683 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:22.866266  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:22.867962  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:22.868336  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 181069us
I20260709 12:28:22.869724  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.874935  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.876433  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.859203 14192 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.006s
W20260709 12:28:22.877131  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62048.5us
I20260709 12:28:22.852288 14652 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:22.899621 14214 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.899856 14652 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.900255 14709 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:22.900851 14709 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:22.901494 14629 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.527s	user 0.002s	sys 0.006s
I20260709 12:28:22.877383  7317 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:22.877444  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.876641  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.891250  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.892637 14268 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:22.894863  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.905805  6310 raft_consensus.cc:2368] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21.
I20260709 12:28:22.895947 14628 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:22.898268 14708 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:28:22.906606  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.911667  6304 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:22.915359  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.915766  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.917682  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:22.918948  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:22.919929  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.921178  6379 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:22.903638 14453 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:22.922752  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.922899 14453 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.923372 14453 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:22.923828  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.924425  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.925034  6377 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:22.927595  6301 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:22.928438  6455 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:22.884595 14047 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.930776  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.931416  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:22.932331  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.933547  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.934095  6303 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:22.934798  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.935375  6459 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:22.936739  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.937469  6455 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:22.938627  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.939262  6305 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:22.940016  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.940069 14678 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.940675  6305 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:22.941059 14658 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:22.925040  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.941519  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.929602  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:22.938367  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:22.946254  6455 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:22.946689  6455 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:22.944563  6305 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:22.926329  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.947852  6307 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:22.954329  7329 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:22.884070 14676 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:22.956887  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.952704  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:22.900290 14652 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:22.936645 14322 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:22.964746  6425 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.810s (SENT)
I20260709 12:28:22.963750  6310 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:22.970630  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.973219  6303 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:22.974630  6303 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:28:22.977367  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.942912 14708 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:22.979753  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:22.979020  6305 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:22.977195  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:22.971735 14629 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:28:22.992576  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.994272  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:22.995048  6458 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:22.988536  6304 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:22.988478  6455 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.028090  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:23.028968  6455 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:22.978464 14452 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:23.040684  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:23.043787  6305 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:22.984408 14708 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:23.043841  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:22.999414 14658 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:22.909277 14646 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:22.991683 14715 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:22.996979 14645 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:28:23.046869  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:23.053906 14713 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:28:22.955044 14643 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.054647  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.060304 14579 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:23.060982 14677 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:23.062441 14702 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:22.996662 14690 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:28:22.982641 14214 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.080s
I20260709 12:28:23.062922 14714 ts_tablet_manager.cc:933] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.042299 14452 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.063704  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:23.064891  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:23.122370  6310 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:23.027675 14358 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.066258  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.068425  6383 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:23.068688  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:23.070597 14564 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.073105  6455 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:23.073549  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:23.104312 14709 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.081063  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:22.999166 14322 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:28:23.044306 14692 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:23.108996  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54432.5us
I20260709 12:28:23.062840 13772 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.064630 14186 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:23.082221 14657 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.908s	user 0.009s	sys 0.003s
I20260709 12:28:23.155009 14657 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:23.073940 14645 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.156179 14645 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:23.154418 14692 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:23.161032 14692 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:23.124312 14452 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:23.162846 14657 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.164245  7316 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:23.110785 14635 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.113039 14702 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.129652  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.112329 14628 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:23.079818  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3463) took 2650 ms. Trace:
I20260709 12:28:23.228437  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3593) took 1152 ms. Trace:
I20260709 12:28:23.001602 14512 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.114286 14643 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:23.082405 14715 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:23.134908  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:23.044296 14566 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:28:23.166883  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.167543  6455 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:23.226935  6459 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:23.131862  6378 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:23.248188  6451 rpcz_store.cc:276] 0709 12:28:22.075987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.669510 (+593523us) service_pool.cc:224] Handling call
0709 12:28:22.670169 (+   659us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:22.670481 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.670539 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:22.682842 (+ 12303us) log.cc:844] Serialized 26 byte log entry
0709 12:28:22.762247 (+ 79405us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:22.762284 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.762354 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.228117 (+465763us) raft_consensus.cc:1685] finished
0709 12:28:23.228148 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.228353 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":39350,"threads_started":1,"wal-append.queue_time_us":195364}
I20260709 12:28:23.239938  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.228900  6302 rpcz_store.cc:276] 0709 12:28:20.417689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:20.417868 (+   179us) service_pool.cc:224] Handling call
0709 12:28:20.434088 (+ 16220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:20.434475 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:20.630810 (+196335us) log.cc:844] Serialized 14 byte log entry
0709 12:28:21.915047 (+1284237us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:21.966086 (+ 51039us) log.cc:844] Serialized 485 byte log entry
0709 12:28:21.966280 (+   194us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:22.278102 (+311822us) log.cc:844] Serialized 14 byte log entry
0709 12:28:22.303186 (+ 25084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.303269 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:22.900630 (+597361us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:23.067825 (+167195us) raft_consensus.cc:1685] finished
0709 12:28:23.067864 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.067997 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":888104,"prepare.queue_time_us":102437,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":906527,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":30515,"spinlock_wait_cycles":66898304,"thread_start_us":463806,"threads_started":2,"wal-append.queue_time_us":680244}
I20260709 12:28:23.214883 14658 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:28:23.243889 14720 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:28:23.132630 14358 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.265396  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.233029 14702 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:23.269428  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.280869  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.281776  6383 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:23.285210  6455 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:23.263964 14690 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:23.290102  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.223384 13772 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.316653  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.318178  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:23.321584  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.226435 14635 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.267653 14658 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:23.324666  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.256304 14047 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:23.350878 14727 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.082s	user 0.010s	sys 0.000s
I20260709 12:28:23.351779 14358 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:23.353497 14730 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:28:23.351014  7314 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 30. Current term is 31. Ops: []
W20260709 12:28:23.359246 14718 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:28:23.360642  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3591) took 1316 ms. Trace:
I20260709 12:28:23.361032  6454 rpcz_store.cc:276] 0709 12:28:22.044685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.634657 (+589972us) service_pool.cc:224] Handling call
0709 12:28:22.661138 (+ 26481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:22.733855 (+ 72717us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.733892 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:22.734702 (+   810us) log.cc:844] Serialized 26 byte log entry
0709 12:28:22.958428 (+223726us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:22.958477 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.958585 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.360404 (+401819us) raft_consensus.cc:1685] finished
0709 12:28:23.360430 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.360573 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35901,"raft.queue_time_us":127447,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":35346,"thread_start_us":206898,"threads_started":2,"wal-append.queue_time_us":282711}
I20260709 12:28:23.367837 14214 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.368461 14214 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.370560 14214 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.375962 14715 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:23.376418 14707 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.559s	user 0.003s	sys 0.000s
I20260709 12:28:23.381126  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:23.383991  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1212) took 1293 ms. Trace:
I20260709 12:28:23.384400  6382 rpcz_store.cc:276] 0709 12:28:22.115272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.151374 (+ 36102us) service_pool.cc:224] Handling call
0709 12:28:23.383813 (+1232439us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b5400109fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:23.383932 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1215974144}
I20260709 12:28:23.385655  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3278) took 1094 ms. Trace:
I20260709 12:28:23.389421  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:23.394887  6379 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:23.386035  6380 rpcz_store.cc:276] 0709 12:28:22.291198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.291350 (+   152us) service_pool.cc:224] Handling call
0709 12:28:22.295899 (+  4549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:22.534668 (+238769us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.534703 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:22.554093 (+ 19390us) log.cc:844] Serialized 26 byte log entry
0709 12:28:22.682367 (+128274us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:22.682404 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.682502 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.375839 (+693337us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:23.385331 (+  9492us) raft_consensus.cc:1685] finished
0709 12:28:23.385356 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.385585 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10294,"raft.queue_time_us":27879,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"spinlock_wait_cycles":77830016,"thread_start_us":282106,"threads_started":2,"wal-append.queue_time_us":213661}
I20260709 12:28:23.395443  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:23.403651  6382 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:28:23.403728  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:23.404737  7318 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:23.405861  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.393493 14186 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:23.399632 14566 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:23.410933  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:23.423674 14659 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.536s	user 0.010s	sys 0.010s
I20260709 12:28:23.424110 14659 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:23.425292  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.402473 14683 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.421402 14692 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:23.431073  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 264245us
I20260709 12:28:23.432435  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.410734 14721 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:23.436991  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.412582 14186 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:23.438807  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:23.444686  6379 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:23.445585  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.408638 14652 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.429323 14692 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.448827 14579 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:23.452631 14579 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:23.452904 14662 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.453073 14579 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:23.446920 14214 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:23.430869 14566 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:23.463941 14659 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.445909 14186 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:23.444479 14564 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.447959 14721 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:23.416148 14658 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:23.458674 14692 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:23.465081 14453 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.000s
I20260709 12:28:23.488298  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.488822  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.448853 14708 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:23.487246 14628 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:23.493877 14453 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.494032 14452 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:23.482537 14679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.006s	sys 0.002s
I20260709 12:28:23.464921 14739 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:23.503196 14735 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:28:23.505751  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:23.509331  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:23.518007  7318 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:23.513398  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:23.513253  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:23.518793  6304 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:23.527483  7318 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:23.528189  6304 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:23.528429  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:23.529228  7318 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:23.529273  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:23.532871  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44102.3us
I20260709 12:28:23.533907  6272 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.507284  6383 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.8, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:23.530319 14739 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:23.547441  6300 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:23.547710 14739 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:23.548887  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:23.549695  6300 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 30: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:23.550462  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:23.551417 14452 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.551923 14676 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:28:23.553071 14733 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:28:23.554855  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:23.556084 14732 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:28:23.557255  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.557713 14565 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:23.546938 14747 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:28:23.562731  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:23.563962  6303 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
W20260709 12:28:23.564998 14697 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.078s	user 0.001s	sys 0.001s
I20260709 12:28:23.565521 14628 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:23.566808  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:23.567366  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.568461  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1414) took 1362 ms. Trace:
I20260709 12:28:23.568955  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:23.569461 14641 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:23.569939 14641 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:23.570820 14641 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.571897 14641 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:23.572667 14664 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.574030 14652 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.574272 14739 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.574777 14721 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:23.576314  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.576565 14677 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:23.576452 14566 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.578073  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:23.578888  6459 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:23.579911  6454 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 15. Current term is 16. Ops: []
I20260709 12:28:23.568842  6458 rpcz_store.cc:276] 0709 12:28:22.205757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.008549 (+802792us) service_pool.cc:224] Handling call
0709 12:28:23.041508 (+ 32959us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:23.041883 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:23.041915 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:23.051821 (+  9906us) log.cc:844] Serialized 26 byte log entry
0709 12:28:23.408880 (+357059us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:23.428781 (+ 19901us) log.cc:844] Serialized 14 byte log entry
0709 12:28:23.438530 (+  9749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.438638 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.566206 (+127568us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b18003ee4e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:23.568133 (+  1927us) raft_consensus.cc:1685] finished
0709 12:28:23.568239 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.568396 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16419,"prepare.queue_time_us":35070,"raft.queue_time_us":9908,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":22945,"spinlock_wait_cycles":14129280,"thread_start_us":274340,"threads_started":1,"wal-append.queue_time_us":353222}
I20260709 12:28:23.578238 14656 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:28:23.582130 14452 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:23.578725  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:23.583043 14452 raft_consensus.cc:2764] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:28:23.580850 14710 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.745s	user 0.000s	sys 0.001s
I20260709 12:28:23.583715 14452 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:28:23.584575 14452 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:23.585034 14452 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:23.585709  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 620983us
I20260709 12:28:23.586782 14047 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.576973 14739 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.588244  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:23.589443  6451 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:23.591547  7329 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
W20260709 12:28:23.517194 14679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.006s	sys 0.003s
I20260709 12:28:23.593466  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:23.594370  6452 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:23.513885 14658 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:23.599475  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.600299  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.602455  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.585471 14452 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:23.607606  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:23.605151  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.608819  7318 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:23.606897  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.612087  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:23.613389  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:23.615721  6425 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.009s (SENT)
W20260709 12:28:23.619110 14742 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:28:23.620476 14736 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:28:23.621371 14716 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:23.622556 14564 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:23.623155 14214 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:23.625007 14453 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:23.606112 14747 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:28:23.566452 14628 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:23.636526  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:23.582461 14656 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:23.659204  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:23.662583  7318 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:23.637743  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:23.684188  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 115344us
I20260709 12:28:23.617136 14677 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:28:23.686766  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49170.1us
I20260709 12:28:23.589821 14664 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.690449  7318 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
W20260709 12:28:23.691741  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76039.3us
I20260709 12:28:23.711376  6272 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.607563 14739 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:28:23.714905  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:23.611037 14414 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:28:23.636705 14593 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:23.715680  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:23.731006  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.666702 14641 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:23.656658 14676 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 12:28:23.634611 14643 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.099s
I20260709 12:28:23.616747 14565 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:23.595580 14047 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 12:28:23.737419 14647 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:23.743387  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1452) took 1088 ms. Trace:
I20260709 12:28:23.743841 14664 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.691332 14635 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:23.684058 14628 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.733206 14593 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.745769 14660 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:23.747313 14314 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:28:23.748836 14694 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 1.266s	user 0.003s	sys 0.000s
I20260709 12:28:23.748847  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.750275  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3112) took 1009 ms. Trace:
I20260709 12:28:23.751055  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1451) took 1423 ms. Trace:
I20260709 12:28:23.752326  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3600) took 1569 ms. Trace:
I20260709 12:28:23.713896 14566 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:23.758183  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:23.766736  6308 rpcz_store.cc:276] 0709 12:28:22.740528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.741135 (+   607us) service_pool.cc:224] Handling call
0709 12:28:23.750036 (+1008901us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b54000b3454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:23.750211 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49698176}
W20260709 12:28:23.730508 14645 consensus_meta.cc:287] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.574s	user 0.004s	sys 0.008s
I20260709 12:28:23.760135  7317 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:23.761626 14214 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:23.766908  7319 rpcz_store.cc:276] 0709 12:28:22.327683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.327834 (+   151us) service_pool.cc:224] Handling call
0709 12:28:22.441490 (+113656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:22.554452 (+112962us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.554501 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:22.554541 (+    40us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:22.554573 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.750811 (+1196238us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400479ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:23.750855 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.750985 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259276032}
I20260709 12:28:23.778792  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.774138 14641 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:23.764295  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 159225us
I20260709 12:28:23.717192 14627 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:23.763406  7314 rpcz_store.cc:276] 0709 12:28:22.655299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.860248 (+204949us) service_pool.cc:224] Handling call
0709 12:28:22.885066 (+ 24818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:23.387644 (+502578us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.743134 (+355490us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400508554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:23.743323 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762862336}
I20260709 12:28:23.766974  6456 rpcz_store.cc:276] 0709 12:28:22.183126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.819829 (+636703us) service_pool.cc:224] Handling call
0709 12:28:22.820135 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:22.909872 (+ 89737us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:22.909915 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:22.942998 (+ 33083us) log.cc:844] Serialized 26 byte log entry
0709 12:28:23.084441 (+141443us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:23.084487 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.751688 (+667201us) spinlock_profiling.cc:243] Waited 67.2 ms on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:23.751758 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:23.751829 (+    71us) raft_consensus.cc:1685] finished
0709 12:28:23.751850 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:23.752245 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2119,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"spinlock_wait_cycles":154566016,"thread_start_us":99588,"threads_started":1,"wal-append.queue_time_us":188828}
I20260709 12:28:23.734546 14646 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:23.770974 14645 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.768911 14593 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:23.769143 14314 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:23.804786  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:23.784732 14641 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.765407 14753 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:28:23.811156 14314 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.812362 14314 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:23.812182 14186 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.803620  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:23.777707 14565 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:23.786540  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.807237 14748 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:23.807603 14746 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:23.788313 14702 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:23.780951 14214 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:23.823040  7314 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: [26.20-26.20]
I20260709 12:28:23.787124 14192 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:28:23.885514  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63669.8us
W20260709 12:28:23.887475  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.805974 14749 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:23.880584  6304 raft_consensus.cc:2368] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
W20260709 12:28:23.878835  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 101292us
W20260709 12:28:23.903507 14754 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:28:23.906304 14452 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:23.909036 14452 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:23.911003 14214 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:23.951062  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:23.966460  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.891119  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:23.775470 14628 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:23.976830 14755 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:28:23.977576 14628 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:23.814162 14641 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:23.884419 14627 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:23.806128 14660 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:28:23.823796 14748 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:23.984257  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 93231.8us
I20260709 12:28:23.996793  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:24.001825  6303 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:28:23.975454 14192 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:23.889899 14640 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.108s
I20260709 12:28:23.892472 14676 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.108s
I20260709 12:28:23.993337 14649 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:23.915565 14716 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.095s
I20260709 12:28:23.993886 14748 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.021149 14676 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:28:24.023838 14579 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.570s	user 0.011s	sys 0.000s
I20260709 12:28:24.043826 14593 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.002146 14452 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:23.997957 14692 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:24.049295  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.015774 14642 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.058856 14702 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:24.060298 14645 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:24.061897 14192 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:24.072062  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54550.1us
I20260709 12:28:24.003858 14708 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:24.072674 14186 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:24.099977 14047 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.100740 14047 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.076336 14676 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:24.084851 14649 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:24.116706 14764 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:28:24.119117 14649 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:24.120886 14658 consensus_meta.cc:287] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 12:28:24.102651 14761 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.131s
I20260709 12:28:24.090451 14642 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.123631 14692 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:24.088766 14579 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.128906  7314 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:24.123731 14642 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:24.132993 14593 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:24.133039 14690 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.817s	user 0.013s	sys 0.000s
I20260709 12:28:24.134930 14690 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:24.116688 14047 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:24.131409 14676 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.137276 14676 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:24.145131  7316 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43762 (request call id 1114) took 1520 ms. Trace:
I20260709 12:28:24.145337  7335 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:24.145598  7316 rpcz_store.cc:276] 0709 12:28:22.624669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.731166 (+106497us) service_pool.cc:224] Handling call
0709 12:28:24.144923 (+1413757us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b540048a154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:24.145066 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825724160}
I20260709 12:28:24.147359  7316 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
W20260709 12:28:24.148916 14757 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:28:24.151305 14579 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:24.155845  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:24.156688  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.162583  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:24.163617  7318 raft_consensus.cc:2368] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:28:24.163848  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.164552  6454 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.168756  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:24.172397 14734 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.795s	user 0.009s	sys 0.002s
I20260709 12:28:24.127593 14641 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.194289  6454 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:24.200135  6308 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:28:24.199138  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:24.207599 14593 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:24.208550 14593 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:24.212292  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3601) took 1995 ms. Trace:
I20260709 12:28:24.212780  6450 rpcz_store.cc:276] 0709 12:28:22.216791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.065317 (+848526us) service_pool.cc:224] Handling call
0709 12:28:23.065665 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:23.076196 (+ 10531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:23.076223 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:23.076611 (+   388us) log.cc:844] Serialized 26 byte log entry
0709 12:28:23.315168 (+238557us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:23.315214 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.315309 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:24.171564 (+856255us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:24.211924 (+ 40360us) raft_consensus.cc:1685] finished
0709 12:28:24.211954 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.212215 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":984,"raft.queue_time_us":187,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":13210,"spinlock_wait_cycles":24136448,"thread_start_us":191776,"threads_started":1,"wal-append.queue_time_us":262751}
I20260709 12:28:24.213320  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.209713 14593 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.217902  6454 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:24.204957 13772 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:24.220716 14641 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:24.221088 14706 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.403s	user 0.000s	sys 0.003s
I20260709 12:28:24.223628 14596 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.222985  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:24.224155 14596 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.225024  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1197) took 1294 ms. Trace:
I20260709 12:28:24.225034 14652 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:24.225960 14652 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.226290 14596 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:24.227236  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1448) took 1962 ms. Trace:
I20260709 12:28:23.911320  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:24.227609  7332 rpcz_store.cc:276] 0709 12:28:22.264485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.282287 (+ 17802us) service_pool.cc:224] Handling call
0709 12:28:22.314882 (+ 32595us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:22.329391 (+ 14509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:22.329424 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:22.486955 (+157531us) log.cc:844] Serialized 52 byte log entry
0709 12:28:22.671436 (+184481us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:22.671475 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:22.671541 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:24.220463 (+1548922us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:24.226922 (+  6459us) raft_consensus.cc:1685] finished
0709 12:28:24.226941 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.227162 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14071,"raft.queue_time_us":28228,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":35851,"spinlock_wait_cycles":53280000,"thread_start_us":175887,"threads_started":1,"wal-append.queue_time_us":271133}
I20260709 12:28:24.225726  7335 rpcz_store.cc:276] 0709 12:28:22.941848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.942623 (+   775us) service_pool.cc:224] Handling call
0709 12:28:24.145131 (+1202508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:24.145738 (+   607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.224773 (+ 79035us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540048a150. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:24.224950 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2716988800}
I20260709 12:28:24.226507 14652 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:24.233220 14495 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:24.236956  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3562) took 1360 ms. Trace:
I20260709 12:28:24.237455  6385 rpcz_store.cc:276] 0709 12:28:22.876200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.876323 (+   123us) service_pool.cc:224] Handling call
0709 12:28:23.386689 (+510366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:23.386990 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:23.387030 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:23.387091 (+    61us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:23.387138 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.387203 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.236733 (+849530us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b5400109fd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:24.236882 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127064832}
I20260709 12:28:24.238632 14596 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:24.239235 14596 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.241341 14596 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:24.269145 14671 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:24.271313 14672 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:24.272471  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.275110  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1217) took 1679 ms. Trace:
I20260709 12:28:24.275949 14676 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:24.278709 14668 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:24.279136 14668 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:24.279995  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:24.281584  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:24.282290  7335 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:28:24.280020 14668 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.292114  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:24.275470  6381 rpcz_store.cc:276] 0709 12:28:22.595274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.595484 (+   210us) service_pool.cc:224] Handling call
0709 12:28:22.610683 (+ 15199us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:22.884525 (+273842us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:22.903151 (+ 18626us) log.cc:844] Serialized 14 byte log entry
0709 12:28:23.105280 (+202129us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:23.114908 (+  9628us) log.cc:844] Serialized 26 byte log entry
0709 12:28:23.115280 (+   372us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:23.115343 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.274736 (+1159393us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54003d36d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:24.274778 (+    42us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:24.274824 (+    46us) raft_consensus.cc:1685] finished
0709 12:28:24.274917 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.275052 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40639,"prepare.queue_time_us":18042,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":68689,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":15306,"spinlock_wait_cycles":348651904,"thread_start_us":286041,"threads_started":2,"wal-append.queue_time_us":360152}
I20260709 12:28:24.295876  6351 leader_election.cc:400] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:28:24.296245  6351 leader_election.cc:403] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:28:24.295182 14653 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:24.298765 14768 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:24.305135 14653 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:24.306921 14728 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.981s	user 0.005s	sys 0.014s
I20260709 12:28:24.307099  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.308526  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.308967 14653 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:24.309288  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3572) took 1419 ms. Trace:
I20260709 12:28:24.303717  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:24.309743  6377 rpcz_store.cc:276] 0709 12:28:22.889660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.925630 (+ 35970us) service_pool.cc:224] Handling call
0709 12:28:22.925924 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:23.043702 (+117778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:23.043737 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:23.044226 (+   489us) log.cc:844] Serialized 26 byte log entry
0709 12:28:23.126052 (+ 81826us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:28:23.154437 (+ 28385us) log.cc:844] Serialized 14 byte log entry
0709 12:28:23.166759 (+ 12322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:23.552122 (+385363us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:24.308962 (+756840us) spinlock_profiling.cc:243] Waited 79 ms on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:24.309018 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:24.309068 (+    50us) raft_consensus.cc:1685] finished
0709 12:28:24.309083 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.309221 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33532,"prepare.queue_time_us":116406,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":85287,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":18334,"spinlock_wait_cycles":216115584,"thread_start_us":23820,"threads_started":1,"wal-append.queue_time_us":216806}
I20260709 12:28:24.312208  7335 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:24.309191  6300 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:24.305981 14768 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:24.313130  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.319872 14333 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:24.311254 14573 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:24.300031 14668 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:24.328869  7331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1196) took 1782 ms (client timeout 1675 ms). Trace:
W20260709 12:28:24.329335  7331 rpcz_store.cc:269] 0709 12:28:22.546064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.546251 (+   187us) service_pool.cc:224] Handling call
0709 12:28:24.328630 (+1782379us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540056f254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:24.328799 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258463104}
I20260709 12:28:24.330298 14653 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:24.330794 14653 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:24.331187 14653 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:24.332453 14663 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:24.333403 14663 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:24.333746 14663 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:24.334620  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.334707 14663 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.338698 14663 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:24.339931 14768 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:24.348834 14771 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:28:24.348949  6307 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:28:24.350181  6307 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 10
I20260709 12:28:24.350464  6307 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:28:24.350776  6307 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:24.352824 14664 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:24.353386 14664 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:24.355532 14652 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.360244 14596 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:24.360826 14596 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.361546  6455 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:28:24.363891  7319 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:28:24.364862 14579 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:24.368489 14596 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:24.360821  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.354380 14664 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.370003 14656 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.370687 14664 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:24.369902  6451 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:28:24.373049  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.375219  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.380820  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.355715 14653 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.381390  6456 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:24.397390  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.397184  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.399400  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:24.399773  6451 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:24.399863  6452 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:24.397192  6458 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:28:24.402000  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.403450  6377 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:24.404719  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.404982  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.407011  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.409116  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.405725  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.296707 14748 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.411068  6450 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:24.415091  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.418478  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.418650  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.420099  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:24.420238  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:24.422715  6381 raft_consensus.cc:2368] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26.
I20260709 12:28:24.422775  7335 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [19.16-21.17]
W20260709 12:28:24.424847  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6ea0c2d7699e4559823a28a280a0e713. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:24.401607 14656 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:28:24.463937  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.468228  6456 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:24.470295  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.471007  6459 raft_consensus.cc:3060] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:28:24.413926 14652 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:24.473527  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:24.475615  6451 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
W20260709 12:28:24.489077  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.464747 14192 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:24.412670 14721 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:28:24.519308 14596 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:24.469381 14671 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:28:24.523811  6348 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:24.524060 14653 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:24.524415 14566 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:28:24.535601  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 536959us
I20260709 12:28:24.542192  6459 raft_consensus.cc:2468] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:24.557399  7335 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:24.565227  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:24.572252  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 84473.6us
I20260709 12:28:24.550385 14573 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:28:24.583339  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:24.564363 14640 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:24.585677  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:24.586763  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:24.588821  7331 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:24.569520 14665 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:24.590296  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:24.591408  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:24.593246  7335 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:24.593246  7288 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.631s (SENT)
I20260709 12:28:24.594985  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:24.578989 14714 thread.cc:641] tablet-download-pool [worker] (thread pool) Time spent creating pthread: real 1.218s	user 0.019s	sys 0.012s
I20260709 12:28:24.596763  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:24.598635  7319 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:28:24.599085  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 169028us
W20260709 12:28:24.607472  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 610732us
I20260709 12:28:24.609508  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3500) took 1092 ms. Trace:
I20260709 12:28:24.610064  6307 rpcz_store.cc:276] 0709 12:28:23.517046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.552498 (+ 35452us) service_pool.cc:224] Handling call
0709 12:28:23.552802 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:24.351261 (+798459us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.603754 (+252493us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b54002df0d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:24.609449 (+  5695us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1830048512}
I20260709 12:28:24.588500 14768 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:24.571564 14647 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.024s
I20260709 12:28:24.615160  6308 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:28:24.591284 14668 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:24.713999  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 104486us
W20260709 12:28:24.713904  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 105656us
I20260709 12:28:24.642869  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:24.589845 14192 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:24.772501  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:24.776240  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 133467us
I20260709 12:28:24.778329  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.778765  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.779027  6377 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:24.780354  6377 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:24.646974  6381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:24.607450 14186 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:24.602319 14579 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:24.619403 14747 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.594249 14702 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 12:28:24.682442 14775 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:28:24.598083 14672 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:24.711908 14593 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.684070 14676 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:28:24.729135  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 47145.7us
I20260709 12:28:24.739928  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3503) took 1217 ms. Trace:
W20260709 12:28:24.742471  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.550s (SENT)
I20260709 12:28:24.807878  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:24.745110 14779 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:28:24.783509 14721 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:24.719893 14753 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:24.801998 14785 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:28:24.751649 14647 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:24.729714 14640 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.044s
I20260709 12:28:24.783985  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.785192  6454 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:24.835027  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.835780  6454 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:24.786316  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.836634  6455 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:24.789409  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.789354  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.771327 14668 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:24.791417  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.792397  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.793175  6459 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:24.794349  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.785940 14625 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:24.798015  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.798636  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.748560 14761 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:28:24.803625  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.805485  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.848789  6452 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:24.807629  6304 rpcz_store.cc:276] 0709 12:28:23.522682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.529043 (+  6361us) service_pool.cc:224] Handling call
0709 12:28:24.739693 (+1210650us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:24.739853 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":799403264}
W20260709 12:28:24.599206 14714 thread.cc:608] tablet-download-pool [worker] (thread pool) Time spent starting thread: real 1.238s	user 0.020s	sys 0.012s
I20260709 12:28:24.850550 14714 tablet_copy_client.cc:323] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:28:24.850634  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.782967 14711 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:24.852113 14711 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:24.813799  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71387.8us
I20260709 12:28:24.818825 14579 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:24.736850  6376 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
W20260709 12:28:24.819234 14774 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:28:24.843876  6384 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:28:24.844830  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:24.845425  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:24.806717 14676 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:24.848230  6380 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 27 because replica has last-logged OpId of term: 26 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 19.
I20260709 12:28:24.851516  6304 raft_consensus.cc:2376] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26. Re-sending same reply.
I20260709 12:28:24.853013  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.865477  6382 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:24.760525  7335 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:24.853794  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.854842  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.867269  6383 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.861388  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.836261 14721 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:24.815788 14788 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:28:24.791283 14186 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.862249  6458 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:28:24.868912  7288 leader_election.cc:336] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.763s (SENT)
I20260709 12:28:24.847113 14625 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:24.846026 14495 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:24.871206  6457 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:28:24.846057 14668 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:24.873481  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:24.887303  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.881048  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:24.888234  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.882998  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.811009 14593 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.881099  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:24.896363  6459 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:24.823079 14753 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.898089  6307 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:24.898975  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.899344  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.899693  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.902278  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.903800  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:24.905305  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.905829  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.906786  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.839232 14647 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:24.913276  6304 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:24.903072  6307 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.925909  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:24.926986  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46019.5us
I20260709 12:28:24.931874  7288 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:24.934425  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.910792 14649 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:24.935772  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:24.935837 14649 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.886915 14495 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:24.938733 14186 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:24.939198 14777 log.cc:927] Time spent T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:28:24.935153 14674 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:24.940037 14776 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.389s	user 0.005s	sys 0.000s
I20260709 12:28:24.939311 14495 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.941745  7287 leader_election.cc:400] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26.
I20260709 12:28:24.942106 14593 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:24.909956 14314 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:24.943202 14314 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:24.943579 14314 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:24.946769  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:24.948446 14781 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:28:24.949663 14579 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:24.951706 14677 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:24.954764  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.955930  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.956835  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.957432  6302 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:24.958077  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.958770  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:24.959111  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3498) took 1445 ms. Trace:
I20260709 12:28:24.960795  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:24.960454 14625 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.961347  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:24.961824  7335 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:28:24.962183  6456 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:24.962263 14625 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:24.963174  6377 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:24.966379 14790 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:28:24.942636 14783 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:28:24.942129  7287 leader_election.cc:403] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:24.935190  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.967626 14747 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:24.975314  6307 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:24.975397  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.973055  6302 raft_consensus.cc:2393] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:25.032644  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.033790  6304 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:25.034839  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.035487  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:28:24.959604  6310 rpcz_store.cc:276] 0709 12:28:23.513507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.521140 (+  7633us) service_pool.cc:224] Handling call
0709 12:28:23.750197 (+229057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:23.750422 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:23.750759 (+   337us) log.cc:844] Serialized 14 byte log entry
0709 12:28:23.988814 (+238055us) log.cc:844] Serialized 14 byte log entry
0709 12:28:24.354783 (+365969us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b18004e8a40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1
0709 12:28:24.355474 (+   691us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b18004e8a40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:24.516838 (+161364us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b18004e8a40. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:24.573021 (+ 56183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:24.573151 (+   130us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:24.573300 (+   149us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.573571 (+   271us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:24.958921 (+385350us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54000b3450. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:24.959038 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":593995,"prepare.queue_time_us":958,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":84,"raft.queue_time_us":570629,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":17957,"spinlock_wait_cycles":566648576,"thread_start_us":287627,"threads_started":2,"wal-append.queue_time_us":328155}
I20260709 12:28:25.036378  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.047124  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.048767  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.037873  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.986814  6383 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:24.841077 14192 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:25.033608 14789 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:28:24.991058  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.051108  6307 raft_consensus.cc:2393] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:24.888774 13772 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.093s
I20260709 12:28:25.052058  6459 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
W20260709 12:28:25.054320  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:25.036144  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94547.3us
I20260709 12:28:25.055843  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:24.973104 14664 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.562s	user 0.014s	sys 0.000s
I20260709 12:28:25.056963  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.057809  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:24.972380 14756 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:24.970013 14748 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:25.068097  6308 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:25.047767 14579 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26.
I20260709 12:28:25.109133 14756 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:25.087316  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.117245  6307 raft_consensus.cc:2368] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24.
I20260709 12:28:25.110394  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.050288 14649 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:25.118921  6308 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:25.116333 14756 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.118348  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.123737  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:25.125612 14791 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
W20260709 12:28:25.131378 14792 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:28:25.133472  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3513) took 1127 ms. Trace:
I20260709 12:28:25.133993  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.144747  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.145752  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.135404 14435 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:25.146447 14435 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:25.147286 14435 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.148590  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:25.150030  7319 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:28:25.155984 14680 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:25.156550 14680 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:25.157310 14640 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:25.157529 14680 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.158260 14640 consensus_queue.cc:1243] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:25.159795 14647 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:25.160956  7285 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:25.163220 14677 raft_consensus.cc:2804] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:25.163949 14677 raft_consensus.cc:697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:28:25.164513 14656 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.701s	user 0.011s	sys 0.000s
I20260709 12:28:25.165242 14677 consensus_queue.cc:237] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:24.966943 14790 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:25.133857  6301 rpcz_store.cc:276] 0709 12:28:24.005694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.056616 (+ 50922us) service_pool.cc:224] Handling call
0709 12:28:24.057179 (+   563us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:24.579535 (+522356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.579561 (+    26us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:24.580291 (+   730us) log.cc:844] Serialized 78 byte log entry
0709 12:28:24.764957 (+184666us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:24.798983 (+ 34026us) log.cc:844] Serialized 14 byte log entry
0709 12:28:24.799584 (+   601us) log.cc:844] Serialized 14 byte log entry
0709 12:28:24.800188 (+   604us) log.cc:844] Serialized 14 byte log entry
0709 12:28:24.800651 (+   463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.800726 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.133270 (+332544us) raft_consensus.cc:1685] finished
0709 12:28:25.133291 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.133416 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":590911,"prepare.queue_time_us":195179,"prepare.run_cpu_time_us":154,"prepare.run_wall_time_us":4540,"raft.queue_time_us":526946,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":186,"spinlock_wait_cycles":4260736,"thread_start_us":144957,"threads_started":1,"wal-append.queue_time_us":242950}
W20260709 12:28:25.174001  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.176190  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.192022  6302 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:28:25.047672 14414 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:28:25.141168  6307 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:25.197212  6308 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:25.239655  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.050873 14047 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 12:28:25.240372  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.241279  6308 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:28:25.225155  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:24.971772 14676 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:25.159826 14780 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.562s	user 0.002s	sys 0.000s
I20260709 12:28:25.254829  6308 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 16.
I20260709 12:28:25.255877  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.256654  6308 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:25.263957  6379 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
W20260709 12:28:25.185364 14784 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.573s	user 0.004s	sys 0.000s
I20260709 12:28:25.321573  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.323704  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.325255  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:25.270149  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.174386 14709 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.277712  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.328421  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:25.331220  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.203567 14680 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:25.327736  6310 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:25.102779 14747 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.185s
I20260709 12:28:25.334821  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.085459 14628 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:25.333837  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.239144 14652 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:24.966461 14645 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.355s
I20260709 12:28:25.104064 14668 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.345183  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.351698  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.182740 14671 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.154s
I20260709 12:28:25.356106  6310 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:28:25.347141  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.358206  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3522) took 1052 ms. Trace:
I20260709 12:28:25.358744  6307 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:25.358811  6308 rpcz_store.cc:276] 0709 12:28:24.305426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.357359 (+1051933us) service_pool.cc:224] Handling call
0709 12:28:25.357693 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.357878 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:25.357903 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:25.357944 (+    41us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:25.357984 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.358034 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.358140 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:25.360432  6381 raft_consensus.cc:606] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:25.360836  6381 raft_consensus.cc:614] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:25.350945 14628 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.363191  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.363610 14628 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:25.364099  6310 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:25.365166 14656 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.365677 14656 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:25.193360 14672 consensus_queue.cc:1175] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4268023ecea149cd83cba77db53c3100" responder_term: 25 status { last_received { term: 23 index: 17 } last_committed_idx: 17 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:25.086686 14721 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.392427  6307 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:25.395800  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.396844  6307 raft_consensus.cc:2368] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:25.348575 14676 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:25.401924  6307 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:25.335978 14748 consensus_queue.cc:1059] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:28:25.336679 14709 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:25.425882  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 390335us
I20260709 12:28:25.419924  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1201) took 1018 ms. Trace:
W20260709 12:28:25.412288 14663 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.073s	user 0.012s	sys 0.003s
I20260709 12:28:25.354166 14652 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.437856  6454 raft_consensus.cc:2410] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:28:25.427469  7320 rpcz_store.cc:276] 0709 12:28:24.401943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.402196 (+   253us) service_pool.cc:224] Handling call
0709 12:28:24.402532 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:24.402858 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:24.402882 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.417537 (+ 14655us) log.cc:844] Serialized 26 byte log entry
0709 12:28:24.555289 (+137752us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:24.555353 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:24.555469 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.184440 (+628971us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:25.415001 (+230561us) raft_consensus.cc:1685] finished
0709 12:28:25.415033 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.419846 (+  4813us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37849984,"thread_start_us":137342,"threads_started":1,"wal-append.queue_time_us":166557}
I20260709 12:28:25.390010 14314 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.451438  6452 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:28:25.425362 14668 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:25.366351 14790 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.456951  6351 leader_election.cc:400] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:28:25.457329  6351 leader_election.cc:403] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:28:25.445847 14663 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:25.456736 14799 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:28:25.461673 14665 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:28:25.466889 14663 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.463418 14761 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:28:25.478279  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 307617us
I20260709 12:28:25.480816  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:25.487665 14749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.011s	sys 0.020s
I20260709 12:28:25.487396 14047 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:25.478777 14761 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:25.488842 14625 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:25.517582  6425 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.521s (SENT)
W20260709 12:28:25.508991 14749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.012s	sys 0.021s
I20260709 12:28:25.489315 14766 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:25.515471 14793 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:28:25.512674 14649 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:25.536865  6425 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.526s (SENT)
I20260709 12:28:25.541342  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:25.538753  6305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.017s	sys 0.025s
I20260709 12:28:25.516726 14656 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:25.543130  6305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.591s	user 0.018s	sys 0.025s
I20260709 12:28:25.544361  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:25.560154  6425 leader_election.cc:336] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.540s (SENT)
I20260709 12:28:25.560742  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:25.547745 14798 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:28:25.527760 14047 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.543985 14625 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:25.532034 14766 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:25.543108 14702 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:28:25.599260  6451 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:25.603579  7320 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:28:25.544678 14649 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.599304  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:25.615377  7320 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:28:25.615415  6456 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:25.613832  6302 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:28:25.617496  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.618335  6302 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:25.619179  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.620028  6302 raft_consensus.cc:2410] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:28:25.622058 14803 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:28:25.627506 14753 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:25.628391  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3512) took 1624 ms. Trace:
I20260709 12:28:25.625332  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3509) took 1626 ms. Trace:
I20260709 12:28:25.631268  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:25.625110 14802 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:28:25.631026  6305 rpcz_store.cc:276] 0709 12:28:23.999247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.999410 (+   163us) service_pool.cc:224] Handling call
0709 12:28:23.999892 (+   482us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:24.872336 (+872444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.872364 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:24.901315 (+ 28951us) log.cc:844] Serialized 78 byte log entry
0709 12:28:25.544287 (+642972us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:25.617454 (+ 73167us) log.cc:844] Serialized 14 byte log entry
0709 12:28:25.617964 (+   510us) log.cc:844] Serialized 14 byte log entry
0709 12:28:25.618549 (+   585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.618651 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.625009 (+  6358us) raft_consensus.cc:1685] finished
0709 12:28:25.625037 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.625247 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":583623,"prepare.queue_time_us":47276,"raft.queue_time_us":556985,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":10371,"thread_start_us":956499,"threads_started":3,"wal-append.queue_time_us":676754}
I20260709 12:28:25.632727  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:25.639467 14804 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:28:25.632007  6301 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:25.640945  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.630728  6303 rpcz_store.cc:276] 0709 12:28:24.004115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.006084 (+  1969us) service_pool.cc:224] Handling call
0709 12:28:24.006462 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:24.781205 (+774743us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.781257 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:24.785759 (+  4502us) log.cc:844] Serialized 26 byte log entry
0709 12:28:25.066393 (+280634us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:25.066453 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:25.066571 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:25.628128 (+561557us) raft_consensus.cc:1685] finished
0709 12:28:25.628173 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:25.628323 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":682535,"raft.queue_time_us":564040,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8936,"thread_start_us":498125,"threads_started":2,"wal-append.queue_time_us":537203}
I20260709 12:28:25.643682  6303 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:25.644135  6303 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:25.645256  6303 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.639750  7335 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:25.648787  6303 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:25.651398 14711 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:25.635378  7332 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:28:25.636101  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:25.652715  7319 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 31: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:25.635385  6459 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:25.590611 14813 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:25.638262  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.661147  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.661906  6305 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:25.662900  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.661914  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.664822  7320 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:25.666080  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.669688  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.668294  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:25.669746  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:25.680169  6425 leader_election.cc:336] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.758s (SENT)
I20260709 12:28:25.680788  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:25.684396  6303 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:25.637674 14652 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:25.672688  7319 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:25.697671  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.698340  6303 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:25.699640  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.701660  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.666631 14677 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.668272 14668 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:25.707594  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.711438 14047 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:25.722975  6425 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.824s (SENT)
I20260709 12:28:25.723415  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.714433 14649 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:25.717695 14800 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:28:25.718382 14812 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:28:25.695737 14569 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:25.725862  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.726468  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:25.726567  6303 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:25.700657 14813 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:25.732725  6425 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.903s (SENT)
I20260709 12:28:25.733270  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:25.735857  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:25.737970 14656 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:25.667619 14414 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:28:25.668756 14690 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:28:25.742995  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58396 (request call id 56) took 1139 ms. Trace:
I20260709 12:28:25.763818  6450 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:25.769204  6381 rpcz_store.cc:276] 0709 12:28:24.603734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.603930 (+   196us) service_pool.cc:224] Handling call
0709 12:28:25.742786 (+1138856us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b54001f54d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:25.742902 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329141120}
I20260709 12:28:25.742919  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:25.764426  6380 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:25.766462  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:25.771356 14826 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:25.771667  6425 leader_election.cc:336] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.951s (SENT)
I20260709 12:28:25.773037 14766 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:25.773591 14801 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
I20260709 12:28:25.775280  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.772033  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:25.779528  6451 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:28:25.781219 14811 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:28:25.721762 14677 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:25.783036 14817 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:28:25.784166 14677 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:25.784785 14677 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:25.787326 14749 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:25.788460 14753 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:25.788985 14753 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:25.789861 14753 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:25.791306 14753 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:25.794097  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:25.794152  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:25.799142  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:25.801760  6425 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.955s (SENT)
I20260709 12:28:25.802345  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.804402  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:25.807086  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:25.772235 14826 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:25.744148  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
W20260709 12:28:25.821844  6425 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.972s (SENT)
I20260709 12:28:25.852110  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:25.854010  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:25.855394  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:25.856343  6459 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 23
W20260709 12:28:25.857510  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 195723us
I20260709 12:28:25.807077 14677 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:25.805779 14815 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:25.805971 14645 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:25.806563 14254 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:25.859062  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:25.857795 14813 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:25.885834  6459 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:25.809463 14690 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:28:25.879477  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-1ee2be3ef27a4ec7bd5946b81b8ee8c3
I20260709 12:28:25.885809 14826 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:28:25.930003  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:25.894934  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:26.008194  6459 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
W20260709 12:28:25.892839  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 126514us
W20260709 12:28:25.971184  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 228366us
I20260709 12:28:26.009043  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:25.952561  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 98636.5us
W20260709 12:28:25.962627  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 158400us
W20260709 12:28:25.954267 14680 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.000s
I20260709 12:28:25.919917 14823 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 12:28:25.885289 14254 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:25.932673 14495 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:26.010566  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:26.011341  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:26.086890  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:26.044016  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.107136 14254 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:26.109720 14832 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:28:26.094343  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:26.123209  6451 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:28:26.125715 14823 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:26.014746  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:25.897326 14824 ts_tablet_manager.cc:933] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:26.127475  6453 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:26.128222  6459 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:26.108500  6458 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:26.091670  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 161720us
I20260709 12:28:26.103426 14569 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:26.144677  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 134314us
I20260709 12:28:26.145057  6459 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:26.145485  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:26.013792  6425 leader_election.cc:336] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.994s (SENT)
I20260709 12:28:25.887853 14748 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:25.980463 14830 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:25.895090 14645 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:26.148948  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.969569 14831 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:26.147003  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.151252  6450 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:28:26.151647  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 259747us
I20260709 12:28:25.981815 14709 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:26.161463  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 147684us
I20260709 12:28:26.094569 14834 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.163641  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.902956 14805 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.093751 14680 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:26.170768  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:25.895498 14047 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.267s
I20260709 12:28:26.174252  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:26.151235 14753 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:26.151080 14833 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:26.166405 14813 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:26.163445 14748 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.165788 14831 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.166177 14627 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:26.178087  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.174921 14790 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:26.208469  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 196019us
I20260709 12:28:26.178797 14711 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:28:26.211454  6304 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:26.183168 14842 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:26.197769 14645 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:26.172678 14690 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:28:26.167402 14830 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.183395 14833 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:26.177171 14680 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:26.224735  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81354.4us
I20260709 12:28:26.174000 14834 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.215027  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:26.218705  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:26.229360  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 58721us
I20260709 12:28:26.175470 14805 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.164438 14823 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.067s
I20260709 12:28:26.230814  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:26.213285 14831 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:26.218712 14711 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:26.218858 14842 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:26.218427 14677 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:26.237818 14690 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:26.235389  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:26.215173 14748 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:26.228250 14833 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:26.218611 14709 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:26.233850 14827 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:26.266652  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51755.5us
I20260709 12:28:26.248004  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:26.233539 14790 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:28:26.263279  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 103047us
I20260709 12:28:26.265038 14824 tablet_copy_client.cc:323] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:28:26.265403  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:26.265506 14690 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:26.240958 14677 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.269469 14790 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:26.266543  7285 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:26.272782  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:26.269634 14829 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:28:26.277429  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.277667  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.278327  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.279021  6305 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
W20260709 12:28:26.275204 14668 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.001s
I20260709 12:28:26.281644  6303 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:26.281903  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.282729  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:26.283150  6305 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:28:26.286511  6198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 301) took 35791 ms (client timeout 15000 ms). Trace:
I20260709 12:28:26.285867 14790 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.287690 13772 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:26.288491 14844 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:28:26.289788 14835 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:28:26.291780 14842 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.280453 14846 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:26.293629 14834 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:26.307126 14692 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:26.331895 14692 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.333467  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:26.334348 14834 raft_consensus.cc:2764] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:28:26.303022  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:26.304827  6425 leader_election.cc:400] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
W20260709 12:28:26.304847  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.301791  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:26.307508  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:26.311236 14839 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:28:26.278641 14813 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:26.314798 14711 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.315223 14647 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:26.315768 13772 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:26.306628 14815 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:26.303776 14668 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:26.317414 14849 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:28:26.319172 14840 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:26.366623  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.373769 14823 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:26.303586 14709 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:26.326850 14846 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:26.375432  6395 tablet_copy_source_session.cc:215] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:28:26.375219 14823 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:26.375949 14830 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:26.376663 14830 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:26.377086 14823 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:26.344101 14844 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:28:26.379477  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.358247 14834 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.380769  7314 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:28:26.344666 14813 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:26.380987  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.381505 14813 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:26.383417 13772 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.327704  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:26.301577  6198 rpcz_store.cc:269] 0709 12:27:50.434402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.715827 (+3281425us) service_pool.cc:224] Handling call
0709 12:28:26.219434 (+32503607us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:16.977714 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:17.035375 (+ 57661us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:17.035439 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:28:17.035465 (+    26us) tablet.cc:662] Decoding operations
0709 12:28:17.036786 (+  1321us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:17.036892 (+   106us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:17.036934 (+    42us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:28:17.038003 (+  1069us) tablet.cc:701] Row locks acquired
0709 12:28:17.038028 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:28:17.038223 (+   195us) write_op.cc:270] Start()
0709 12:28:17.038364 (+   141us) write_op.cc:276] Timestamp: P: 1783600097038156 usec, L: 0
0709 12:28:17.038392 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:28:17.740172 (+701780us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b5400124420. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3224520 00007f6fa32adb42 00007f6fa32a2f67 00007f6fa32a2da4 00007f6fa6cdd581 00007f6fa6cdcd58 00007f6fa6ce19f1 00007f6fa6ce19a9 00007f6fa6ce1939 00007f6fa6ce1901
0709 12:28:17.741086 (+   914us) log.cc:844] Serialized 3915 byte log entry
0709 12:28:18.806748 (+1065662us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:19.094359 (+287611us) op_driver.cc:464] REPLICATION: finished
0709 12:28:19.466670 (+372311us) write_op.cc:301] APPLY: starting
0709 12:28:19.466815 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:19.507942 (+ 41127us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:19.507987 (+    45us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:19.785462 (+277475us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:19.785853 (+   391us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=0,mrs_lookups=0
0709 12:28:19.785919 (+    66us) write_op.cc:312] APPLY: finished
0709 12:28:19.786467 (+   548us) log.cc:844] Serialized 97 byte log entry
0709 12:28:19.833994 (+ 47527us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:19.834195 (+   201us) write_op.cc:454] Released schema lock
0709 12:28:19.834270 (+    75us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:19.934908 (+100638us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:19.937763 (+  2855us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2056,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":359573,"apply.run_cpu_time_us":18158,"apply.run_wall_time_us":426146,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":660397,"prepare.run_cpu_time_us":104363,"prepare.run_wall_time_us":1508743,"raft.queue_time_us":149207,"raft.run_cpu_time_us":18412,"raft.run_wall_time_us":306290,"replication_time_us":1974351,"spinlock_wait_cycles":121958656,"thread_start_us":1255039,"threads_started":4,"wal-append.queue_time_us":742351,"wal-append.run_cpu_time_us":19413,"wal-append.run_wall_time_us":2472248}]]}
I20260709 12:28:26.305210  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:26.337893  6425 leader_election.cc:400] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:26.340504  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:26.342443  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.345242  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:26.350135  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:26.361848  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:26.361896  7329 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:26.363387  7318 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:26.335588 14790 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:26.369830  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.345860 14845 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:26.442471  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:26.388633  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61026.4us
I20260709 12:28:26.443264  6459 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:26.444593  6305 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:26.444976 14647 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:26.370430 14680 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:26.444953  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:26.448192  6453 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 31 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 20.
I20260709 12:28:26.449178  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.446417  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3644) took 1565 ms. Trace:
W20260709 12:28:26.450398  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112693us
I20260709 12:28:26.450058  6452 rpcz_store.cc:276] 0709 12:28:24.881509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.895266 (+ 13757us) service_pool.cc:224] Handling call
0709 12:28:24.895622 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.452104 (+556482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:26.446183 (+994079us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54000fc654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:26.446358 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264307584}
I20260709 12:28:26.451684  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.452929  6310 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:26.454231  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:26.454969  6310 raft_consensus.cc:2376] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31. Re-sending same reply.
I20260709 12:28:26.456534  6455 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.457001  6455 raft_consensus.cc:614] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:26.444046  6303 raft_consensus.cc:2376] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24. Re-sending same reply.
W20260709 12:28:26.463722  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 113723us
I20260709 12:28:26.465970  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:26.459052  7320 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:26.453176  6305 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 16.
I20260709 12:28:26.446532 14841 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.375329 14846 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:26.371275 14849 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.443403 14845 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:26.459656 14844 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:28:26.483879 14796 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:28:26.485203  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3507) took 2487 ms. Trace:
I20260709 12:28:26.560745  6309 rpcz_store.cc:276] 0709 12:28:23.997705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.998248 (+   543us) service_pool.cc:224] Handling call
0709 12:28:23.998661 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:24.881578 (+882917us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:24.881616 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:24.892860 (+ 11244us) log.cc:844] Serialized 52 byte log entry
0709 12:28:24.966117 (+ 73257us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b4800002958. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:25.416001 (+449884us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:25.416050 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:26.484737 (+1068687us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b54002ddcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:26.484804 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:26.484885 (+    81us) raft_consensus.cc:1685] finished
0709 12:28:26.484906 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:26.485124 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":542261,"raft.queue_time_us":567034,"raft.run_cpu_time_us":3535,"raft.run_wall_time_us":385769,"spinlock_wait_cycles":24821248,"thread_start_us":806439,"threads_started":2,"wal-append.queue_time_us":579315}
W20260709 12:28:26.500837  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1291) took 1586 ms (client timeout 1865 ms). Trace:
W20260709 12:28:26.563706  6304 rpcz_store.cc:269] 0709 12:28:24.914274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.036140 (+121866us) service_pool.cc:224] Handling call
0709 12:28:26.500561 (+1464421us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002d19d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:26.500743 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2677886720}
I20260709 12:28:26.510463 14795 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:26.513444 14761 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:26.517524 14806 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.050s	user 0.008s	sys 0.005s
I20260709 12:28:26.377067 14830 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:26.374161 14831 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.441442 14692 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:26.531284  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79672.1us
I20260709 12:28:26.480616 14671 raft_consensus.cc:3055] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:26.530423  6310 raft_consensus.cc:1275] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:28:26.364674 14254 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.459931 14823 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:26.547473  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:26.573580  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 129284us
I20260709 12:28:26.573834  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:26.657910  6308 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:26.660387  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 113055us
I20260709 12:28:26.574854  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.474351 14815 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:26.524255 14709 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:26.381047 14834 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.474767 14680 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:26.583927 14671 raft_consensus.cc:740] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:26.545182 14849 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:28:26.664889 14806 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:28:26.558055 14826 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:26.567747 14795 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:26.576059 14647 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:26.700877  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.657950  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:26.589722 14672 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.712417  6309 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:26.664847  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.580590  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.723407  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:26.535411 14841 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.583240 14843 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:26.584303 14805 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:26.656409 14761 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:26.859242  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.738716 14314 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:26.739354 14186 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:26.739454 14855 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:28:26.740130 14830 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.703367 14796 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:26.743948 14671 consensus_queue.cc:260] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.750072 14834 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:26.664366 14047 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.727456 14851 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.766654  6308 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:26.807216  6385 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:28:26.801512  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3290) took 2023 ms. Trace:
I20260709 12:28:26.546568 14845 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:26.860306  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 202468us
W20260709 12:28:26.811290  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.899997  6379 rpcz_store.cc:276] 0709 12:28:24.778260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.779438 (+  1178us) service_pool.cc:224] Handling call
0709 12:28:26.801305 (+2021867us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b5400065254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:26.801448 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548616448}
I20260709 12:28:26.823467 14663 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:26.865998  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 285514us
I20260709 12:28:26.866947 14788 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.895920  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:26.900954  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:26.822053  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:26.731555 14652 consensus_queue.cc:1048] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.152s
I20260709 12:28:26.863811 14186 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:26.868227 14400 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:26.879515 14709 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:26.890630 14831 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:26.901994 14573 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.942101 14665 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:26.906345  6308 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:26.994861  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:26.995093  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:26.996230  6456 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:26.869791 14672 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.938997 14834 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:26.936911 14647 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.842957 14795 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:26.966813 14842 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:26.967201 14254 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:26.936331 14671 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:26.959359  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:26.951345  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:27.024422  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 82827.7us
W20260709 12:28:26.990214  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:26.938295  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.027648  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.028613  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.009500 14857 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:28:26.999610  6305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.138s	sys 0.260s
I20260709 12:28:26.874874 14843 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:26.949419 14716 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:28:26.947337 14400 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:27.005679  6457 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:26.953341 14830 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:27.008294 14827 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.003s
I20260709 12:28:27.008373 14846 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:27.015825  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3839) took 2187 ms (client timeout 2657 ms). Trace:
I20260709 12:28:27.034693 14663 raft_consensus.cc:2764] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:28:26.766561 14826 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:27.001502 13772 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.014s	sys 0.018s
W20260709 12:28:27.029966  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 70687.5us
I20260709 12:28:27.025947  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:26.904860 14805 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:27.037735  6305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.139s	sys 0.261s
I20260709 12:28:27.038471 14768 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:26.891361 14796 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.040297  6307 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:28:27.039043  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3297) took 2225 ms. Trace:
I20260709 12:28:27.041260  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:27.095906  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:27.096781  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55641.9us
I20260709 12:28:27.096987  6307 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:26.992432 14702 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:28:27.084810 14796 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:27.000006 14573 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.138895  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3601) took 1603 ms. Trace:
I20260709 12:28:27.139060  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.139683  6383 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:27.017237  6450 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:27.064039  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:28:27.143639  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79732.8us
W20260709 12:28:27.144958  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.035063 14749 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:27.060266  6454 rpcz_store.cc:269] 0709 12:28:24.817367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.844488 (+ 27121us) service_pool.cc:224] Handling call
0709 12:28:27.004900 (+2160412us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54001e32d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:27.005108 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322514816}
I20260709 12:28:26.952629 14788 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.013568 14834 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.084616  6310 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:27.141690  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3645) took 2241 ms. Trace:
I20260709 12:28:26.906261 14851 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.161190 14851 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.049764 14400 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:27.163558  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:27.164397  6304 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:27.103823 14721 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:27.087997  7317 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.726s	user 0.003s	sys 0.046s
W20260709 12:28:27.169534  7317 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.003s	sys 0.046s
I20260709 12:28:27.091297  6456 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:27.050429 14865 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:27.093434  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68175.4us
I20260709 12:28:27.174753  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:27.067361 14827 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:27.025839 14849 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.186371 14628 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:27.186769 14858 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:28:27.186607  6202 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 10 to 18, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 18 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:27.187695 14849 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:27.070465 13772 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.014s	sys 0.018s
I20260709 12:28:27.078872 14768 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:27.094704  6384 rpcz_store.cc:276] 0709 12:28:24.813680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.947807 (+134127us) service_pool.cc:224] Handling call
0709 12:28:24.948110 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:25.049749 (+101639us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:25.492958 (+443209us) log.cc:844] Serialized 14 byte log entry
0709 12:28:25.579895 (+ 86937us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:25.579955 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:25.580033 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.038445 (+1458412us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b540022f454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:27.038837 (+   392us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.038983 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21335,"prepare.queue_time_us":136809,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":2245,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":1785,"spinlock_wait_cycles":233066368,"thread_start_us":402522,"threads_started":2,"wal-append.queue_time_us":132280,"wal-append.run_cpu_time_us":1174,"wal-append.run_wall_time_us":1024646}
I20260709 12:28:27.173924  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.066570 14645 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.090062 14709 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:27.060392 14627 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.050s
I20260709 12:28:27.130709  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.130779  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.046187 14861 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.058s
I20260709 12:28:27.132023  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.132644  7331 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:28:27.132786  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:27.133492  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.055171 14254 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.049s
I20260709 12:28:27.136129  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.139323  6385 rpcz_store.cc:276] 0709 12:28:25.535614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.536896 (+  1282us) service_pool.cc:224] Handling call
0709 12:28:26.801794 (+1264898us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:26.901405 (+ 99611us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.138684 (+237279us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400065250. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:27.138831 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2885722496}
I20260709 12:28:27.034857 14452 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:27.190205 14867 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:28:27.107640 14862 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:27.152467  6457 rpcz_store.cc:276] 0709 12:28:24.899919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:24.920467 (+ 20548us) service_pool.cc:224] Handling call
0709 12:28:27.005222 (+2084755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.045520 (+ 40298us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.141434 (+ 95914us) spinlock_profiling.cc:243] Waited 2.07 s on lock 0x7b54001e32d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:27.141615 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4762527616}
I20260709 12:28:27.278636  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.169863 14841 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:27.279459  6452 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:27.218770  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 6ea0c2d7699e4559823a28a280a0e713 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:28:27.280747  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:27.281045  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 6ea0c2d7699e4559823a28a280a0e713 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-6ea0c2d7699e4559823a28a280a0e713
I20260709 12:28:27.281394  6301 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:27.155836 14663 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:27.283038 14663 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:27.284019 14663 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.157766  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:27.158599 14749 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.059013 14795 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:27.164752  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:27.173686  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.181548 14865 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.182941 14714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:27.105294 14805 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.183321  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:27.183826 14314 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:27.189000  6202 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 20 to 23, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 23 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:27.192811  6454 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:26.905900 14047 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.194293  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.329958  6384 raft_consensus.cc:2410] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 25 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:28:27.196331  6453 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:27.192410  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:27.149631 14838 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:27.043550 14186 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.243568 14826 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:27.243433 14452 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.247088 14672 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.267269  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.383412  6310 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:27.271565 14843 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:27.272184 14813 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.890s	user 0.013s	sys 0.000s
I20260709 12:28:27.179503 14721 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.272687  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:27.274398 14834 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:27.292856  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.186962 14628 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.315048  6450 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:27.315109  6456 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:27.223060 14768 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:27.322878  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 139650us
I20260709 12:28:27.331552  6202 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:27.334931  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 142647us
I20260709 12:28:27.335228  6304 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
W20260709 12:28:27.341952  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 207390us
I20260709 12:28:27.329799  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:27.369894  6301 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:28:27.373275  6376 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 31 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 20.
I20260709 12:28:27.232002 14645 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.389228  6383 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:27.332110 14827 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:27.341884 14495 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:27.415457  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:27.402365  7288 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 2.158s (SENT)
I20260709 12:28:27.444175  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:27.402779  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 109992us
W20260709 12:28:27.421319  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.446655  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.424511 14452 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.388899 14186 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:27.425348  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.426311 14645 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.427807 14844 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.428501 14874 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.429804  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1208) took 1081 ms. Trace:
I20260709 12:28:27.476539  7317 rpcz_store.cc:276] 0709 12:28:26.348549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.348684 (+   135us) service_pool.cc:224] Handling call
0709 12:28:26.348948 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:26.349139 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:27.169945 (+820806us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.428549 (+258604us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.429485 (+   936us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:27.429532 (+    47us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:27.429581 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.429642 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.429752 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41426,"prepare.queue_time_us":25912,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":86,"raft.queue_time_us":42445,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":182,"thread_start_us":1037064,"threads_started":2}
I20260709 12:28:27.454110  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:27.466383  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.460553  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:27.467731  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:27.374002 14838 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.467377  6385 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:27.393919 14756 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:27.433143 14625 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:27.485574  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:27.445917 14827 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.493045  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:27.445705 14859 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:28:27.465687 14834 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:27.497300  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.512591  6451 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:27.399513 14813 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.485018 14851 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:27.449544 14627 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.467228 14823 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:27.508265  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:27.470427 14452 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.451030 14716 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:28:27.459770  6202 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 11 to 18. New cstate: current_term: 18 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.490424 14672 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:27.505272 14827 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:27.528342 14869 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.437s	user 0.000s	sys 0.001s
I20260709 12:28:27.484561 14874 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.274279 14642 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Committing config change with OpId 18.9: config changed from index -1 to 9, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:28:27.514293 14314 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:28:27.572522  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64394.1us
I20260709 12:28:27.573192 14649 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.578680 14047 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.573333 14452 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.578251  6202 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 18 to 19, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:27.505529 14647 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:28:27.543092 14851 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:27.623148  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 192265us
I20260709 12:28:27.514681 14882 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 12:28:27.627923  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:27.629281  6202 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 15 to 19, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 19 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:27.584412 14826 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:27.649091  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.649608  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 218557us
W20260709 12:28:27.650271  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.651100  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.652145  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.642077  6202 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 14 to 17, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 17 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.590361  6456 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
W20260709 12:28:27.606681  6425 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.792s (SENT) [suppressed 53 similar messages]
I20260709 12:28:27.480347 14844 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.621075  6450 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:27.712167  7285 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:27.573612 14627 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.622366 14709 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:27.717270  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 64244.6us
I20260709 12:28:27.616106 14881 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.635677 14047 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:27.725337  6202 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 15 to 19, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 19 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:27.731510  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 573813us
I20260709 12:28:27.636163 14749 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.644793 14880 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:27.722301 14761 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.854s	user 0.008s	sys 0.005s
I20260709 12:28:27.644420 14645 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:27.627879 14495 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:27.726261 14805 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.708315 14865 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.651340 14826 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.614146 14649 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.640442 14716 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 12:28:27.747236  6202 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 17 to 23, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 23 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:27.762552 14495 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.627972 14851 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:27.724916 14677 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:27.749255  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:27.749579  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.750669  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 100173us
I20260709 12:28:27.730018 14047 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:27.747515 14645 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.743083 14844 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.754648 14826 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:27.730909 14709 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.779001 14047 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:27.778991 14886 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:28:27.767336 14845 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.828s	user 0.014s	sys 0.000s
W20260709 12:28:27.766413 14748 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.495s	user 0.009s	sys 0.006s
I20260709 12:28:27.760561 14435 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:27.762404 14874 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:27.763643 14627 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.824656  6202 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:27.825191  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49139.3us
I20260709 12:28:27.756527 14761 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:27.756769 14749 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.851105  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:27.827526 14677 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:27.760320 14628 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.760867 14892 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:27.831918  6452 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
W20260709 12:28:27.832865  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 125495us
I20260709 12:28:27.767498 14721 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:27.784566 14876 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 12:28:27.823764 14842 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.800s	user 0.008s	sys 0.004s
I20260709 12:28:27.782644 14897 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:27.839155 14890 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:28:27.826642 14882 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:27.873035 14882 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:27.874157 14882 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.875850 14882 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:27.744585 14880 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.880769 14880 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.849562  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3534) took 1311 ms. Trace:
I20260709 12:28:27.881618  6309 rpcz_store.cc:276] 0709 12:28:26.538322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.953374 (+415052us) service_pool.cc:224] Handling call
0709 12:28:26.953829 (+   455us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:26.975780 (+ 21951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:26.975825 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:26.991963 (+ 16138us) log.cc:844] Serialized 26 byte log entry
0709 12:28:27.268840 (+276877us) spinlock_profiling.cc:243] Waited 3.24 ms on lock 0x7b18003df480. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:27.502170 (+233330us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:27.502240 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.502331 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:27.841538 (+339207us) raft_consensus.cc:1685] finished
0709 12:28:27.841566 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.849482 (+  7916us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1603,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":9119,"spinlock_wait_cycles":7454080,"thread_start_us":233083,"threads_started":1,"wal-append.queue_time_us":578753}
W20260709 12:28:27.835872 14833 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.564s	user 0.012s	sys 0.006s
W20260709 12:28:27.859334  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.832906 14838 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:27.849958 14878 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
I20260709 12:28:27.737602 14881 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:27.852596 14874 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:27.867573 14884 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
W20260709 12:28:27.870151  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 403676us
I20260709 12:28:27.847970 14813 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:27.894222  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:27.895967  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44967.7us
W20260709 12:28:27.898296  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:27.898555  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:27.901497  6309 raft_consensus.cc:1240] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:28:27.885039 14833 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.849790 14896 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.868359  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:27.901353  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.828629  6456 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:27.918912  6304 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:27.835386 14845 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.919458  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.920182  6456 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 18.
I20260709 12:28:27.842973 14748 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.861905 14627 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.893133 14452 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.930447 14627 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:27.931474 14894 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:28:27.931296 14677 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:27.933053 14887 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:28:27.934072  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.934502  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.935294  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.935461 14709 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.936434  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.937040  6301 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:27.937340  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.937881  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.938534  6456 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:27.892987 14721 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:28:27.939461  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.892823 14871 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.940056  6456 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:28:27.940476 14871 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.940994  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:27.941360 14671 consensus_meta.cc:287] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.906s	user 0.009s	sys 0.001s
I20260709 12:28:27.941756  6456 raft_consensus.cc:2393] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:27.942363 14649 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.942587  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.943331  6454 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:27.887406 14897 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.944242  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.944967  6454 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:27.945350 14897 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:27.945981 14400 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.754s	user 0.008s	sys 0.001s
I20260709 12:28:27.861819 14749 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.947345 14645 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.948933 14749 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.942667  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.864356 14892 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:27.872480 14790 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 1.399s	user 0.015s	sys 0.000s
I20260709 12:28:27.903107  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.905131  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:27.889634 14761 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:27.953714  6455 raft_consensus.cc:606] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:27.954146  6455 raft_consensus.cc:614] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:27.953873 14761 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:27.943332  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3535) took 1404 ms. Trace:
W20260709 12:28:27.955526 14795 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.001s
I20260709 12:28:27.925132  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:27.956327  6304 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:28:27.950945 14830 consensus_meta.cc:287] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.896s	user 0.012s	sys 0.000s
I20260709 12:28:27.957520  6304 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:27.869889 14843 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:27.955063  6308 rpcz_store.cc:276] 0709 12:28:26.538650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.993981 (+455331us) service_pool.cc:224] Handling call
0709 12:28:26.994540 (+   559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:26.994910 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:26.994935 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:27.001240 (+  6305us) log.cc:844] Serialized 485 byte log entry
0709 12:28:27.376212 (+374972us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:27.376258 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.376337 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:27.943125 (+566788us) raft_consensus.cc:1685] finished
0709 12:28:27.943149 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:27.943270 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":336332,"threads_started":1,"wal-append.queue_time_us":562758}
W20260709 12:28:27.960335  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92103.8us
I20260709 12:28:27.920122 14874 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:27.926479 14885 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:28:27.927753 14881 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:27.929126 14891 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:28:27.932925 14627 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.829965 14883 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.085s
I20260709 12:28:27.952977 14892 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:27.956281  6202 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 16 to 17, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 17 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:28:27.975080  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70432.3us
I20260709 12:28:27.950878  6301 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:27.961395  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 354731us
I20260709 12:28:27.976007  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:27.962389  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:27.977437  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:27.977788  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80316.2us
I20260709 12:28:27.963378  6309 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:27.961422 14761 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:27.923715 14902 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:27.964072 14452 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:27.992998 14874 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.994074 14871 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.996706  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55284 (request call id 38) took 1273 ms. Trace:
I20260709 12:28:27.970134 14709 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:27.999451 14864 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.031s	user 0.007s	sys 0.000s
I20260709 12:28:27.999866 14721 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:28.000744 14768 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.000s
W20260709 12:28:28.015386 14862 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.740s	user 0.011s	sys 0.000s
I20260709 12:28:28.002055 14569 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:28.002556 14761 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:27.978737 14842 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:28.041785 14761 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.041983 14842 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.043082 14862 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:28.044224 14842 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.967507 14843 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.046183 14843 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.042326 14761 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:27.990722  6202 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 13 to 14, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 14 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.051661  6302 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:28.007579 14897 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:28.052803  6302 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3168) took 2396 ms (client timeout 1792 ms). Trace:
I20260709 12:28:28.052361 14897 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.052412  6202 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 15 to 18, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 18 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:28.053189  6302 rpcz_store.cc:269] 0709 12:28:25.656753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.656901 (+   148us) service_pool.cc:224] Handling call
0709 12:28:28.052592 (+2395691us) spinlock_profiling.cc:243] Waited 2.39 s on lock 0x7b54002e7a54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:28.052705 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5491516288}
I20260709 12:28:28.054942  6305 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:28:28.055517 14628 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.055603 14897 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.009685 14748 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:27.982959 14796 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.815s	user 0.011s	sys 0.002s
I20260709 12:28:28.056030 14628 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.056716  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3545) took 1296 ms. Trace:
I20260709 12:28:28.056502 14796 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.020139 14813 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:28.057423 14813 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.011986 14833 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:27.983192 14907 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:28.058964  6307 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:28.059006 14907 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:28.059516 14813 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:28.014967  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.060310  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3538) took 1493 ms. Trace:
I20260709 12:28:28.060204 14907 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.057107  6305 rpcz_store.cc:276] 0709 12:28:26.760571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.390650 (+630079us) service_pool.cc:224] Handling call
0709 12:28:28.054599 (+663949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.055286 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.056532 (+  1246us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54002e7a50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:28.056659 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1526278528}
W20260709 12:28:28.021708  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44281.1us
I20260709 12:28:28.054311  6202 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 20 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 22 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.022821  6455 rpcz_store.cc:276] 0709 12:28:26.722959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.723237 (+   278us) service_pool.cc:224] Handling call
0709 12:28:27.996469 (+1273232us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b54002c8154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:27.996602 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2512978688}
W20260709 12:28:28.026362  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46515.3us
I20260709 12:28:28.026180 14645 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:28.070683  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.070830  6309 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 13. (term mismatch)
I20260709 12:28:28.071314  6309 pending_rounds.cc:85] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 12
I20260709 12:28:28.071688  6309 pending_rounds.cc:107] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:28:28.072317  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.035547 14721 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.036636  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1306) took 1664 ms. Trace:
I20260709 12:28:28.037014 14893 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:28.032233 14709 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:27.946425 14400 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.047434 14768 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:28.006331  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 537335us
I20260709 12:28:28.066679  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.112335  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:28.112542  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.113049  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.060734  6307 rpcz_store.cc:276] 0709 12:28:26.568836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.098244 (+529408us) service_pool.cc:224] Handling call
0709 12:28:27.098637 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.059507 (+960870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.060098 (+   591us) spinlock_profiling.cc:243] Waited 960 ms on lock 0x7b540010f754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:28.060255 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2208160512}
I20260709 12:28:28.069569  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:28.113818  6305 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:28.073320  6301 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:28.074075  6309 raft_consensus.cc:2889] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:28.068322  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:28.076085  6202 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 16 to 17, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 17 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:28.117775  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49512.3us
I20260709 12:28:27.924856 14896 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.083415  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1457) took 1014 ms. Trace:
I20260709 12:28:28.136425  6457 rpcz_store.cc:276] 0709 12:28:27.068671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.619159 (+550488us) service_pool.cc:224] Handling call
0709 12:28:27.632004 (+ 12845us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.847578 (+215574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.847625 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:27.847681 (+    56us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:27.847729 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.083146 (+235417us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54001c1954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:28.083205 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.083346 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495196032}
I20260709 12:28:28.087173  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3543) took 1357 ms. Trace:
I20260709 12:28:28.138294  6310 rpcz_store.cc:276] 0709 12:28:26.729718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.384423 (+654705us) service_pool.cc:224] Handling call
0709 12:28:27.417544 (+ 33121us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:27.431333 (+ 13789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:27.431379 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:27.431436 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:27.431483 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.086891 (+655408us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b54001b3fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:28.086955 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.087108 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31139200}
I20260709 12:28:28.089861 13772 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:28.094934  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:28.097078 14916 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:28:28.097657 14573 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.104068 14796 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:28.115111 14756 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.616s	user 0.002s	sys 0.009s
I20260709 12:28:28.100732 14625 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:28.102522  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.078576  6303 rpcz_store.cc:276] 0709 12:28:26.371866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.460852 (+ 88986us) service_pool.cc:224] Handling call
0709 12:28:26.461282 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:26.462267 (+   985us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:26.462311 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:26.463206 (+   895us) log.cc:844] Serialized 52 byte log entry
0709 12:28:26.716925 (+253719us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:27.193454 (+476529us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.193963 (+   509us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.194149 (+   186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.194248 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:27.998831 (+804583us) spinlock_profiling.cc:243] Waited 9.58 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:28.036413 (+ 37582us) raft_consensus.cc:1685] finished
0709 12:28:28.036436 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.036577 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1002869,"prepare.run_cpu_time_us":136,"prepare.run_wall_time_us":9272,"raft.queue_time_us":98067,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":58946,"spinlock_wait_cycles":22023936,"thread_start_us":570432,"threads_started":2,"wal-append.queue_time_us":333343}
I20260709 12:28:28.104806 14692 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:28.071082 14645 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.107848 14833 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:28.110208 14845 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.155840 14843 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:28:28.118665  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:28.185820  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.122262  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.126250  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.187431  6308 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:28.127755  7335 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:28.188786  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:28.128257  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.131141  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.134825  7318 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:28:28.190834  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:28:28.192452  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.193368  7318 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:28.141371  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.195309  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.195616  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.156032 13772 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:28.155325  6454 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:28.155310  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.160804  6453 pending_rounds.cc:85] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 17
I20260709 12:28:28.203321  6453 pending_rounds.cc:107] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:28:28.203756  6453 raft_consensus.cc:2889] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:28.162786  7329 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:28.168569 14716 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:28:28.165577  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.173341 14625 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:28.166599  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.208841  6383 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:28:28.169631  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.052645 14851 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.106769 14642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:51570 (request call id 12) took 4577 ms. Trace:
I20260709 12:28:28.175004  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.181102  6300 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:28:28.131940 14748 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:28.110432 14400 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.215613 14902 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:28.215170 14642 rpcz_store.cc:276] 0709 12:28:23.504234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:23.505470 (+  1236us) service_pool.cc:224] Handling call
0709 12:28:23.595417 (+ 89947us) log.cc:844] Serialized 485 byte log entry
0709 12:28:24.326046 (+730629us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b5400044320. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa3227b26 00007f6fa322149a 00007f6fa3220714 00007f6fa3200eaa 00007f6fa3207d53 00007f6fa3207cd9 00007f6fa3207c69 00007f6fa3207c31 00007f6fa3206dfd 0000000000388634
0709 12:28:28.079043 (+3752997us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127486,"raft.queue_time_us":230498,"raft.run_cpu_time_us":10822,"raft.run_wall_time_us":506440,"spinlock_wait_cycles":7053440,"thread_start_us":209857,"threads_started":2,"wal-append.queue_time_us":381213,"wal-append.run_cpu_time_us":2981,"wal-append.run_wall_time_us":1148870}
I20260709 12:28:28.229955 14899 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.213902  6275 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:28.182296  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.183260  6202 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 17 to 18, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 18 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:28.184060  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.196624  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.204052 14796 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.147161  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.182804 14692 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.200244  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.201532  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.201908  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.202302  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.202680  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.124935  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.239840  7318 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:28.239176  7319 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:28.240489  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:28.204870  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.242092  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.201028 14813 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:28.242307  6202 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 21 to 23, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 23 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:28.247993  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:28.248555  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:28.200982 14912 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:28:28.206741  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.207721  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.253438  7316 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:28.253525 14851 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.253818  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:28.254462  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.255151  7316 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:28.259473  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:28.196622 14833 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.211594  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:28.182384 14645 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:28.215185  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.276111  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.217806  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.218647  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.231689  6377 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
W20260709 12:28:28.198130  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79550.5us
I20260709 12:28:28.231987  6303 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:28.217636 14896 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.232656  6384 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:28.231489  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.235730  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.279017  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:28.235821  6379 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:28.242151  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.099133  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:28.243018  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.240527  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.248656  6450 raft_consensus.cc:606] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:28.278584  6457 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:28.282655  6450 raft_consensus.cc:614] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:28.250756  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:28.283385  6383 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:28.215476 14435 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.252449  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.283740  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:28.284371  7319 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:28.284719  6457 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:28.210887  6305 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:28.286127  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.287177 14748 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:28.287827  6305 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:28.205530  7335 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:28.288091  6455 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.16-22.16]
I20260709 12:28:28.288849 14435 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.288568  6301 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:28.283954 14907 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:28.292194  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:28.292464 14840 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.292986 14840 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.288453  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:28.261652  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:28.254068  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.222960 14843 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:28.283859  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:28.304339  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:28.304955  7287 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:28.213366 14897 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:28:28.245592 14813 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:28.304592  6202 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 8 to 15, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 15 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:28:28.295421 14827 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.685s	user 0.008s	sys 0.004s
I20260709 12:28:28.309902  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:28.310197  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:28.232570 14826 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.271607 14845 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:28.298858  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:28.287772 14748 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:28.241009 14796 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.325196  6303 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:28.230611 14899 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.294605 14768 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.303750 14838 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.303107 14214 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:28.335894 14756 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:28.384192 14756 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.384624 14756 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:28.336354 14435 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.340123 14840 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.340768 14827 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:28.388496  6458 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:28.387903 14838 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.304126 14896 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:28:28.389612 14896 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:28:28.390009 14896 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:28.327165 14893 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.390934 14214 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:28.391448 14214 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:28.391268 14893 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.392156 14843 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.393105 14435 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.383122  7331 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:28:28.343290 14761 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.345593 14826 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.346308  6454 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:28.308861 14883 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:28.400414 14883 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.400935 14883 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:28.402446 14761 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.403841  6307 raft_consensus.cc:2368] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
W20260709 12:28:28.348815  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50016us
I20260709 12:28:28.405344 14827 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.312785 14716 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:28.407258 14716 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.407377 14893 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.407780 14716 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:28.400307 14435 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.410671 14314 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:28.313711 14813 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:28.411258 14314 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.411374 14883 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.411792 14671 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:28.274762 14851 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.049s
I20260709 12:28:28.413442 14314 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.354048  6457 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:28.355790  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.415071 14838 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.322934 14874 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.415813  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:28.415901 14874 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.416586  7316 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:28.300619 13772 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.370025 14721 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.370841 14709 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.348167 14841 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:28.315918 14842 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:28.397835  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46461.7us
I20260709 12:28:28.351640  6202 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 18 to 19, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 19 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:28.373407 14908 raft_consensus.cc:2764] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:28:28.355084 14897 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.438752  6459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:28.354359 14748 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:28.410307 14435 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.446899  7316 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:28.451607  6459 raft_consensus.cc:606] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:28.465055  6459 raft_consensus.cc:614] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:28.461351  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:28.467957  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57016.1us
I20260709 12:28:28.437352 14883 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.437798 14874 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.441697 14833 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.473047  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.472872  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:28.444254 14841 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.486881 14841 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:28.486570 14892 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.487841 14047 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:28.489687 14892 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.491382 14846 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.450479 13772 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:28.491394  6377 raft_consensus.cc:2368] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:28.494752  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3684) took 1175 ms. Trace:
I20260709 12:28:28.495175  6458 rpcz_store.cc:276] 0709 12:28:27.319850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.730605 (+410755us) service_pool.cc:224] Handling call
0709 12:28:27.745295 (+ 14690us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.388977 (+643682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.494510 (+105533us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400344254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:28.494692 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478720384}
I20260709 12:28:28.497761  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.498966  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.499989  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.500975  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.501539  6381 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 17.
I20260709 12:28:28.479156 14625 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.503063 14628 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.504964 14625 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.390983 14896 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.508131 14896 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:28:28.509680 14796 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.469496 14897 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:28.484762 14926 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:28:28.485280 14254 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:28.462105 14788 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.502315  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.512631  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:28.521422  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.525296  7320 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 16. Current term is 17. Ops: []
I20260709 12:28:28.526033  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:28.527189  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49593.4us
I20260709 12:28:28.515264 14254 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:28.516649 14625 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:28.532925  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:28.524884 14826 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.522713 14716 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.525578 14709 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:28.537431  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:28.532352 14893 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:28:28.531086 14414 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:28:28.533730 14625 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:28.539544 14314 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:28.539567 14935 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:28.645361 14625 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:28.666559 14934 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
W20260709 12:28:28.667575 14933 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.133s	user 0.001s	sys 0.000s
W20260709 12:28:28.668615 14931 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:28:28.676636 14813 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.688107 14813 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.695909 14716 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.630512 14756 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.702639 14897 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.707252 14748 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.712798 14756 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.714886 14645 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.717751 14645 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.710158 14748 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.538898 14826 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.745711 14826 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.535727 14899 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.658020 14714 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43762 (request call id 1113) took 5945 ms. Trace:
I20260709 12:28:28.728080 14841 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.760581 14841 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:28.757596 14714 rpcz_store.cc:276] 0709 12:28:22.599118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.669033 (+ 69915us) service_pool.cc:224] Handling call
0709 12:28:23.124566 (+455533us) ts_tablet_manager.cc:934] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
0709 12:28:24.515044 (+1390478us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b6caa 00007f6fa01b6bd2 00007f6fa821b131
0709 12:28:27.180373 (+2665329us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:28:28.485502 (+1305129us) tablet_metadata.cc:665] Metadata flushed
0709 12:28:28.485630 (+   128us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:28:28.519552 (+ 33922us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:28:28.521024 (+  1472us) tablet_replica.cc:206] Creating consensus instance
0709 12:28:28.525524 (+  4500us) ts_tablet_manager.cc:989] Replica registered.
0709 12:28:28.539330 (+ 13806us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":341,"spinlock_wait_cycles":75329408,"tablet-copy.queue_time_us":224275,"thread_start_us":1591210,"threads_started":2}
I20260709 12:28:28.808422  6309 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:28.809329  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.811280  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:28.531785  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:28.831655  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.539166  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:28.850311  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.859141 14186 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.662380 14254 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.864571 14645 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:28.738348 14935 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:28.526341 14214 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.883858  6453 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 12. (term mismatch)
I20260709 12:28:28.907327  6453 pending_rounds.cc:85] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:28:28.907652  6453 pending_rounds.cc:107] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:28:28.908061  6453 raft_consensus.cc:2889] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:28.887897 14714 tablet_copy_client.cc:806] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:28:28.910265 14714 tablet_copy_client.cc:670] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:28:28.888552 14647 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:28.911082 14647 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:28.912290  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 373006us
I20260709 12:28:28.913017  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.913996  6425 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:28.897853 14896 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:28.902767 14186 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.903790  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:28.904253  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 372382us
I20260709 12:28:28.904706 14414 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:28.901176 14254 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:28.921478  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:28.921809  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:28.923197  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.923951  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:28.927232  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.924870  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3691) took 1432 ms. Trace:
I20260709 12:28:28.924186 14186 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:28.928859  7314 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:28.925468 14414 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:28.951954  7320 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:28.928980 14897 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:28.946135 14414 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:28.947552 14692 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.949255  7335 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 31.
I20260709 12:28:28.958034  6382 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:28.928062  6452 rpcz_store.cc:276] 0709 12:28:27.492378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.867465 (+375087us) service_pool.cc:224] Handling call
0709 12:28:27.867790 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:28.176425 (+308635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:28.176460 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:28.176512 (+    52us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:28.176559 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:28.902371 (+725812us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54002c9054. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:28.907188 (+  4817us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:28.924646 (+ 17458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709373696}
I20260709 12:28:28.913017 14647 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.947077 14645 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.972488  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:28.972913 14214 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.975490  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:28.979009 14897 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:28.971634 14625 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:28.980104  7329 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
W20260709 12:28:28.969080 14860 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.977s	user 0.010s	sys 0.032s
I20260709 12:28:28.982442 14625 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:28.986377 14935 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:28.997233 14935 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:28.999167 14935 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.006700  6379 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:29.010092  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1147) took 2670 ms. Trace:
I20260709 12:28:29.019193  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:29.019783  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46107.6us
I20260709 12:28:29.022022  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:29.013036  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 99391.3us
I20260709 12:28:29.001211 14897 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.026795 14645 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.012142  7332 rpcz_store.cc:276] 0709 12:28:26.339127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.339255 (+   128us) service_pool.cc:224] Handling call
0709 12:28:26.339682 (+   427us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:26.348812 (+  9130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:26.348844 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:26.357965 (+  9121us) log.cc:844] Serialized 78 byte log entry
0709 12:28:26.722190 (+364225us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:26.733235 (+ 11045us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.045483 (+312248us) log.cc:844] Serialized 14 byte log entry
0709 12:28:27.045733 (+   250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:27.045807 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:28.926738 (+1880931us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:29.009317 (+ 82579us) raft_consensus.cc:1685] finished
0709 12:28:29.009360 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:29.009756 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4334,"prepare.queue_time_us":12258,"prepare.run_cpu_time_us":111,"prepare.run_wall_time_us":6618,"raft.queue_time_us":221675,"raft.run_cpu_time_us":518,"raft.run_wall_time_us":31505,"spinlock_wait_cycles":6507904,"thread_start_us":570335,"threads_started":2,"wal-append.queue_time_us":507082}
I20260709 12:28:29.037599  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.047331 14625 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.051375 14625 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.055380  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.061998 14645 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.063030 14645 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.068120  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.053154 14647 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:29.069074  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.077615  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:29.078167  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:29.078399  6454 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:29.077672  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.079967  6453 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:29.085799  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.084641 14645 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.096145 14907 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.074265 14897 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.101111  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:29.101574  6385 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1241) took 1829 ms (client timeout 1865 ms). Trace:
I20260709 12:28:29.102415  6453 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:29.103055  6382 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:29.106671  6452 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:29.109135 13772 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:29.102247  6385 rpcz_store.cc:269] 0709 12:28:27.272224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.272381 (+   157us) service_pool.cc:224] Handling call
0709 12:28:29.100917 (+1828536us) spinlock_profiling.cc:243] Waited 75.6 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:29.101253 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173776256}
I20260709 12:28:29.097680 14625 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.110111 13772 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.110164 14907 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.120649  6385 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:28:29.126331  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.127147  6385 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:29.129320 14892 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:29.138273 14664 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:29.140363 14909 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:28:29.141983  6300 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1294) took 4226 ms (client timeout 2158 ms). Trace:
W20260709 12:28:29.142428  6300 rpcz_store.cc:269] 0709 12:28:24.915307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.145528 (+230221us) service_pool.cc:224] Handling call
0709 12:28:29.141525 (+3995997us) spinlock_profiling.cc:243] Waited 3.03 s on lock 0x7b54000b4854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:29.141743 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6979416320}
I20260709 12:28:29.144941 14935 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:29.141788 14047 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.653s	user 0.002s	sys 0.030s
I20260709 12:28:29.111819  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:29.146631  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.147702 14716 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:29.152592 14716 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:29.153357 14664 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:29.153988 14664 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:29.154237 14716 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.156932 14625 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:29.157419  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 70112.7us
I20260709 12:28:29.157888 14625 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.160029 14692 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:29.147794 13772 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.146037 14935 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.160692 14692 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:29.160751  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:29.180366  7320 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:29.186168 14692 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.189395  6380 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:29.166235 14716 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:29.197489  6454 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:29.207037  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.208057  6380 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:28:29.210894  6377 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:29.211815  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:29.216154  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.217186 14748 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.217949 14748 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.220153 14896 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.220347 14671 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:29.220835 14896 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.223409 14625 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.224047 14748 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.223453 14896 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.225198 14907 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.176030 14897 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.226032  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:29.213996  6425 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:29.227257  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:29.228330 14897 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.228480  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:29.228807  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:29.196736 14909 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:28:29.214083 14826 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:29.206192 14935 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.237690  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.236109 14625 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:29.243217 14748 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:29.243713 14748 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.244606 14748 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:29.246323  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.248112  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.248962  6383 raft_consensus.cc:2368] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:29.249068  6456 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:29.246852  7285 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:29.250178  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.251843  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.253333  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.257288  6383 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:29.258479 14756 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:29.259068 14897 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.259526 14645 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.258540 14918 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:29.247407  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.260892  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.262130  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.155856 14664 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:29.263480  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.264523  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.262866 14896 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:29.264698  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.264701  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:29.265844  6459 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:28:29.266184 14414 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.269474  6381 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
W20260709 12:28:29.235330  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 74755.5us
I20260709 12:28:29.270805  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.271651  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.271682  6378 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:29.272281  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.273720  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:29.273666  6452 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:29.275702  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.289811  6382 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:28:29.279773  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49513.4us
I20260709 12:28:29.278229  6304 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:29.287868  6380 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19.
I20260709 12:28:29.293187  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:29.296190  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.297284  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:29.298952  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.299137  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.298718 14897 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.299645  6456 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:29.301594  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.299791  6304 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:29.302618  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.298789  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:29.306654  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.277053 14756 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:29.310770  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.274678 14647 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.304742  6302 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:29.277866 14918 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:29.314831  6300 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:29.302196 14897 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.315814  6450 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:29.315968  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.316439  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.316815 14845 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:29.279641 14935 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.327507  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.317360  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.317685  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.312868 14756 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:29.318418  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.319620  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.320492  6458 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:29.320781  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92416us
I20260709 12:28:29.321004 14645 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.322183  6457 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:29.330632  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.312860  6383 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:29.323093  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 31.
I20260709 12:28:29.323318  6376 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:29.323956  6385 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:29.315001 14647 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.319711  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.321907  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.333426 14935 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.334534  6309 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:29.334957  6456 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:29.333438  6303 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:29.334700  6381 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:29.337227  6304 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:29.342042 14935 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.344360  6301 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:29.344877  6307 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:29.313444 14896 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:29.291857 14254 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:29.345494  6310 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:29.336768  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:29.351300  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.354135 14254 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.361438  6380 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:28:29.359069  6453 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:29.381268  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.384333  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:29.384769  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:29.385252  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97732.5us
I20260709 12:28:29.385453 14647 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.386229 14813 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.386893 14813 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.389230 14813 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.390719 14845 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:29.391207 14845 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.391538 14845 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:29.385740  6384 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:29.394812  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.397012  6309 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:29.370294 14897 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:29.402972 14897 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.401626  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.404775  6376 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:29.405544 14897 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.369712 14935 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.406454 14935 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.409564 14935 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.401767 14414 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.383176 14826 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.406383  6459 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:29.394045 14645 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.385896 14314 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.473410  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.473966 14314 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.476748 14314 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.427634 14254 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.430261  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.432256 14813 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.478065 14813 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.478645 14826 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.480587 14813 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.435094 14897 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.481047 14826 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.481616 14897 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:29.483970 14897 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.435416 14918 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.438973  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:29.440910 14948 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.258s	user 0.003s	sys 0.000s
I20260709 12:28:29.485460 14918 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:29.485818  6382 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:29.443051 14896 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.445590  6307 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:29.450120 14664 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:28:29.450233  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 82085.8us
W20260709 12:28:29.455732 14943 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.232s	user 0.003s	sys 0.005s
I20260709 12:28:29.398324  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:29.464949  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.470801  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.492110  6303 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:29.494725  6450 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:29.497813  6453 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:29.500813  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.501559  6303 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:29.507992  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.514921  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.516252  6379 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:29.496438 14935 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.520428  6300 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Committing config change with OpId 18.9: config changed from index -1 to 9, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:28:29.522428  6308 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:29.526645 14841 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:29.527567  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 157628us
I20260709 12:28:29.528258  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.528174  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3605) took 1178 ms. Trace:
I20260709 12:28:29.541298  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.541554  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:29.542084  7329 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:29.542905 14862 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:29.542829  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.535492  6379 rpcz_store.cc:276] 0709 12:28:28.349597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:29.094878 (+745281us) service_pool.cc:224] Handling call
0709 12:28:29.152199 (+ 57321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:29.518450 (+366251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:29.527861 (+  9411us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400064854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:29.528080 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827087104}
I20260709 12:28:29.548521  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.501050 14845 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.559257  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.559465  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.563134  6382 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:29.560258  6384 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:29.565773  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.570873  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.573365  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:29.545611 14756 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.575659  6309 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 32: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:29.582295 14756 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:29.583817 14954 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:28:29.587100  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:29.587857  7314 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23.
I20260709 12:28:29.590713  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:29.595150  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.599179  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.604751  7314 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:29.553967 14935 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.615236  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.616062  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.616874  6454 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:28:29.616773  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.617636  6450 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
W20260709 12:28:29.619349 14962 log.cc:927] Time spent T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:28:29.619454  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.620282  6308 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:29.622587 14813 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:29.623142 14813 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:29.623668 14813 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:29.624966  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.627534  6304 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:28:29.632362 14955 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.057s	user 0.002s	sys 0.000s
I20260709 12:28:29.615447  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.633023  6381 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:29.622130 14748 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.635116  6453 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:28:29.639634  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.641011 14748 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.640004  7332 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
W20260709 12:28:29.644640 14953 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:28:29.647454  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.648586  6301 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 32: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:29.649869 14862 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.650642  6303 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:29.650750 14862 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.655941 14748 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:29.656920  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 96230us
I20260709 12:28:29.656651 14748 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.654385 14862 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:29.663190 14952 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:28:29.671425 14748 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.671860 14845 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:29.674346 14748 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.674583  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:29.675495 14826 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:29.675915 14826 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.676100 14897 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:29.676362 14826 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:29.675915  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.649780 14935 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.684725  7329 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:29.682168  6380 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:29.693714  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.695413  7335 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:29.697091  6303 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:29.698128  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:29.700212  7332 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:29.704172  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.704339  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.702529  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.707149  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.708142 14314 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:29.708570 14314 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:29.709272 14314 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:29.709406  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:29.710640  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55574us
I20260709 12:28:29.711580  6301 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:29.714880  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:29.717327 14862 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:29.717873 14862 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:29.695587 14897 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:29.718890 14862 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.719611 14897 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.720139  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.721035  6453 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:29.721203 14841 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:29.722004  6395 tablet_copy_source_session.cc:215] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:28:29.723754  6395 rpcz_store.cc:275] Call kudu.tserver.TabletCopyService.BeginTabletCopySession from 127.0.0.1:46610 (request call id 1237) took 2593 ms. Trace:
I20260709 12:28:29.675215 14748 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:29.724334  6395 rpcz_store.cc:276] 0709 12:28:27.129794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:27.134735 (+  4941us) service_pool.cc:224] Handling call
0709 12:28:29.078082 (+1943347us) spinlock_profiling.cc:243] Waited 60.3 ms on lock 0x7b5400044320. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa321edc0 00007f6fa32b14a5 00007f6fa32b13ee 00007f6fa8247036 00007f6fa8250a3e 00007f6f9fb1427b 00007f6fa82509ba 00007f6fa824bacc 00007f6fa8246c7b 00007f6fa823b8d1
0709 12:28:29.343117 (+265035us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b540023e6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32b13f7 00007f6fa8247036 00007f6fa8250a3e 00007f6f9fb1427b 00007f6fa82509ba 00007f6fa824bacc 00007f6fa8246c7b 00007f6fa823b8d1 00007f6fa72ef4b4 00007f6fa72ef442
0709 12:28:29.721757 (+378640us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b540023e6d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa82471bb 00007f6fa8250a3e 00007f6f9fb1427b 00007f6fa82509ba 00007f6fa824bacc 00007f6fa8246c7b 00007f6fa823b8d1 00007f6fa72ef4b4 00007f6fa72ef442
0709 12:28:29.723533 (+  1776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538802176}
W20260709 12:28:29.725198  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.708523  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.734562  7332 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 32.
I20260709 12:28:29.695693 14671 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:29.738847  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.752947 14748 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.755699 14671 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.756397 14671 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:29.756670 14748 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:29.738154  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:29.759891  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.758476  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:29.764372  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:29.767055  6451 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:29.734252 14935 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:29.768683  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 199407us
I20260709 12:28:29.735265 14692 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.718293 14314 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.770917  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:29.779378  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.757393 14748 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.772503 14214 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:29.788705  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.769799 14935 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:29.802264  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64207.7us
W20260709 12:28:29.804453  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.803362  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.789405 14625 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:29.793279 14935 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:29.804877 14756 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:29.827121 14756 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:29.828526 14756 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.832582  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.801678 14214 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:29.845299  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54978.7us
I20260709 12:28:29.784312 14692 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.810587 14625 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:29.855545 14254 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.856437 14254 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.856904  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.859030 14254 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:29.862514 14960 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.155s	user 0.000s	sys 0.002s
I20260709 12:28:29.868399 14692 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.870075  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.871387 14625 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:29.875313 14961 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.107s	user 0.003s	sys 0.000s
I20260709 12:28:29.850821 14813 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.887251  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:29.888940  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.894848  6380 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:29.916038  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:29.917752  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.901228 14862 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:29.928249  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:29.929492  6300 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:29.931046  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:29.934579  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:29.937536  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59389.1us
I20260709 12:28:29.939783 14813 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.940001 14314 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.942553  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:29.876708 14826 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:29.944360 13772 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:29.945190  6378 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:29.946962  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.950790  6380 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:28:29.951320  6376 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:29.952749  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.957521  6381 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
W20260709 12:28:29.959182  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.961591  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.966060  6382 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:29.964051  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:29.967695  6385 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:29.947770 14813 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:28:29.964748  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 161511us
W20260709 12:28:29.952030 14959 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.296s	user 0.002s	sys 0.000s
I20260709 12:28:29.970459  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.902944 14214 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.950371 14748 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:29.979423  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.962627 14314 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.970105 14813 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:29.983080  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:29.986965  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:29.983883 14862 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 12:28:29.987520  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:29.988583 14826 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:29.989082 13772 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:29.990130  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.990630  7332 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:29.991323  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:30.033859 13772 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:29.992115  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.975035 14692 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:29.992962  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:29.993352  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:29.997483 14664 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:30.000075 14748 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:30.010679  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62663.7us
I20260709 12:28:30.010613 14896 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:30.012703  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:30.019265 14671 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.020628  6457 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:28:29.991994  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:30.036886  7320 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:30.039670  7314 raft_consensus.cc:2368] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25.
I20260709 12:28:30.030931 14813 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:30.040056  7335 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 32: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:30.048580  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:30.053241 14254 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:30.037801 14314 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.062150 14314 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.065898  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:30.064287  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:30.062960  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:30.076067  6384 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:30.066778 14967 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:28:30.078891  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 161498us
I20260709 12:28:30.041317 14748 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.088758  6385 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:30.043179 14896 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.099159  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.100076  6454 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:30.053539 14918 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:28:30.112844  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 48662.5us
I20260709 12:28:30.115072  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:30.106374  6459 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 31
W20260709 12:28:30.116747  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 123500us
I20260709 12:28:30.079203 14254 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.095038 14748 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:30.117712  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:30.131112  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83095.1us
I20260709 12:28:30.098456 14664 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.105486 14645 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:30.119346  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:30.104728 14896 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:30.049273 14692 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:30.119927 14969 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:28:30.079422 14967 consensus_queue.cc:1243] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:28:30.116348 14314 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:30.134557  6451 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:30.118110 14918 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:30.145738  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 158362us
W20260709 12:28:30.147949 14972 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:28:30.145566 14974 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:30.172268 13772 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.171469  6425 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23.
I20260709 12:28:30.172991  6425 leader_election.cc:403] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:28:30.162261 14918 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:30.145388 14756 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:30.151331 14845 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:30.180116  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95115us
I20260709 12:28:30.185288  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.148411 14664 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.189152 14664 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.196079  6451 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:30.160844 14254 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:30.197561  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82581.3us
W20260709 12:28:30.206588  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.208829 14254 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.196535 14863 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:30.210912 14671 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.211407 14863 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:30.212412 14671 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:30.214296 14625 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.216512 14625 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:30.220885  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 122717us
I20260709 12:28:30.225442  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:30.212363 14824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ea0c2d7699e4559823a28a280a0e713. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:30.233984  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 94271.2us
I20260709 12:28:30.210362 14991 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:28:30.233536  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 73836.3us
I20260709 12:28:30.241487  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:30.243996  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124744us
I20260709 12:28:30.238140  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:30.190243 14845 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:30.244228 14973 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.126s	user 0.002s	sys 0.000s
W20260709 12:28:30.273438 14983 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:28:30.276057 14692 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.240796 14991 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:30.278756 14991 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:30.196481 14974 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.283476  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:30.284765 14994 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:30.285261 14994 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.285449 14974 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.285630 14994 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:30.288444  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.249388 14978 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23.
I20260709 12:28:30.289999  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:30.290270 14714 tablet_copy_client.cc:538] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:28:30.292477  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:30.293567 14988 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:30.288736  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:30.294759  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53392.7us
I20260709 12:28:30.295533  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:30.297204  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.212606 14863 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.298902 14863 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:30.286046 14974 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:30.301829 14974 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:30.302748 14985 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:30.303220  6424 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:30.303233  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 119098us
I20260709 12:28:30.304021 13772 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:30.317529  6459 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:28:30.319444  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:30.319589  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:30.320485  7320 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:30.324126  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:30.325424  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:30.303861 14974 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:30.313992 14254 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:30.313885 14692 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:30.318758 14978 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.028s
W20260709 12:28:30.334416  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 100320us
I20260709 12:28:30.336143  6351 leader_election.cc:400] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
I20260709 12:28:30.351415  6351 leader_election.cc:403] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:30.336937 14998 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.352681  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.352883 14998 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.351720 14692 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.354245 14692 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:30.355079 14998 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.337291  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:30.358865 14995 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:30.359752 14985 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:30.359522 14995 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.361951 14995 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.363109  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.363931 14254 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:30.364341 14254 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.364717 14254 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:30.342641 14992 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:30.344545 14896 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.339041  6451 pending_rounds.cc:85] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 14
I20260709 12:28:30.373833  6451 pending_rounds.cc:107] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:28:30.374166  6451 raft_consensus.cc:2889] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:30.368518  7319 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:30.364727  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.343180 14974 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.372157 14978 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:30.377475  6457 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:30.377631 14978 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:30.366580 14664 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19.
W20260709 12:28:30.376022  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.377152 14974 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:30.379333  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51022.9us
I20260709 12:28:30.294775 14988 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:28:30.380935  7335 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:30.335917 14845 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.318265 14314 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:30.338867 14935 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.003s
I20260709 12:28:30.398347  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:30.400712  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85172.1us
I20260709 12:28:30.399510  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.405539  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:30.408583  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:30.411332  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.410193  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.396486 14992 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:28:30.417070  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45394us
W20260709 12:28:30.414601  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47129.1us
I20260709 12:28:30.414793 14897 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:28:30.419894  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:30.421150 14999 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:30.422852  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:30.386740 14813 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.390105 14862 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:28:30.442371  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.451628 14998 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:30.452239 14998 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:30.453148 14998 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.414460 14997 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:30.423305 14991 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.458567 15009 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:30.458989 14997 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:30.459204 15009 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.459735 15009 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:30.460366 14991 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.460098 14997 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.461014 14991 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:30.403965 14935 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.464177 14935 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.440125 14992 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:30.465597 14992 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.400034 14756 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:28:30.438338 14664 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:30.471257 14664 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.471969 14664 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:30.472169 14756 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:30.449869 14625 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:30.458660  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.470373  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.404493 14645 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:28:30.425529 14214 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:30.474685 14714 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:28:30.506624 14314 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.509199 14896 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.476100 14813 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.486898 14999 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:30.474411 14748 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.519199 14254 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.450398 14918 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.510742 14214 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.411334 14978 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:30.540752  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.523263 14994 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.535400 14254 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:30.540966  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 121195us
I20260709 12:28:30.543655  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:30.540915 14214 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:30.553468  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 135423us
I20260709 12:28:30.560689 14692 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.554602 14826 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:30.563651 14826 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.564158 14994 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.567406 14748 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.567828 14978 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:30.568434 14645 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:30.568873 14645 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:30.549180 14897 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:30.539223 14999 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:30.581363 14645 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.501390 15009 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.586779  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:30.588282 15013 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:30.588320 14918 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.591049  6459 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:30.616974  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.617677  6303 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 17.
I20260709 12:28:30.592818 14692 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.595757 14664 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.610996  7319 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:30.612962  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.613904  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.619741  6309 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:30.612958  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 186659us
W20260709 12:28:30.591907  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:30.618198  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 193309us
I20260709 12:28:30.618714  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.620733  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.621501  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.623596  6300 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:30.625012  6301 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:30.625926  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:30.623225  6302 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:30.630893  6308 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:30.647531  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.647580  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.632057  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.647039  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.653857  6304 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:30.653700  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.656147  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:30.657285  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.657850  6300 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:30.661633  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.661840  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.662102  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.664604  6309 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:30.666538  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.654816 14692 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:30.672664  6301 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:30.678602  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.679627  6305 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:30.646922 14845 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.032s
I20260709 12:28:30.658704 15002 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.671325 14692 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:30.700909  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.705546 14996 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:30.706691 15006 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:30.699815  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.714460  6300 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:30.722105  6305 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:30.743026  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.698510 14692 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:30.733842 14826 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:30.775837  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.779528  6305 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:30.697702 15010 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
W20260709 12:28:30.787489  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.790951  7329 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:28:30.786686 14826 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.794915 14826 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.797228 14826 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.798844  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:30.803089 14826 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.803655 14826 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.805722 14826 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:30.807370  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:30.810328 13772 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:30.810775 13772 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:30.811084 13772 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:30.827054  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 91539.3us
I20260709 12:28:30.831773  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:30.833192 15034 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:30.741395 14214 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.838090  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:30.837965  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.853674  6452 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:30.839700 14664 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.839984  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.696864 15002 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.844003 14692 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:30.844995 14824 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:43762 (request call id 1142) took 5207 ms. Trace:
I20260709 12:28:30.792865 15009 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:30.847486  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:30.849429 14826 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.875583  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:30.867584  6274 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.571s (SENT)
I20260709 12:28:30.849743  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:30.852818  7319 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:30.855269  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.877329  6459 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:30.877364  6450 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:30.783938 15006 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:30.856936 14254 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.022s
I20260709 12:28:30.862811  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:30.871232  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:30.876799  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:30.872694  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.876266  6310 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:30.859169  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:30.879190  6451 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:30.881171  6453 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:30.872325  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 174184us
I20260709 12:28:30.849903 14999 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:30.879663  6305 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:30.874790 14997 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:30.884074 14989 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:28:30.884104  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:30.894626  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:30.894666  6309 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:30.884428 14974 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:28:30.892005  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.896529  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:30.898854  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.903865  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:30.905251 14845 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:30.905731 14845 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:30.905926  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:30.906931  6452 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:30.906648 14845 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:30.910830  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:30.874428 15034 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:30.910962 14314 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:30.931367  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:30.884683 15008 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.204s
I20260709 12:28:30.908597 14974 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:31.036021  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.036980  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.037050  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.038583  6384 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
W20260709 12:28:31.039939  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 108693us
I20260709 12:28:31.041080  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:30.926209  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:31.042820  6458 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:31.026844 15017 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 12:28:30.924520 14994 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:31.046540  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.055896  6382 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:30.915318 14625 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:31.056552  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 189005us
W20260709 12:28:31.044962  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 162441us
I20260709 12:28:31.050093  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.051357  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:30.885344 14826 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:31.054231  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:30.924800 14896 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.048175  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.060163  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.067087  6381 raft_consensus.cc:3060] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:31.060807  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.061280  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.047768  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:31.028426 14314 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.059006  6379 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:31.068228  6376 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:31.070766  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:31.071576 15026 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:28:31.071439  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.074576 15019 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:31.079778 14214 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.080227 15037 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:31.081810  6455 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:30.913090 14999 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.047559 14998 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:31.082638  6385 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:31.083266 14692 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.083611  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.085445 15002 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.048619 14974 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:31.086093  6453 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:31.086683  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.066480  6351 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.083289  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.067247  6378 raft_consensus.cc:2376] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:28:31.090884  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:30.911438 14993 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.330s
I20260709 12:28:30.875587 14824 rpcz_store.cc:276] 0709 12:28:25.637362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.637502 (+   140us) service_pool.cc:224] Handling call
0709 12:28:26.149682 (+512180us) ts_tablet_manager.cc:934] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
0709 12:28:30.210582 (+4060900us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:28:30.536487 (+325905us) tablet_metadata.cc:665] Metadata flushed
0709 12:28:30.536650 (+   163us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:28:30.844026 (+307376us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:28:30.844437 (+   411us) tablet_replica.cc:206] Creating consensus instance
0709 12:28:30.844765 (+   328us) ts_tablet_manager.cc:989] Replica registered.
0709 12:28:30.844876 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":285,"tablet-copy.queue_time_us":223085,"thread_start_us":150910,"threads_started":2}
I20260709 12:28:31.095203 13772 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:30.911291 14997 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.097898 15035 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.057874 15008 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:31.098995  6377 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:31.097810  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:30.915066 14863 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.148s
I20260709 12:28:31.102329  6458 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:28:31.097152  6451 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:31.055377 14994 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.114938  6381 raft_consensus.cc:2468] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:31.122231  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:31.096760 14314 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:31.147125  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.101411 15019 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:31.156948  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 90582.5us
I20260709 12:28:31.158589 13772 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:31.151280  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.103643 14998 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.165596  6310 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:30.923565 15006 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.121562 15037 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.173285  6456 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:31.176183  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54086.2us
I20260709 12:28:31.177157  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.155144 14824 tablet_copy_client.cc:806] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:28:31.157744 14994 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:31.102053 14896 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.157405 14978 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:31.186744  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:31.173864 14935 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:31.193356 14645 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:31.194749  6456 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:31.183650 14967 raft_consensus.cc:2764] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:28:31.206178 14999 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:31.207046 14995 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.169010 15019 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:31.170004 14998 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:31.173431 15028 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:31.220619  6301 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
W20260709 12:28:31.234826  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.243516  6274 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.508s (SENT)
I20260709 12:28:31.244226  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:31.244681  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67632.3us
I20260709 12:28:31.245788  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.217118 14935 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:31.251952  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:31.252311  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.255936  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:31.228978 14824 tablet_copy_client.cc:670] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:28:31.258606  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:31.232122 14214 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.262244  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:31.260624  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:31.243146 14992 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:31.270231  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124331us
I20260709 12:28:31.275177 14214 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.276607  6305 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:31.250952 14845 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:31.288329  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44869.6us
W20260709 12:28:31.289022  6274 leader_election.cc:336] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.519s (SENT)
I20260709 12:28:31.289413  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.258941 14935 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:31.286971  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 246017us
I20260709 12:28:31.242653 14999 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:28:31.157467 15035 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.301503  6458 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:31.253679 14918 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:28:31.231002 14995 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.307371  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.319213  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:31.320037  6301 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:31.317965  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:31.321723  6308 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:31.326165  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:31.322917 15039 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:31.331228 15050 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:31.333621 14988 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:31.334040 14988 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.334851 14988 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.336076 14988 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:31.339906 14214 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.340497 15027 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.341032 15027 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.343612 15028 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:31.344501 14845 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:31.344908 14845 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.345269 14845 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:31.345167 15028 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:31.290163 15043 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:31.346587 15040 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.346845 14841 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:31.347211 14841 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:31.348321 14841 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.349543 14841 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:31.322688 15049 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:31.339942  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94294.4us
I20260709 12:28:31.331387 14254 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:31.351243 15049 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.157503 15034 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.191802 15037 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:31.394016 15044 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:28:31.394791 15051 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:31.395414 15051 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.396356 15051 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:31.397171  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.397722 15051 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:31.401183 15015 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:31.401654 15015 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:31.402381 15031 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:31.402590 15015 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.403767 14997 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:31.404614  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:31.405408  6454 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:31.409130 15023 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:31.409533 15023 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.347234 15040 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.410754  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:31.411391  6458 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:31.290460 14992 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:31.413810  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.415711  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.403079 15031 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.426281  6459 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:28:31.411443 15049 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:31.320956 15011 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.106s
I20260709 12:28:31.403790 15015 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:31.456650  6302 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:31.457631  6454 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:31.413090 15040 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.426443 15002 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.482527 14756 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:28:31.473665  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.447865 14826 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:31.493710  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.331282 15039 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.491355 14756 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:31.499789 15047 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:28:31.500275  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:31.468860  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 284369us
I20260709 12:28:31.516983  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:31.519656  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 104044us
W20260709 12:28:31.491755  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 198110us
W20260709 12:28:31.525635  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.521600  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:31.536800  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.537233 14214 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:31.537724  6381 raft_consensus.cc:2410] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:28:31.537976 14214 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:31.538345 15043 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:31.539362 14214 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.544934  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:31.546993  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.410365 15023 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.564148  6377 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:31.565778  6376 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:31.531011 15040 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:31.519160 15027 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.490953 15002 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.573032  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.543009 14967 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:31.579667  6381 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:28:31.580544  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 273249us
W20260709 12:28:31.582597  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 293583us
W20260709 12:28:31.583509  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64279us
I20260709 12:28:31.590301  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:31.585880  6274 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.528s (SENT)
I20260709 12:28:31.593776  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.574107 15023 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:31.596290  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:31.572490 15040 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.603184  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.578529 14967 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.560166 15009 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:28:31.604202 15052 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:31.605144  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.606472  6425 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:31.620360 15053 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:28:31.620863 14978 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:31.609212 14967 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:31.599038 15058 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.624617  7314 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:31.625645  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:31.628798  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:31.626405  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 110225us
I20260709 12:28:31.605914 15040 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:31.609061 14997 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:31.601442 14826 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:31.543551 15031 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.613437 14254 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:31.616861 15013 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:31.640395  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54571.8us
I20260709 12:28:31.617072 15052 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:31.638376  6425 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:31.644202  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.623831 14995 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:31.646370  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.624028 14974 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.659181  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:31.665975  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:31.666373 15061 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:28:31.667863 15019 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.669791  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:28:31.676364  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.631327 15066 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:31.636394 15035 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.637095 14997 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.638330 15050 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:31.687129 15050 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:31.687398 15039 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:31.670790  6274 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.864s (SENT)
I20260709 12:28:31.679630  6379 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:31.681144  6458 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:31.633556 14978 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:28:31.682255 15019 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:31.682976 14998 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.682899 14845 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.612903 14756 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.693276  6454 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 17. Current term is 18. Ops: []
I20260709 12:28:31.649457 15052 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:31.651546 14826 raft_consensus.cc:2804] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:31.695714  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49767.7us
I20260709 12:28:31.700757  6377 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:31.697023 15027 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.007s
I20260709 12:28:31.704909 15002 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.706676 15064 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:31.661991 14897 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.046s
W20260709 12:28:31.707932 14625 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.000s
I20260709 12:28:31.708149 14998 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:31.709713 14991 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:31.710764  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:31.711007 14314 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.010s
W20260709 12:28:31.712634  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58327.7us
W20260709 12:28:31.741797  6274 leader_election.cc:336] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.626s (SENT)
I20260709 12:28:31.742301  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.678401 14995 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:28:31.714859  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 55766.9us
I20260709 12:28:31.669862 14992 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:31.722461  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:31.689333 14997 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:31.741245 14994 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
W20260709 12:28:31.737502  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 67777.4us
W20260709 12:28:31.736275  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.746697  6457 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:31.750854  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:31.635280 15058 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.735373 14756 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:31.755661  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:31.740908 14991 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:31.700939 14918 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:28:31.766667  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:31.744961 14625 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:31.741375 14826 raft_consensus.cc:697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:31.748188 14995 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:31.785312  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43547.9us
I20260709 12:28:31.785468  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:31.790933  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:31.806679  6274 leader_election.cc:336] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.137s (SENT)
W20260709 12:28:31.792323  6305 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.504s	user 0.015s	sys 0.008s
I20260709 12:28:31.801468  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:31.807710  6305 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:31.808037  6456 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
W20260709 12:28:31.812551  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.758298 14991 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:31.744264 14314 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:31.802464 15068 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:28:31.759807 14998 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:31.822188  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:31.803869 14978 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:31.726434 14967 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:31.825502  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.834986 14826 consensus_queue.cc:237] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.839277 15083 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.839970 15083 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.824756 14625 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:31.843631 14625 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:31.844030 14625 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:31.847476 15078 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:31.848867 15019 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:31.849789 15019 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:31.850121 15019 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.851030 15019 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.852284 15019 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:31.827337  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:31.853634 15077 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:28:31.854220 14918 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:31.855079 14692 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:31.855962  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.855707 14692 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.811565 14995 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:31.735443 15013 consensus_queue.cc:1175] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 27 status { last_received { term: 27 index: 20 } last_committed_idx: 20 last_received_current_leader { term: 27 index: 20 } }
I20260709 12:28:31.831554  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:31.851432 14845 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.867337  6458 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 31: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:31.858006  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51350us
I20260709 12:28:31.759128 14994 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.823412  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:31.866415  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1497) took 1384 ms. Trace:
W20260709 12:28:31.869027  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45714us
I20260709 12:28:31.870162  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:31.868597  7335 rpcz_store.cc:276] 0709 12:28:30.479269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:30.479421 (+   152us) service_pool.cc:224] Handling call
0709 12:28:30.512018 (+ 32597us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:30.521417 (+  9399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:30.521463 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:30.529346 (+  7883us) log.cc:844] Serialized 52 byte log entry
0709 12:28:30.648603 (+119257us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:30.698526 (+ 49923us) log.cc:844] Serialized 14 byte log entry
0709 12:28:30.698841 (+   315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:30.698943 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:31.548684 (+849741us) spinlock_profiling.cc:243] Waited 6.39 ms on lock 0x7b180035e360. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1
0709 12:28:31.801372 (+252688us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b180035e360. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:31.852451 (+ 51079us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b180035e360. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:31.857690 (+  5239us) raft_consensus.cc:1685] finished
0709 12:28:31.857720 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:31.863524 (+  5804us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1015,"prepare.queue_time_us":92055,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":12264,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":9754,"spinlock_wait_cycles":53102592,"thread_start_us":130887,"threads_started":2,"wal-append.queue_time_us":177433}
I20260709 12:28:31.873952 14994 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:31.874946  6451 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:31.875322 14935 consensus_meta.cc:287] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.559s	user 0.010s	sys 0.000s
I20260709 12:28:31.875746 14935 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.879248  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.880640 15041 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:31.881137 15041 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.881541 15041 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:31.879271  6377 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:31.882457  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.885040  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.885939  6383 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:31.887171  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:31.871941 14998 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.888370 14998 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.890575  7329 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:31.892045 14967 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.893348  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.894524 15037 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:31.894953 14994 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:31.895454 14994 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:31.895861 14994 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:31.896852 14664 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:31.897300 14664 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.867741 15083 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.887972  6458 raft_consensus.cc:2393] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:31.900537  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:31.895875  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:31.903322  6274 leader_election.cc:336] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.727s (SENT)
I20260709 12:28:31.904000  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.906462  6425 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:31.916254  6381 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:31.877903 14978 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:31.919364 14254 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:31.928936  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.896847 14692 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.929219  6351 leader_election.cc:400] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25.
I20260709 12:28:31.934170  6351 leader_election.cc:403] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:31.936789 15007 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:31.937307 15007 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:31.938380 15007 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.940352 15007 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:31.898134 14664 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.947930 14664 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:31.951947  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:31.953981 15056 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25.
I20260709 12:28:31.954749 15083 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:31.954909  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51624.6us
W20260709 12:28:31.955864  6274 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.760s (SENT)
I20260709 12:28:31.956295  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:31.957247 14254 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:31.976110 14896 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:31.976516  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.976893 14896 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:31.977432 14896 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:31.977963 15086 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:31.979141  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:31.978808 15086 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:31.980069  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.980914  6379 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:31.981194 15086 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:31.981904  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.982252 15058 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:31.982939  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.983078  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.983174  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:31.984234  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:31.984995  6458 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:28:31.985414  6458 pending_rounds.cc:85] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 15
I20260709 12:28:31.985679  6458 pending_rounds.cc:107] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:28:31.985924  6458 raft_consensus.cc:2889] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:31.934952 14978 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:31.926915 15049 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 12:28:32.002923 14978 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.001636  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:32.004828  7314 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:31.994599 14748 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:32.006965 14748 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.007620  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:31.914796 14967 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:28:32.008510  7329 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.008823 14967 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:28:32.009258 14967 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:32.009938 14748 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.010300 14967 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.011659 14967 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:32.012063 14998 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:31.995005 15017 log.cc:927] Time spent T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:28:32.021456  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.084008 14896 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.027062 15086 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.085635 15062 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:32.085530 15086 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.086156 15062 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:32.086681 15062 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:32.091912 15088 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:32.092665 15088 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:32.093173 15088 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:32.097453 15062 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:32.099612  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78484.9us
W20260709 12:28:32.019623 14988 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.683s	user 0.016s	sys 0.014s
I20260709 12:28:32.015131 14314 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.123579  7314 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:28:32.121822  6302 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.128890  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.142980  6304 raft_consensus.cc:2368] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22.
I20260709 12:28:32.130617  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.143618  6376 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:32.132992 15040 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.133783 14994 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:32.134984 15015 consensus_meta.cc:287] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.667s	user 0.010s	sys 0.001s
I20260709 12:28:32.061115 14998 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:32.151705  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.145939  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:32.154677  6378 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:32.155664  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.161058  6301 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:28:32.165182  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3735) took 1063 ms. Trace:
I20260709 12:28:32.165601  6451 rpcz_store.cc:276] 0709 12:28:31.102108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.156893 (+ 54785us) service_pool.cc:224] Handling call
0709 12:28:31.157246 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:31.875472 (+718226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:32.164946 (+289474us) spinlock_profiling.cc:243] Waited 717 ms on lock 0x7b54002c7c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:32.165118 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1649953024}
W20260709 12:28:32.169549  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 260719us
I20260709 12:28:32.169709  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:32.170966  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.170475  7314 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.154058 14998 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:32.179738  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.170686 14254 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:32.184788 14845 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.185391 14845 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.188750 14756 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:32.189821 14756 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:32.141762 14991 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.207305  6377 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:32.212383  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:32.197642  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.213542  6304 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.167265 15010 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:32.109767  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:32.221009  6450 raft_consensus.cc:606] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:32.221388  6450 raft_consensus.cc:614] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:32.247802  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:32.248744  7335 raft_consensus.cc:2410] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 24 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
W20260709 12:28:32.250219  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.251978  6451 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:32.252511  6459 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:32.247870  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:32.254609  7332 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:32.159927 14625 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.259063  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.261500  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.259783  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.262162  6383 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:32.267844  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.266116  6451 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:32.190264 14756 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:32.270435  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.272202  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:32.273231  6451 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:32.275839  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.224064 15015 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:32.271178  6304 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:32.276897  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.273502  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:32.280774  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.283476  7332 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:32.258752 14748 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.288784  6457 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:32.288762  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:32.261135 14826 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.292862  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:32.298149  6453 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:32.301994  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.300344  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:32.303869  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:32.292189 15052 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.555s	user 0.005s	sys 0.005s
I20260709 12:28:32.273661 14988 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:28:32.271226 15041 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.315351  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:32.321710  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.322510  6385 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:32.322908  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.323650  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.323838  7287 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:32.324319  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.324501  6385 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.323529  6376 raft_consensus.cc:2376] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22. Re-sending same reply.
I20260709 12:28:32.301616 14845 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:32.338584  6455 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:32.313088 14824 tablet_copy_client.cc:538] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:28:32.301882 14978 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:32.281860 14995 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.335871  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.340066  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:32.326659  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:32.312469 15037 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.343119  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.343801  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.344734  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:32.342314  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.342486  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:32.351815  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1339) took 1107 ms. Trace:
I20260709 12:28:32.296607 14748 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.360931  6376 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:32.362277  6384 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:32.296911 14826 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.331852 14862 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:28:32.370596  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.342219 15045 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:32.359572  6307 rpcz_store.cc:276] 0709 12:28:31.230543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.230675 (+   132us) service_pool.cc:224] Handling call
0709 12:28:31.257915 (+ 27240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:31.297822 (+ 39907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:31.297855 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:31.308978 (+ 11123us) log.cc:844] Serialized 26 byte log entry
0709 12:28:31.564299 (+255321us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:31.597787 (+ 33488us) log.cc:844] Serialized 14 byte log entry
0709 12:28:31.610033 (+ 12246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:31.610140 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:32.185437 (+575297us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b180042c460. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:32.328882 (+143445us) raft_consensus.cc:1685] finished
0709 12:28:32.337410 (+  8528us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:32.337637 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44733,"prepare.queue_time_us":37142,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":2879,"raft.queue_time_us":41578,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":18926,"spinlock_wait_cycles":12931328,"thread_start_us":175777,"threads_started":1,"wal-append.queue_time_us":244024}
I20260709 12:28:32.399475  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.401484  6384 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:32.354593 15034 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:32.403214  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.407051  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.359658 14254 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:32.392544 15092 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:28:32.320494 15052 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.426296  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.430867  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.320314 14756 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.414865  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:32.354455 15006 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:28:32.375944  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.331087 15039 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:28:32.397333 15040 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.297782 14998 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.454289 14998 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:32.454900  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.414610 15042 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.187s
I20260709 12:28:32.457335  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1333) took 1228 ms. Trace:
I20260709 12:28:32.457523 14756 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:32.369072 15037 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:32.457713  6310 rpcz_store.cc:276] 0709 12:28:31.228721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.228873 (+   152us) service_pool.cc:224] Handling call
0709 12:28:31.229551 (+   678us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:31.258058 (+ 28507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:31.258093 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:31.902173 (+644080us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b2c00033c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:31.902591 (+   418us) log.cc:844] Serialized 26 byte log entry
0709 12:28:32.162921 (+260330us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:32.162964 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:32.163272 (+   308us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:32.457051 (+293779us) raft_consensus.cc:1685] finished
0709 12:28:32.457073 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:32.457277 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11960,"raft.queue_time_us":2232,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":4496,"spinlock_wait_cycles":11555456,"thread_start_us":259911,"threads_started":1,"wal-append.queue_time_us":392134}
I20260709 12:28:32.460165 15037 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:32.462010  6310 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:32.461913 15088 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.463074  6459 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:32.424830  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:32.457928  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:32.465088  6425 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:32.462958  6385 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.433039 15094 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.466185 15031 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:32.416069  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.392375 14664 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
W20260709 12:28:32.435036 15085 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.267s	user 0.000s	sys 0.001s
I20260709 12:28:32.371775 15062 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:32.475319 15098 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:28:32.466450 15094 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.477425 15094 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.477603 15041 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:32.478230  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.473541 14671 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.005s
I20260709 12:28:32.466718 15049 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:32.479024 15049 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:32.479374 15049 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:32.479533 15056 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:32.479933 15056 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:32.480129 14826 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.480791 15056 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.424367 14254 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:32.444770 15019 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.000s
I20260709 12:28:32.448559 15006 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:32.482754 15006 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:32.483726 14254 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:32.483616 15006 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.484591 15006 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:32.449029 14625 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.416002 15086 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.449533 14991 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.449891 15030 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:32.487883  6381 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:32.437382  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.453429 15083 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:32.493546  6452 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:32.464087  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:32.415848 14988 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.495421  6456 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:32.495730 14988 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:32.463233 15093 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:28:32.443244  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.488587  6309 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:32.474234 15088 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.502753  7332 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
W20260709 12:28:32.504113  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 158141us
I20260709 12:28:32.449440 14748 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.471284 14995 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.513368  6379 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:32.512010  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.515182  6380 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
W20260709 12:28:32.523706 15007 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.006s
W20260709 12:28:32.511197 15099 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:28:32.527168  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.528658  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.529508  6380 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:32.529743  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.530710  6380 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:28:32.531843 15052 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.533880 14988 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:32.530911 14862 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:32.509752 15062 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
I20260709 12:28:32.538105 14862 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.539083 15083 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.539657 15083 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.541847 15083 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:32.542639 14824 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:28:32.544392  6382 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 32.
I20260709 12:28:32.552062 14935 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:32.553010 15049 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:32.555832 14997 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.785s	user 0.010s	sys 0.002s
I20260709 12:28:32.556207 15045 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:32.558353 15002 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:32.559159 15042 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:28:32.559926  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 145474us
W20260709 12:28:32.558386 15066 consensus_meta.cc:287] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.856s	user 0.011s	sys 0.000s
I20260709 12:28:32.566885  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.589293  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.590148  6377 raft_consensus.cc:2376] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25. Re-sending same reply.
I20260709 12:28:32.591336  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.592833  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.593573  6377 raft_consensus.cc:2376] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 19. Re-sending same reply.
I20260709 12:28:32.567273  6380 raft_consensus.cc:1240] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:28:32.595646  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.596421  6380 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:32.568879  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:32.569484  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.571084 15056 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:32.598886  6378 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:32.572978 14974 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.576277  6376 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:32.507128 14978 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:32.600927  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.601163 14978 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.602046  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.603305  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.560551 15039 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:32.608130  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:32.610432  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.617194  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:32.617895  6458 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:32.620532  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.623039  6377 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:32.627374  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.630931  6383 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:32.634630  6376 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:32.636070 15049 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:32.649704  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.671589  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.584476 15042 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:32.680008  6378 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:32.580229 14989 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:28:32.612679 15066 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:32.683099  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.661147  6380 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:32.688095  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:32.686035  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:32.646415  6274 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.566s (SENT)
I20260709 12:28:32.696393  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:32.697165  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:32.691901  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.698247  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51923.6us
I20260709 12:28:32.698621  6300 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:32.696321  6455 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:32.702050  6378 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:32.615242 15040 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.076s
I20260709 12:28:32.615862 14845 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.075s
I20260709 12:28:32.711784  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.713347  6451 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:32.713614  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.713935  7314 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:32.703163  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.716161  6303 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:32.703289  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.704101  6456 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:32.718596  6309 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:32.606986 15034 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
I20260709 12:28:32.721000  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:32.725396  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.613054 15002 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.697479  6453 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:32.587525 14997 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.717044  6307 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:32.734957  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:32.738760  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.722432  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.754238  6378 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:32.723371  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:32.718641  6458 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:32.717065  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:32.704101  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.618584 15030 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:28:32.725991  6452 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:32.756686  6377 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:32.727041  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.727355  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:32.756990  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:32.757503  7314 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:32.756268  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.758975  6303 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:32.694730 14692 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:32.761365  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:32.733227  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.705013  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.763948  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.764662  6380 raft_consensus.cc:2376] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22. Re-sending same reply.
I20260709 12:28:32.765658  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.767041  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:32.768021  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:32.768316  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.769315  6380 raft_consensus.cc:2393] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:32.759910  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.771238  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:32.772408  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.773139  6305 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:32.770534  7314 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:32.731865 15009 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:32.770283  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:32.778005  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54751.3us
I20260709 12:28:32.779170  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.779624  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.780269  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.783057  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.802697  6377 raft_consensus.cc:2410] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:32.733101 15011 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:32.781049  6454 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:32.726928 14967 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
W20260709 12:28:32.806569  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 95106.2us
I20260709 12:28:32.808216  6305 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:32.812124  6300 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:28:32.814731  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.815426  6305 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:32.810024  6303 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:32.810739  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.816478  6377 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:32.812084  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.811002  6310 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:32.811000  7314 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:28:32.818897  6380 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:32.819633  6351 leader_election.cc:400] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22.
I20260709 12:28:32.819984  6351 leader_election.cc:403] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:28:32.820940  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:32.826604  6377 raft_consensus.cc:2393] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:32.827582  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:32.828094  6377 raft_consensus.cc:1217] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Deduplicated request from leader. Original: 23.18->[24.19-27.20]   Dedup: 24.19->[27.20-27.20]
I20260709 12:28:32.828907  6377 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: [24.19-27.20]
I20260709 12:28:32.830170  6377 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:32.832309  6380 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:28:32.833669  6380 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:32.834105  6380 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:32.835045  6380 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:32.836316  6380 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:32.828187  6307 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:32.843300  7335 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:32.809000 14988 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:32.812889 15034 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:32.712929 15045 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:32.844933  6274 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.538s (SENT) [suppressed 35 similar messages]
I20260709 12:28:32.848508  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:32.821336 14756 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:32.775745 14692 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.861363  6380 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:28:32.862589  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46669.6us
W20260709 12:28:32.865283  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.860728  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet e44e2bcbe27f4a9192cc9acd16b966ad: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:32.863829  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:32.849264 15034 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:32.859478 15056 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.880121  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.884472  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:32.869071 14988 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22.
I20260709 12:28:32.818398 15011 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.771621 15042 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.864475 14756 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:32.927888  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47925.7us
I20260709 12:28:32.877672 15058 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:28:32.895839 15110 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 12:28:32.954241  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 101189us
I20260709 12:28:32.925202  6304 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:32.961135  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.963743  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.965910  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:32.967783  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:32.916266 15109 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:28:32.915647 15107 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:28:32.915068 15106 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:28:32.953102  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.896245 14998 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:28:32.897516 15112 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:32.903921 14692 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.907616 15011 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:32.911721 15083 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:32.916637 15010 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:32.894972 15056 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:32.947826 14988 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:32.951198 14978 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:32.920931 15015 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.076s
I20260709 12:28:33.003115  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3358) took 1024 ms. Trace:
I20260709 12:28:32.983498  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:32.994911 15058 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:33.003894 15083 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:33.010265 14998 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:33.010466 14692 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
W20260709 12:28:33.010597 15111 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:28:33.011333 15010 consensus_queue.cc:1059] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 12:28:33.011936 15011 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:32.905112 15034 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.012466 15118 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:33.003571 15112 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:33.005280  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:33.020522  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 133226us
I20260709 12:28:33.020963  6381 rpcz_store.cc:276] 0709 12:28:31.978947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:31.982822 (+  3875us) service_pool.cc:224] Handling call
0709 12:28:33.002889 (+1020067us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400109ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.003047 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159253248}
I20260709 12:28:33.023885 15023 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:33.031591  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:33.078917  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47461.3us
W20260709 12:28:33.079093  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 322189us
W20260709 12:28:33.073875 15106 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:28:33.083375 15116 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:33.083596 15015 raft_consensus.cc:3055] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:33.077575 15010 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:33.022439 14978 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:33.088518  6254 auto_rebalancer.cc:629] Failed to schedule move for tablet d53ab19d424b4a85af2153ef24520678: Illegal state: Replica 4268023ecea149cd83cba77db53c3100 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:33.091660  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:33.073143 15023 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.089314  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:33.114404  6309 raft_consensus.cc:2410] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:33.113260  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:33.123806  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:33.127960  6300 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:33.084051 15105 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:33.127121 14993 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:28:33.135720 14978 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.087329 15015 raft_consensus.cc:740] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:33.137336 15015 consensus_queue.cc:260] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.129923 15013 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.139593  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.141036 15058 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:33.141485 15058 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.141830 15058 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:33.142328 14756 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.144141 15056 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.145205 15002 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.141809 15015 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:33.087637 15034 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:33.123446 15023 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:33.091379 14997 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.139673 15013 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.158377 15105 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:33.209360  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 243503us
I20260709 12:28:33.174347 15002 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.212375  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:33.217322  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.218340  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:33.222537  6300 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:33.091553 15116 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.227885  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.213825 15105 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:33.098740 15083 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.240875  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:33.150800 14998 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.257081  6459 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:33.257622  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:33.262213  6455 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:33.154901 15112 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.257064  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.258373  6458 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:33.241789 15010 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:33.265748  6309 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
W20260709 12:28:33.266320  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.273116  6455 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:33.273348  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:33.224380 15031 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:28:33.274153  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:33.254211  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:33.268110  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:33.269750  6457 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:33.223006 15002 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.270550  6453 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:33.276762  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:33.270217  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.278371  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.276481  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:33.278949 13772 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:33.283403 13772 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:33.283768 13772 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:33.277033 15121 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:33.275817  6459 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
W20260709 12:28:33.284758  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.282593 15066 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:33.282243 15083 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:33.277060  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.303984  6452 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:28:33.305117  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:33.275981 15010 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.274077 14862 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:33.307862  6459 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:33.281105 15056 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:33.309106 15031 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
I20260709 12:28:33.275521 15122 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:33.302718 15066 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:33.326335  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:33.328199  6459 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:33.329833 13772 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.306427 14845 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:33.309648 14997 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:33.335511 14862 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:33.336127 14967 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:33.307993 15120 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:33.351111 15002 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.330993 15066 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:33.352385  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:33.371390  6300 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:33.315301 15010 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:33.359125 15013 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.308288 14664 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:33.327543 15113 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.042s
W20260709 12:28:33.366505  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 1.19279e+06us
I20260709 12:28:33.329077 15122 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:33.359150 15116 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.372942  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:33.364015 14978 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:28:33.347110  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70130.3us
I20260709 12:28:33.381246  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:33.369544  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:33.381915  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 310848us
I20260709 12:28:33.379732 15122 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:33.405211  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.411896  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.443841  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:33.377805 14664 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:33.447573 13772 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:33.448905  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.451853  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.453231 15015 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:33.453478  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.454061 15015 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.454288 15041 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:33.454854 15041 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:33.454573 15015 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.455982 15041 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.457567 15041 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:33.458032  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.469507  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3660) took 1029 ms. Trace:
I20260709 12:28:33.458959  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.458971  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.469956  6383 rpcz_store.cc:276] 0709 12:28:32.440321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.440510 (+   189us) service_pool.cc:224] Handling call
0709 12:28:33.469365 (+1028855us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54001950d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.469456 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":403394944}
I20260709 12:28:33.470371  7335 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:33.459816 14664 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.460457 15042 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:33.453230  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.464314 14997 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.468864  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3746) took 1169 ms. Trace:
I20260709 12:28:33.468525 15058 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.482764 15058 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.456846 15015 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.382726 15035 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:33.483634 15035 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.484027 15035 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:33.480516 14997 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.487879 14997 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:33.488394 14997 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:33.381367 14978 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:33.489389 14978 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:33.494731  7320 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:33.479205  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.479254  6451 rpcz_store.cc:276] 0709 12:28:32.299828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.300040 (+   212us) service_pool.cc:224] Handling call
0709 12:28:33.468679 (+1168639us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b54000eb854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.468777 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943605632}
I20260709 12:28:33.478811  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:33.480405 15120 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:33.511993 14664 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:33.513661 14997 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.514279 14997 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.514868 14214 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:33.510566  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:33.518067  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.519883  6302 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
W20260709 12:28:33.523236  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 269405us
I20260709 12:28:33.477401 15105 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.528967  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:33.529157  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 224121us
I20260709 12:28:33.529515  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:33.540728  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 322417us
I20260709 12:28:33.518922 15004 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:33.523056 15116 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:33.530469  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:33.544955 15008 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:33.550570  6381 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:33.541765  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:33.543771 13772 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:33.508569 15023 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.543807  6377 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:33.551930  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.552702  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:33.553515  6377 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:33.555084 13772 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:33.546144  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.546865  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.557044  6385 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:33.558123  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.558746  6385 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:33.559650  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:33.523185 15120 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:33.548374  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:33.550328  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:33.576007  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.578919 13772 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.580031  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:33.565016  6383 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:33.542788 15015 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:33.542769 15058 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.585000  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.591770  6384 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 32: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:33.590564  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:33.592722  6383 raft_consensus.cc:2376] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25. Re-sending same reply.
I20260709 12:28:33.592203  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.595448 15112 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:33.596495  6378 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:33.564043 15120 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.597302 15120 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:33.598317 15023 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.598935 15023 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:33.599356 15023 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.603168 15023 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:33.607198 15108 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.734s	user 0.002s	sys 0.003s
I20260709 12:28:33.611994  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:33.612733  6381 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:33.616518  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.616333 14978 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.617472  6451 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:33.618662 15066 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:33.619858  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3599) took 1369 ms. Trace:
I20260709 12:28:33.620119  6308 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:33.620275  6304 rpcz_store.cc:276] 0709 12:28:32.249938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.288277 (+ 38339us) service_pool.cc:224] Handling call
0709 12:28:33.619632 (+1331355us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b54000aa854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.619774 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1431403392}
I20260709 12:28:33.620893 15066 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:33.577425 15121 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:33.578536 14988 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:33.621531 15066 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:33.622092 14988 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:33.623167 14988 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.625828  6451 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:33.628368 15105 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.629786 15058 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.631939 15058 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.635164  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:33.636052  6384 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 20 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:28:33.637080 14714 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:33.639807 14714 tablet_bootstrap.cc:492] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:28:33.639896 14997 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:33.545527 15004 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:33.646850 15040 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:33.638607 14978 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:33.649226  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.650796  6381 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:33.722500  6384 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:28:33.577409 14998 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:33.652848 15122 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.583460 15056 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.076s
I20260709 12:28:33.659338 14994 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:33.583243 14993 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:28:33.665284  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:33.653491  6383 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:33.669850 15015 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.670601  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:33.671239 14714 ts_tablet_manager.cc:1403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 3.197s	user 0.091s	sys 0.054s
I20260709 12:28:33.671839  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:33.684142  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:33.754029  6305 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:33.686373  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:33.688445  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.698730  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3654) took 1349 ms. Trace:
I20260709 12:28:33.699615  6379 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:33.705430  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:33.755038  6303 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:33.708976  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114347us
I20260709 12:28:33.709722 15121 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:28:33.710932 15066 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:28:33.711578 14997 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:33.711648  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:33.712720 15035 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.718840  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3678) took 1148 ms. Trace:
I20260709 12:28:33.719034 15004 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.700138 14214 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:28:33.720324 15133 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:33.723304  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:33.754287 15015 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:33.702180 15040 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:33.727632 14978 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.726589  7331 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 32.
I20260709 12:28:33.728646 14692 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:33.729925 14254 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:33.646026 15120 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:33.732287  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 185602us
I20260709 12:28:33.732945  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:33.808985  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:33.740906  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1274) took 1012 ms. Trace:
W20260709 12:28:33.740929  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 297148us
I20260709 12:28:33.743815  6377 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:33.658102 14756 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 12:28:33.756767  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:33.809976  6380 rpcz_store.cc:276] 0709 12:28:32.728313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.868571 (+140258us) service_pool.cc:224] Handling call
0709 12:28:32.868889 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:33.061565 (+192676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:33.061601 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:33.065421 (+  3820us) log.cc:844] Serialized 26 byte log entry
0709 12:28:33.399185 (+333764us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:33.399248 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.724035 (+324787us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b5400109fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:33.724102 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:33.724166 (+    64us) raft_consensus.cc:1685] finished
0709 12:28:33.724187 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:33.740836 (+ 16649us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32011,"raft.queue_time_us":27713,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":1906,"spinlock_wait_cycles":378244352,"thread_start_us":311582,"threads_started":1,"wal-append.queue_time_us":432277}
I20260709 12:28:33.765578  6303 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:33.731330 14998 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:33.766873  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78550us
I20260709 12:28:33.790464 14997 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:33.755157  6382 rpcz_store.cc:276] 0709 12:28:32.349068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.370097 (+ 21029us) service_pool.cc:224] Handling call
0709 12:28:33.698501 (+1328404us) spinlock_profiling.cc:243] Waited 95.8 ms on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.698648 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":220239616}
I20260709 12:28:33.773592  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:33.776767  6307 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:33.777884  6376 rpcz_store.cc:276] 0709 12:28:32.570181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.603097 (+ 32916us) service_pool.cc:224] Handling call
0709 12:28:33.718681 (+1115584us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b5400038254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.718786 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70718208}
I20260709 12:28:33.777292  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3665) took 1206 ms. Trace:
I20260709 12:28:33.829655  6379 rpcz_store.cc:276] 0709 12:28:32.565488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.574156 (+  8668us) service_pool.cc:224] Handling call
0709 12:28:33.699192 (+1125036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:33.756192 (+ 57000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.771332 (+ 15140us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001f4ad0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:33.771471 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2586501376}
I20260709 12:28:33.783962  6305 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:33.725652  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3739) took 1473 ms. Trace:
I20260709 12:28:33.796949 14714 raft_consensus.cc:359] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:33.770735 14994 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:33.777473 15034 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:33.777766 15083 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:28:33.767401  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 96140.4us
I20260709 12:28:33.816725  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:33.814769  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 250883us
I20260709 12:28:33.802711 15004 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:33.845382  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:33.808751 15009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.009s	sys 0.007s
I20260709 12:28:33.848949  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:33.831681  6454 rpcz_store.cc:276] 0709 12:28:32.251862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:32.253402 (+  1540us) service_pool.cc:224] Handling call
0709 12:28:32.274345 (+ 20943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:32.806559 (+532214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:33.725365 (+918806us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b54001c1e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:33.725579 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156895616}
I20260709 12:28:33.880384  6459 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:33.827760  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 71130.3us
I20260709 12:28:33.827152 14998 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:33.866813  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:33.869033  6455 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:33.853308  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79796.1us
I20260709 12:28:33.881830  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:33.853863 14714 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:28:33.854962 15126 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:33.807200 14978 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.810227 14254 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:33.879856 15009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.065s	user 0.010s	sys 0.008s
I20260709 12:28:33.808292 14692 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.867681 14214 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:33.914126 14214 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:33.915341 14214 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.827049 15008 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.067s
I20260709 12:28:33.875061 15083 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:33.815322 15058 consensus_queue.cc:1059] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 12:28:33.944381  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 127786us
I20260709 12:28:33.854034 14994 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.875978 15122 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.904713  6455 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:33.906917 15013 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:33.902081 14862 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.902194 15118 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:28:33.902395 15009 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:33.913256  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:33.979950  6454 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:33.802891 15133 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.983157 15133 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:33.932850  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:33.850730 15031 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:28:33.827822 15010 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:33.883262 15034 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:33.991748 15034 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.995865 15010 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.827069 14993 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:28:33.895612  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:33.997394  6458 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:34.003587  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70883.1us
I20260709 12:28:34.005713  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:33.947090 15083 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:33.943778  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:34.012718  6459 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:33.964689 14692 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.964892 14254 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.896907 15126 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:33.973852  6307 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
W20260709 12:28:33.976673  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 105324us
I20260709 12:28:34.011009 13772 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.975968 15035 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:33.977310 15140 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:34.042882  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95950.2us
I20260709 12:28:33.916767 15143 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.120s
I20260709 12:28:33.901299  6351 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:34.030366 13772 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:33.976670 15013 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:34.044015  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 227800us
I20260709 12:28:34.066649  6424 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:34.067777  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:34.068581  6458 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:34.063287  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:34.070762 13772 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.035933 14994 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:33.917438 15040 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.044826 15136 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:33.888465 14714 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:34.037472 15126 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:34.116740  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:34.101011 15146 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:28:34.073547 15035 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:34.098868 15155 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:34.100190 15137 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:28:34.152806  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:34.009106 14862 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.065244 15010 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:34.070381 15013 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:34.065438 15118 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:34.110260  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:34.113554  7332 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:28:34.116626  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:34.126288  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:34.087904 15150 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:33.980670 14997 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:34.157902  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 256666us
I20260709 12:28:34.165081  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:34.192759  7319 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
W20260709 12:28:34.166993  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92549.3us
I20260709 12:28:34.068894 15140 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.183559  7329 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:34.201946  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 135388us
I20260709 12:28:34.183902  6458 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:34.201154  7319 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:28:34.147621 15040 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:34.193804  6351 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:34.201231  7320 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:34.207265  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:34.213783  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 207326us
I20260709 12:28:34.215649  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:34.169818 15120 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.180938 15151 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:34.236414  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.184612 15143 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.154069 14994 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
W20260709 12:28:34.214332 15146 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:28:34.117961 15136 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.161784 15035 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.245465  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.249740  6310 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 17.
I20260709 12:28:34.245553  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.250871  6303 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:28:34.252657  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.246418  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.247869 15156 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:34.248668  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.267980  6304 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:34.114008 15009 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.271154  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.271973  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.272341  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.256889  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.273136  6302 raft_consensus.cc:2368] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
W20260709 12:28:34.273932  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.255753  6309 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:34.275292  6307 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:34.228242 15116 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:34.190759 14967 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:28:34.276357  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.277938  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.275561 15156 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.162304 15155 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.288075 15140 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.288408  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:34.290832  6309 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:34.300267 14994 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:34.312083  6310 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:34.305117  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:34.316102  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 250003us
I20260709 12:28:34.318383  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:34.320170  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 126451us
I20260709 12:28:34.241621 14978 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.347189  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.348608  6300 raft_consensus.cc:1275] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:28:34.346992 14824 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.053s	user 0.004s	sys 0.000s
I20260709 12:28:34.356567  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.416275  6302 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:34.364482  6303 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:34.361867  7335 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:34.368261  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:34.361521  7329 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:34.380971  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.348913  6309 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:28:34.248967 15143 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.276948 15153 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:34.307442 15167 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:34.425405  7319 raft_consensus.cc:2393] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 18: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:34.418468  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:34.422851  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54132.4us
I20260709 12:28:34.388973  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:34.408782  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.426923  7320 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:28:34.428064  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 220882us
I20260709 12:28:34.431843  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:34.437448  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56795.2us
I20260709 12:28:34.439179  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:34.360015 15133 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:34.375070 15140 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.442878  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.378782 15118 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:34.463186  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44911.8us
I20260709 12:28:34.449924 13772 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:28:34.248988 15151 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:34.424604 15144 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.388s	user 0.001s	sys 0.001s
I20260709 12:28:34.425213 15167 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.438934  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:34.464063 13772 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:34.388797 15152 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:28:34.407231 14714 ts_tablet_manager.cc:1434] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.641s	user 0.007s	sys 0.008s
I20260709 12:28:34.304889 15015 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.402719 15031 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.398823 15168 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.410759 15136 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:34.465034 15133 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.497298 15133 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.384827 15171 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:34.498888 15171 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:34.499655 15143 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:34.499887 15171 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.501377 15156 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.318787 15116 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.504282 15116 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.467873 15166 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:34.517699 15126 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.437034 15041 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.478394  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.478804 13772 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:34.477190  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:34.478804  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:34.533808  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56744.5us
I20260709 12:28:34.535895  6425 leader_election.cc:400] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:34.536197  6425 leader_election.cc:403] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
W20260709 12:28:34.467280 15004 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.582s	user 0.014s	sys 0.000s
I20260709 12:28:34.489617  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:34.518631 15153 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:28:34.519558 14862 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:34.537736  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48226.2us
W20260709 12:28:34.521181 15160 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:28:34.344803 14993 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 12:28:34.521845 15168 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:34.540521  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:34.541926 15168 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.542471  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:34.543229  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64513.8us
I20260709 12:28:34.543679  7320 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:34.544126 14664 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:34.544618 14664 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:34.545428 14664 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.546514  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:34.548413 14756 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:34.552218  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:34.462761 15142 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:34.554432 15142 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:34.554898 15142 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:34.387684 14994 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:34.558784 14994 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.561059 14994 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.537863 14967 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:34.561738 14967 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:34.562706 14967 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:34.524314  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48801.9us
I20260709 12:28:34.571907  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:34.541975  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:34.581475  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:34.582579 15116 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:28:34.589190 14756 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:34.589704 14756 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.590055 14756 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:34.605880  6303 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 29: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:34.531015 15023 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.481010 15167 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:34.614305  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78546.4us
W20260709 12:28:34.614943  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 136622us
I20260709 12:28:34.623054  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:34.481853 15173 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:28:34.490694 14714 ts_tablet_manager.cc:1467] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Tablet startup took 737ms
I20260709 12:28:34.658313  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.631625  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:34.470634 15140 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.477012 15006 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.136s
I20260709 12:28:34.613943 15162 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:28:34.669432  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 102471us
I20260709 12:28:34.670840  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.678013  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:34.624109 14994 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:34.679399  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3702) took 1097 ms. Trace:
I20260709 12:28:34.679030  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:34.481184 15118 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.525493 15031 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.685587 13772 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.686885  6450 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:34.612079 15023 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.663066 15066 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:34.645280 15153 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:34.607168 15041 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.688833  6385 rpcz_store.cc:276] 0709 12:28:33.582210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:33.647075 (+ 64865us) service_pool.cc:224] Handling call
0709 12:28:33.647407 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:33.647843 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:33.647874 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:33.663138 (+ 15264us) log.cc:844] Serialized 26 byte log entry
0709 12:28:33.814429 (+151291us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:33.977900 (+163471us) log.cc:844] Serialized 14 byte log entry
0709 12:28:34.043294 (+ 65394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:34.043539 (+   245us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:34.679100 (+635561us) raft_consensus.cc:1685] finished
0709 12:28:34.679133 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:34.679305 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":278635,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":182188,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":12050,"thread_start_us":327848,"threads_started":3,"wal-append.queue_time_us":279051}
I20260709 12:28:34.674644 15143 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:34.672339 15162 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.676177 15133 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:34.705686 13772 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.708078  6457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:34.704161 15153 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.695024 14997 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:34.679546 15136 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.031s
I20260709 12:28:34.706655 14994 raft_consensus.cc:2764] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:28:34.696460 15035 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:34.729760  7329 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:34.723533 15162 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:34.725812 15151 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:34.748096  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75141.2us
I20260709 12:28:34.696707 15165 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:28:34.734313 15133 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:34.731004 14997 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.758986  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:34.730777 15153 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:34.754273 15083 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.718s	user 0.001s	sys 0.011s
I20260709 12:28:34.754146 15136 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:28:34.783313 15179 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
W20260709 12:28:34.663086 14714 ts_tablet_manager.cc:1469] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Trace:
0709 12:28:22.599118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:22.669033 (+ 69915us) service_pool.cc:224] Handling call
0709 12:28:23.124566 (+455533us) ts_tablet_manager.cc:934] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
0709 12:28:24.515044 (+1390478us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b6caa 00007f6fa01b6bd2 00007f6fa821b131
0709 12:28:27.180373 (+2665329us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:28:28.485502 (+1305129us) tablet_metadata.cc:665] Metadata flushed
0709 12:28:28.485630 (+   128us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:28:28.519552 (+ 33922us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:28:28.521024 (+  1472us) tablet_replica.cc:206] Creating consensus instance
0709 12:28:28.525524 (+  4500us) ts_tablet_manager.cc:989] Replica registered.
0709 12:28:28.539330 (+ 13806us) inbound_call.cc:177] Queueing success response
0709 12:28:30.474078 (+1934748us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:28:33.766562 (+3292484us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:28:33.794317 (+ 27755us) tablet_replica.cc:261] Starting instrumentation
0709 12:28:33.795001 (+   684us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":341,"mutex_wait_us":8833,"raft.queue_time_us":31254,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":502,"spinlock_wait_cycles":75329408,"tablet-copy.queue_time_us":224275,"tablet-download-pool.queue_time_us":5906,"tablet-download-pool.run_cpu_time_us":1878,"tablet-download-pool.run_wall_time_us":1373257,"thread_start_us":1591210,"threads_started":2}
I20260709 12:28:34.786116 15133 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.761371  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.755905 15151 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:34.794749 15140 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.796355  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:34.797376 15023 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.756259 14997 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:34.813315  6382 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:28:34.809182  6376 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:34.814400  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:34.787139 15181 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:28:34.816426  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:34.830010  6377 raft_consensus.cc:2376] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26. Re-sending same reply.
I20260709 12:28:34.830909  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:34.831648  6382 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:34.742754 14664 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.819039  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:34.819799  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:34.820585  6380 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:34.817564  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:34.822332  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:34.826603 15136 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:34.842321  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:34.818156  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56862.1us
I20260709 12:28:34.844213  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.775172 14994 consensus_queue.cc:1048] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:28:34.772784 15155 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:34.848670  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:34.788964 15173 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:34.813148 15151 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:34.781584 15066 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
W20260709 12:28:34.835079 15161 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:28:34.916779  6382 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:28:34.919139  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 240159us
I20260709 12:28:34.816896 15192 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:34.791004 15083 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.841003 15015 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:34.847123 15126 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:34.841197 15163 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
I20260709 12:28:34.917335  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:34.923475  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:34.845793 15136 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:34.942456  6378 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:34.917740 15004 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:34.916489  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67933.5us
I20260709 12:28:34.948467  6381 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:34.845978 14664 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:34.916252 15176 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:34.915756 15178 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:28:34.942509 15175 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:28:34.935698 15125 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:34.995195  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:35.000742 15190 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:28:35.002257 15183 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:28:34.917603 15171 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:28:35.002574 15118 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:35.006033 14998 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 1.105s	user 0.014s	sys 0.000s
I20260709 12:28:35.006981 15173 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.056743 14998 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:34.960700 15031 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:34.940194 15023 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.066709  6385 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:28:35.051932 15184 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 12:28:34.959753 15136 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:35.049804 14824 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:28:35.022337 15040 consensus_meta.cc:287] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.809s	user 0.007s	sys 0.004s
I20260709 12:28:35.062512 15045 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.020257 15041 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.036136 15125 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:35.085063  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 161681us
I20260709 12:28:34.936802 15192 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.103757  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:35.106209  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:35.099781 15045 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.094659 15194 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:35.111339 15194 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:35.112322 15194 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.090234 14998 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.114161 14998 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:35.048846 15151 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.115707 15151 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:35.118142 15189 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:35.118791 15189 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.121218  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.114764 14998 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:35.122509  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.123159  6453 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:35.112905 15045 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.124524 15197 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:35.125924 14664 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.128207 15189 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.113651 15125 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.118793 15194 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:35.129125 15125 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:35.022442 15004 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.123461 14998 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:35.129222 15116 consensus_meta.cc:287] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:28:35.139942  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:35.139889  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 158337us
I20260709 12:28:35.019841 14756 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.081651 15178 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:35.037312 15142 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.145550  6456 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.12-22.12]
I20260709 12:28:35.146291  6384 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:35.093818 15171 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.087617 15066 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:28:35.147250  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:35.149996  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.154637  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:35.073297 15023 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.161196 15129 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:28:35.135610 15176 consensus_queue.cc:1175] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 28 status { last_received { term: 26 index: 18 } last_committed_idx: 18 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: [27.19-27.19]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:35.168673 15171 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.155817 15178 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:35.175294  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.176306 15004 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.147877 15210 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:35.182296 15083 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.183211 15066 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:35.147019 15116 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:28:35.136227 14997 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.156507  7331 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:35.277693 15066 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:35.278178 15002 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:35.279186 15186 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:35.278997 15066 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.279873 15186 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.280241 15002 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:35.280643 15066 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:35.281141 15002 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:28:35.282694  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.283475 15159 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:35.283931 15159 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:35.284896 15159 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.186873  6453 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:35.216035 15162 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.187844 15008 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:35.218150 15145 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 1.027s	user 0.006s	sys 0.000s
W20260709 12:28:35.188050 15013 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.994s	user 0.008s	sys 0.004s
I20260709 12:28:35.187522 15192 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.182329 14756 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:35.226199  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 71660.3us
W20260709 12:28:35.226171  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79019.2us
W20260709 12:28:35.228976  6351 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.009s	sys 0.046s
I20260709 12:28:35.233225  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.234189  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.235791 13772 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:35.228721 14994 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:35.215032  6458 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:35.278528 15200 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:35.334272  6451 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:35.287034  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.324131 13772 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.225144 15129 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:35.224931 15177 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:35.308228  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:35.338835  6351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.013s	sys 0.049s
I20260709 12:28:35.335944  6453 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:35.339803  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.334586  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:35.302476 15167 consensus_meta.cc:287] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.641s	user 0.002s	sys 0.009s
W20260709 12:28:35.309736 15199 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:28:35.292528 15153 consensus_meta.cc:287] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.512s	user 0.010s	sys 0.000s
I20260709 12:28:35.352053  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.282280 15186 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.286425 15159 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:35.353502  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.278084 15202 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:35.232568 15178 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:35.237159 15006 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.355638  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.342043 15200 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.338899 15013 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.357940 15206 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:35.360859  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.361265 15192 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.361786 14824 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=3 applied=9 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:28:35.362571 15142 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.330457 15116 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:35.297574 15171 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.391185  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.422283  6377 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:35.398597  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.399055  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.399509  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.307698 15023 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.344844 15151 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:28:35.346326 14994 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:35.306126 15164 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:35.432054 15164 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.432425 14994 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.279037 15187 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:35.412834  6451 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:35.407609 15165 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:35.382162 15153 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.396054 15083 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.350872 15177 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.419936 15136 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.426574 15206 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:35.429594 14824 tablet_bootstrap.cc:492] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:28:35.443519 15116 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.470619 14824 ts_tablet_manager.cc:1403] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 2.932s	user 0.057s	sys 0.032s
I20260709 12:28:35.468963  6379 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:35.440088 15218 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:35.434564 15164 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.435096 15162 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.419080 15105 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.437188 15200 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.416332 15006 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.449604  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-1ee2be3ef27a4ec7bd5946b81b8ee8c3 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:28:35.422894 15192 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.355039 15167 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.344763 15008 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.406541  6453 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:35.442404  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:35.481703  6458 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:28:35.482776  6456 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:35.406987 15202 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.418130  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:35.484066  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66017us
I20260709 12:28:35.485860  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.486531  6379 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:35.489477  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 154974us
I20260709 12:28:35.490303  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:35.490422  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.491134  6308 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:35.465503  6377 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:35.402153 15208 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:35.443917 15210 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:35.498735  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.498997  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:35.499461  6307 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:35.499800  6309 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:35.450438 15125 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.423502  6383 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:35.502408  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.502657  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.503114  6308 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:35.478392  6455 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:35.479532  6457 raft_consensus.cc:606] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:35.493085  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.494115  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.495460  7319 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:35.495712  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.505123  7314 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:35.505467  6377 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:35.496048  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.496724  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.507524  7332 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 32.
I20260709 12:28:35.451817 15040 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.026s
I20260709 12:28:35.511289  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.512228  6305 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:35.512588  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.513908  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:35.515175  6309 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:35.518586  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.519253  6302 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 32.
I20260709 12:28:35.519682  6304 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:35.519927 15167 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.521673  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.524950  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:35.526100  6380 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:35.468324 15083 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.523187  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:35.529187 14664 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:35.529304  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:35.529661 14664 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.498392  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-1ee2be3ef27a4ec7bd5946b81b8ee8c3 on tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:35.417220 15129 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.534739  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:35.519819 15143 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:35.534929  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:35.541831  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 44964.3us
I20260709 12:28:35.452034 15187 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.508019 15023 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:35.477339 15218 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.530666  6351 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:35.509227 15173 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.527024  6383 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
W20260709 12:28:35.732349  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.532248  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.541626  6379 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:35.732910  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.733726  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.545836  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:35.734381  6379 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:35.546651  6453 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:28:35.734962  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 204362us
W20260709 12:28:35.735200  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 189448us
I20260709 12:28:35.735888  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.736459  6453 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:28:35.737216  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.737707  6451 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:35.737864  6383 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:28:35.479278 15105 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.652272 15149 raft_consensus.cc:3055] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:35.599164 15217 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:35.746290  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.747112  6379 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:35.747761  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.748462  6451 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:35.726752 15159 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.647692 15083 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.542104 15142 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.068s
I20260709 12:28:35.645742 15008 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.507692 15166 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:28:35.760838  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.741114 14756 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:35.764021  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.744061 15171 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:35.540548 15185 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:35.550045 15162 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:35.536708 15210 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.645500 15192 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.508078 15208 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.531924 15133 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:35.530339 14664 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:35.640904  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.643404 15041 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:28:35.637823 15172 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.115s	user 0.005s	sys 0.000s
I20260709 12:28:35.614532 15218 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.612977 15212 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:35.545722 15116 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:35.726840  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 178111us
I20260709 12:28:35.627964 14824 raft_consensus.cc:359] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:35.644081 15153 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.522047 15125 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:35.542656 15143 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:35.588291  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53423.5us
W20260709 12:28:35.730247  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 182043us
I20260709 12:28:35.860145  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:35.797385  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:35.535336  6377 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:35.738792  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.634598  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1521) took 1762 ms. Trace:
I20260709 12:28:35.648345 15013 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:35.750691  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.640477 15177 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.751315  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.869825  6458 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:35.773540 14756 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:35.752408  7331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1242) took 1637 ms (client timeout 2098 ms). Trace:
I20260709 12:28:35.870352  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:35.752856  6453 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
W20260709 12:28:35.871806  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136107us
I20260709 12:28:35.871961  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.757961  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:35.753896  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.873149  6451 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:35.589737  6452 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:35.728582 14988 consensus_queue.cc:1059] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:28:35.764832  6384 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:35.862882 15143 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:35.762249 15159 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.743585 15149 raft_consensus.cc:740] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:35.773221 15162 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.740563 15173 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.788919  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.789779  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.885061  7332 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:35.790477  7314 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:28:35.745954 15170 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
W20260709 12:28:35.810802  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:35.834872  7335 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:35.771109 15185 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.767176 15171 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:35.867010  6303 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:35.867273  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.867601  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:35.776966 15105 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.863339 15189 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:35.873452  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.864706  6450 rpcz_store.cc:276] 0709 12:28:33.872528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:33.872737 (+   209us) service_pool.cc:224] Handling call
0709 12:28:33.889216 (+ 16479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:34.694699 (+805483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:35.634303 (+939604us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54000c21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:35.634529 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1824990592}
I20260709 12:28:35.549204 14997 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.901243 14998 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.918530 14998 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.920571 14998 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.921361 15149 consensus_queue.cc:260] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.922417 15149 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:35.900328  6310 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:35.915352  6377 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:28:35.925913 15208 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.927181 14756 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.929100  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.933332 15125 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:35.870699  7331 rpcz_store.cc:269] 0709 12:28:34.109619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.109855 (+   236us) service_pool.cc:224] Handling call
0709 12:28:35.746806 (+1636951us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54006c16d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:35.746939 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2093161216}
I20260709 12:28:35.776042 15136 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.935830  6309 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:28:35.936244 15105 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.936787 15105 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.917912  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.939337  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.939738  7316 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:35.877019  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.926004  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1277) took 1118 ms. Trace:
I20260709 12:28:35.940918  6385 rpcz_store.cc:276] 0709 12:28:34.807385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.807539 (+   154us) service_pool.cc:224] Handling call
0709 12:28:34.817991 (+ 10452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:35.067247 (+249256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:35.925789 (+858542us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:35.925949 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571243776}
I20260709 12:28:35.941474  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.878243  6384 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:28:35.942412  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.940452  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.945223  6385 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:35.880846  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:35.787071 15041 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:35.546569 15206 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.872532  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:35.862289 15192 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.868610 15008 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:35.792250 15040 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:35.897182 15173 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.966387 15040 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.967239  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:35.968958 15040 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.872507  6453 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:35.970559 14664 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.971469 15192 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.972914 14664 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.973656 15192 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:35.955665 15143 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.978978 15143 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.979748 15008 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:35.980832 15008 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.982120  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.983460  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.984055  6456 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:35.984890  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.985915  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.885396 15162 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:35.978292  6301 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [20.11-21.12]
W20260709 12:28:35.905457  7320 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1252) took 1359 ms (client timeout 1676 ms). Trace:
I20260709 12:28:35.990206  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:35.988749  7320 rpcz_store.cc:269] 0709 12:28:34.546130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.546299 (+   169us) service_pool.cc:224] Handling call
0709 12:28:35.905264 (+1358965us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54007a0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:35.905395 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347992448}
I20260709 12:28:35.858906 14824 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
W20260709 12:28:35.894599 15221 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 12:28:35.894641 15123 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:35.909044 15066 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.628s	user 0.014s	sys 0.000s
I20260709 12:28:35.909754 15159 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.883594 15150 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:35.882994 15153 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.895224 15202 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:35.895655 15222 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
I20260709 12:28:35.915138 15142 consensus_queue.cc:1059] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:35.915300  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:35.898774 15223 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:28:35.915750 14994 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:36.029342 15153 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:35.898952 15185 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:36.031888 15153 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.916065  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:35.899124 15171 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:35.899892  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.037528  6303 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:36.038365  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.897234 15187 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:35.917430 15217 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.900642  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.039011 15142 raft_consensus.cc:3055] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:36.039626 15142 raft_consensus.cc:740] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:36.039433 15217 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.040611 15142 consensus_queue.cc:260] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.042979 15105 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.043381  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.044020  6456 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:36.043669 14824 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:36.045053  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.045765  6456 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.045855 14824 ts_tablet_manager.cc:1434] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.575s	user 0.014s	sys 0.010s
W20260709 12:28:36.046222 14824 ts_tablet_manager.cc:1467] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Tablet startup took 575ms
I20260709 12:28:35.930377  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.057945  6302 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 20.
I20260709 12:28:35.796923 15188 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:35.955623  6379 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:35.956668  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:35.966634 15041 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.791296 15178 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:35.961612  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.062212  6384 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:35.961949  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:35.962204  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 89763.3us
I20260709 12:28:35.959139  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:35.964545  6310 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 19.
I20260709 12:28:35.875981 15116 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:28:36.053755 15105 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:35.885733  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:36.009234  6453 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:36.012691  6385 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:36.082932  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.083660  6379 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 32: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:36.083823  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.084412  6385 raft_consensus.cc:2376] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22. Re-sending same reply.
I20260709 12:28:35.990272 15208 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:36.027503  7335 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:36.131735  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.027393  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.133271  6453 raft_consensus.cc:1240] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:36.133430  6300 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:35.991325 15177 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.135861 15177 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.044091  7318 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 32.
I20260709 12:28:36.027381 15150 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:36.047706 15185 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.027165 15123 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.048728  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:35.941403  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:35.942476  6458 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.050384  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.039059  6303 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:36.051323  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:36.051259 15217 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:35.957690  7331 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:36.067389  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:36.066365  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 107323us
I20260709 12:28:35.904949  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.080343  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.081578  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.093716  6455 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:36.096904  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:36.098436 15149 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.100848 15168 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:28:36.119225 15129 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:36.121668 13772 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:36.123438 15212 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:36.134615  7317 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.132439  6384 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:36.149801  6452 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:36.155664  6459 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:28:36.190382  6195 maintenance_manager.cc:419] P 08c731ab128f4ef881e4c3c73ff48bd8: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033374
I20260709 12:28:36.191886  6459 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:28:36.192032  6456 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:36.192317  6459 pending_rounds.cc:85] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:28:36.192584  6459 pending_rounds.cc:107] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:28:36.192847  6459 raft_consensus.cc:2889] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:36.193262  6310 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 18.
I20260709 12:28:36.081391 15202 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:36.194303 15202 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.194792 15202 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:36.178936 15187 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.195158  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.195751 15187 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.196472  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.197059 15150 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.197481  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.197999 15187 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.198189 15150 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:36.163126  6454 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
W20260709 12:28:36.199506  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 102681us
I20260709 12:28:36.169138  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3390) took 1348 ms. Trace:
W20260709 12:28:36.200287  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 120056us
W20260709 12:28:36.200878  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 149630us
I20260709 12:28:36.201404  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.200116  6376 rpcz_store.cc:276] 0709 12:28:34.820845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.820965 (+   120us) service_pool.cc:224] Handling call
0709 12:28:34.821255 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:34.926751 (+105496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:34.926790 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:34.927230 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 12:28:35.938389 (+1011159us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b1800136fc0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:36.054677 (+116288us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:28:36.054716 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.054776 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:36.165789 (+111013us) raft_consensus.cc:1685] finished
0709 12:28:36.165809 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.169064 (+  3255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114143,"raft.queue_time_us":135668,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":73514,"spinlock_wait_cycles":2688384,"thread_start_us":116125,"threads_started":1,"wal-append.queue_time_us":1193255}
I20260709 12:28:36.202735  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:36.203269  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.203352  7318 raft_consensus.cc:1217] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEARNER]: Deduplicated request from leader. Original: 22.13->[22.14-22.14]   Dedup: 22.14->[]
I20260709 12:28:36.194322  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.204249 13772 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:36.204591  6310 raft_consensus.cc:2393] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:36.204818 13772 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:36.197104  6302 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:28:36.205295 13772 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:36.205432  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.202701  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:36.205830  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:36.206104  6384 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:36.213771  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:36.216787  6300 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:36.217713  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.211356  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:36.219410  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.221935  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:36.222219  6456 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:36.223121  6450 raft_consensus.cc:2393] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.223035  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.223006  7335 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:36.224677  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.134582  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.224977  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.225354  6300 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:36.225840  6379 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:36.223846  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.062733 15188 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.230311  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:36.134635 15208 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.236351  7331 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:28:36.239044  6383 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28.
I20260709 12:28:36.168680 15178 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:36.200416 15212 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:36.213459 15168 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:36.246008  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.226028 15116 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:36.236779  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.248739  6302 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:36.232165 15112 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:36.225374 15118 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:36.315263  6380 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:36.232335 15010 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:36.314337 15125 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.318853  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.335927 15125 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.336887  6450 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 19: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.338253  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.339612  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.319114  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.327924 15118 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:36.341236  6376 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:36.352250  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:55284 (request call id 42) took 1644 ms. Trace:
I20260709 12:28:36.352430  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.352741  6457 rpcz_store.cc:276] 0709 12:28:34.707787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:34.707933 (+   146us) service_pool.cc:224] Handling call
0709 12:28:36.352076 (+1644143us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:36.352160 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1726234368}
I20260709 12:28:36.134136 15105 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:36.355966  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.314234 15042 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.323073  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.317883 15013 raft_consensus.cc:2764] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:28:36.340479  6450 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.327445 15112 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:36.327966  6304 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:28:36.354087  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.354131  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:36.324345  6300 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:36.356283  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.355790  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:36.230846 15120 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 12:28:36.316498 15116 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:36.326206  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.217001 15149 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.250396  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.217998 14993 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:36.241868 15040 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:28:36.370115  6457 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:36.362325  6380 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:36.376243  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:36.379338  6300 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:36.409988 14993 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:36.411170  6300 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:36.411093 14993 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.235038  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.412457  6307 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
W20260709 12:28:36.256539  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.387990  6379 raft_consensus.cc:2376] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29. Re-sending same reply.
I20260709 12:28:36.364529  6450 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:36.343945 15185 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.401772 13772 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.415319  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.382688  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.420322  6385 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:36.421516  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.422160  6385 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:36.366123  6376 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:36.425491  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.426235  6376 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:36.416409  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.413812  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3719) took 1013 ms. Trace:
I20260709 12:28:36.427518  6383 rpcz_store.cc:276] 0709 12:28:35.399901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.738563 (+338662us) service_pool.cc:224] Handling call
0709 12:28:36.413639 (+675076us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b540023ebd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:36.413756 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859258752}
I20260709 12:28:36.237385 14756 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:36.243556 15208 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:36.368832 15194 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:28:36.382979 15116 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:36.438704 15156 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.444295  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.466818  6383 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:36.467782  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.459419 15121 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:36.459192 15171 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.461431 15105 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:36.461971 15235 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:28:36.462818  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:36.454470 15162 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.484527  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:36.366083 15042 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.453301  6450 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:36.419015 14254 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:36.471375  6384 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:36.493780  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3717) took 1094 ms. Trace:
I20260709 12:28:36.423218 15008 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:36.472500  6383 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:36.473555  6310 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:28:36.423914 15168 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.344888 15118 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.475260 15112 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.476951  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3706) took 1116 ms. Trace:
I20260709 12:28:36.480031 13772 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.496405  6376 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:36.497208  6450 raft_consensus.cc:606] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:36.538954  6450 raft_consensus.cc:614] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:36.494251  6384 rpcz_store.cc:276] 0709 12:28:35.399391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.470721 (+1071330us) service_pool.cc:224] Handling call
0709 12:28:36.471055 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:36.493421 (+ 22366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.493706 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:36.500051  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3720) took 1099 ms. Trace:
W20260709 12:28:36.502689  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.540284  6384 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:28:36.541157  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1282) took 1131 ms. Trace:
I20260709 12:28:36.539726  6383 rpcz_store.cc:276] 0709 12:28:35.400136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.471453 (+1071317us) service_pool.cc:224] Handling call
0709 12:28:36.472030 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:36.499681 (+ 27651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.499979 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:36.541744  6384 rpcz_store.cc:276] 0709 12:28:35.409657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.539713 (+1130056us) service_pool.cc:224] Handling call
0709 12:28:36.539989 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:36.540863 (+   874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.541092 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:36.499504 15008 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:36.506654  6452 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:36.516353 15168 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:36.478818 15156 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.504272  6382 rpcz_store.cc:276] 0709 12:28:35.361024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.361141 (+   117us) service_pool.cc:224] Handling call
0709 12:28:35.361493 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:35.361894 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:35.362199 (+   305us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.099306 (+737107us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b18004d78e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:36.112996 (+ 13690us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b18004d78e0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a
0709 12:28:36.404183 (+291187us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.404988 (+   805us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.405636 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:28:36.407114 (+  1478us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:36.407335 (+   221us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.407566 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.407663 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:36.476688 (+ 69025us) raft_consensus.cc:1685] finished
0709 12:28:36.476713 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.476885 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1687,"prepare.queue_time_us":1372,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":53281,"raft.run_cpu_time_us":569,"raft.run_wall_time_us":22911,"spinlock_wait_cycles":60233088,"thread_start_us":300229,"threads_started":2,"wal-append.queue_time_us":65362}
I20260709 12:28:36.545341  6384 raft_consensus.cc:1217] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Deduplicated request from leader. Original: 23.18->[24.19-28.21]   Dedup: 24.19->[27.20-28.21]
I20260709 12:28:36.504123 14254 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:36.548899  6379 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:28:36.524271 15171 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.569355  6384 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 28. Current term is 29. Ops: [24.19-28.21]
I20260709 12:28:36.576525  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1291) took 1152 ms. Trace:
I20260709 12:28:36.576964  6384 rpcz_store.cc:276] 0709 12:28:35.424066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.544668 (+1120602us) service_pool.cc:224] Handling call
0709 12:28:36.545108 (+   440us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:36.576131 (+ 31023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.576451 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:36.549973  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 177124us
I20260709 12:28:36.530088 15010 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:36.057591 14824 ts_tablet_manager.cc:1469] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Trace:
0709 12:28:25.637362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:25.637502 (+   140us) service_pool.cc:224] Handling call
0709 12:28:26.149682 (+512180us) ts_tablet_manager.cc:934] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
0709 12:28:30.210582 (+4060900us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:28:30.536487 (+325905us) tablet_metadata.cc:665] Metadata flushed
0709 12:28:30.536650 (+   163us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:28:30.844026 (+307376us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:28:30.844437 (+   411us) tablet_replica.cc:206] Creating consensus instance
0709 12:28:30.844765 (+   328us) ts_tablet_manager.cc:989] Replica registered.
0709 12:28:30.844876 (+   111us) inbound_call.cc:177] Queueing success response
0709 12:28:32.538213 (+1693337us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:28:35.470982 (+2932769us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:28:35.613912 (+142930us) tablet_replica.cc:261] Starting instrumentation
0709 12:28:35.614710 (+   798us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":285,"mutex_wait_us":6466,"raft.queue_time_us":5646,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4181,"tablet-copy.queue_time_us":223085,"tablet-download-pool.queue_time_us":44961,"tablet-download-pool.run_cpu_time_us":1408,"tablet-download-pool.run_wall_time_us":985288,"thread_start_us":150910,"threads_started":2}
I20260709 12:28:36.533967 15120 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:36.524044 15125 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.578459  6384 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:28:36.566823 15194 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:36.605576 13772 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:36.592343 14254 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.608712 15042 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.592617 15039 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.018s
I20260709 12:28:36.609961 15149 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.403692 15212 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.579577 15188 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.610709 15039 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:36.619880  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:36.621532  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:36.601859 15120 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.605273 15008 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:36.638971  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.652717 15008 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:36.652856  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:36.653674 15245 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:36.655059  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:36.655933 15245 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:36.656291 15245 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:36.656630 15245 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:36.660050  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.642093  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:36.642935 15156 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.623389 15040 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:36.662153 15045 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:36.649093 15162 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:36.615705 15202 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.605371 15194 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:36.620034 15039 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:36.663717  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.639611 15180 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:36.664717  7288 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.663378  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-6ea0c2d7699e4559823a28a280a0e713 received from {username='slave'} at 127.0.0.1:46610
I20260709 12:28:36.667269  6378 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:36.670277  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-6ea0c2d7699e4559823a28a280a0e713 on tablet 6ea0c2d7699e4559823a28a280a0e713 with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:36.673069  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 296954us
I20260709 12:28:36.674065  6305 raft_consensus.cc:2368] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 25. Current term is 26.
I20260709 12:28:36.669281 15180 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.674333 15042 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:36.676896 15045 raft_consensus.cc:2804] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:28:36.704162 15045 raft_consensus.cc:697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:36.683126  7317 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:28:36.705055 15232 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:28:36.705181 15045 consensus_queue.cc:237] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.684268 15247 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:28:36.695144  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3707) took 1333 ms. Trace:
I20260709 12:28:36.707266  6381 rpcz_store.cc:276] 0709 12:28:35.361302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.393095 (+ 31793us) service_pool.cc:224] Handling call
0709 12:28:35.402201 (+  9106us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:35.836966 (+434765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:35.850624 (+ 13658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:35.850699 (+    75us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:35.850755 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.694885 (+844130us) spinlock_profiling.cc:243] Waited 435 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:36.694941 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.695084 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":999401344}
W20260709 12:28:36.695552  6425 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.018s	sys 0.031s
W20260709 12:28:36.709030  6425 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.018s	sys 0.032s
W20260709 12:28:36.709323  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 581129us
I20260709 12:28:36.696219 15194 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:36.697702 15202 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:36.684537 15042 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:36.684417 15039 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.733628 15039 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:36.712028  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:36.714605  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.717279  6300 raft_consensus.cc:1240] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 18. Current term is 19. Ops: []
I20260709 12:28:36.719635  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:36.786764  6307 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:36.682652 15180 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:36.797405 15180 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.816386  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.819463  6381 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:36.820163  6381 raft_consensus.cc:606] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:36.820531  6381 raft_consensus.cc:614] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:36.833788  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:36.814420 15042 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:36.801290 15180 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:36.839587  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:36.836151  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 124194us
I20260709 12:28:36.847047  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:36.847888  7319 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:28:36.857131  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:36.858019  6304 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:28:36.866449  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.881497  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3715) took 1489 ms. Trace:
I20260709 12:28:36.882189  6385 rpcz_store.cc:276] 0709 12:28:35.392484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.445287 (+1052803us) service_pool.cc:224] Handling call
0709 12:28:36.445855 (+   568us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.446391 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.446432 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.470090 (+ 23658us) log.cc:844] Serialized 26 byte log entry
0709 12:28:36.601133 (+131043us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:28:36.865990 (+264857us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.881104 (+ 15114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.881191 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:36.881241 (+    50us) raft_consensus.cc:1685] finished
0709 12:28:36.881257 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.881424 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17453,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":107,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":16207,"thread_start_us":338643,"threads_started":2,"wal-append.queue_time_us":188850}
I20260709 12:28:36.886615  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:36.860462 15039 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:36.902397  7319 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
W20260709 12:28:36.907793  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:36.856401 15208 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:36.931525  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51714.7us
I20260709 12:28:36.896526 15039 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:36.936394  6454 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3771) took 1798 ms. Trace:
I20260709 12:28:36.937181  6454 rpcz_store.cc:276] 0709 12:28:35.138258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.138565 (+   307us) service_pool.cc:224] Handling call
0709 12:28:35.145289 (+  6724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.213022 (+1067733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.936126 (+723104us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400345654. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:36.936335 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2339990144}
I20260709 12:28:36.946341  6381 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:28:36.943565 15039 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:36.907065 15247 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:28:36.951025  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1290) took 1533 ms. Trace:
I20260709 12:28:36.947710 15006 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:36.954638  6383 rpcz_store.cc:276] 0709 12:28:35.417404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.542121 (+1124717us) service_pool.cc:224] Handling call
0709 12:28:36.542586 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:36.542894 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:36.894762 (+351868us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.950686 (+ 55924us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:36.950739 (+    53us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:36.950786 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.950851 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.950963 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6390,"prepare.queue_time_us":56285,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":119,"raft.run_wall_time_us":2482,"thread_start_us":375393,"threads_started":2}
W20260709 12:28:36.954195 15249 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:28:36.966789 15248 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:28:36.972810  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3398) took 1097 ms. Trace:
I20260709 12:28:36.973219  6384 rpcz_store.cc:276] 0709 12:28:35.875136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.593489 (+718353us) service_pool.cc:224] Handling call
0709 12:28:36.612678 (+ 19189us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.613260 (+   582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.613293 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.615095 (+  1802us) log.cc:844] Serialized 26 byte log entry
0709 12:28:36.833832 (+218737us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:36.833876 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.833949 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:36.972550 (+138601us) raft_consensus.cc:1685] finished
0709 12:28:36.972613 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.972749 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":196,"raft.queue_time_us":1152,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":3223,"thread_start_us":167960,"threads_started":1,"wal-append.queue_time_us":300865}
I20260709 12:28:36.976274  6384 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:28:36.979871  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1288) took 1568 ms. Trace:
I20260709 12:28:36.972204  6383 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:36.980325  6376 rpcz_store.cc:276] 0709 12:28:35.411415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.539813 (+1128398us) service_pool.cc:224] Handling call
0709 12:28:36.540041 (+   228us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.551633 (+ 11592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.551681 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.552290 (+   609us) log.cc:844] Serialized 26 byte log entry
0709 12:28:36.862153 (+309863us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:36.905892 (+ 43739us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.906107 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:36.906185 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:36.979630 (+ 73445us) raft_consensus.cc:1685] finished
0709 12:28:36.979651 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:36.979804 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11187,"prepare.queue_time_us":46079,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":6926,"raft.queue_time_us":26432,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7386,"spinlock_wait_cycles":4218624,"thread_start_us":275766,"threads_started":1,"wal-append.queue_time_us":376634}
I20260709 12:28:36.978102  6304 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:36.954643 15245 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:36.997800 15180 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:37.001407 15259 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
W20260709 12:28:37.004900 15116 consensus_meta.cc:287] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.522s	user 0.006s	sys 0.006s
I20260709 12:28:37.017042 15041 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:37.017784 15212 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:37.023833  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.026015  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3721) took 1625 ms. Trace:
I20260709 12:28:37.057020  6379 rpcz_store.cc:276] 0709 12:28:35.400376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.485397 (+1085021us) service_pool.cc:224] Handling call
0709 12:28:36.539334 (+ 53937us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.570177 (+ 30843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.025716 (+455539us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b54003f25d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:37.025935 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":79843200}
I20260709 12:28:37.026597 15058 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:37.032119  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1371) took 1164 ms. Trace:
W20260709 12:28:37.033154  6378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3708) took 1671 ms (client timeout 2040 ms). Trace:
I20260709 12:28:37.038936 13772 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:37.048785  7320 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:37.062274 13772 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.056584  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:37.061546  6378 rpcz_store.cc:269] 0709 12:28:35.361569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.363699 (+  2130us) service_pool.cc:224] Handling call
0709 12:28:37.032989 (+1669290us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003e94d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:37.033096 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2858885376}
I20260709 12:28:37.060918  6305 rpcz_store.cc:276] 0709 12:28:35.867545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.881106 (+ 13561us) service_pool.cc:224] Handling call
0709 12:28:37.031859 (+1150753us) spinlock_profiling.cc:243] Waited 468 ms on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:37.032026 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1076232064}
I20260709 12:28:37.068064  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:37.069365  6351 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:37.070860  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:37.073348  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 351036us
I20260709 12:28:37.072674  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:37.073642 15159 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:37.115651 15159 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:37.116155 15159 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:37.077239 15167 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:37.119692 15167 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:37.120263 15167 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:37.044183 15180 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.128832 15180 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.135176 13772 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.136022 15142 raft_consensus.cc:3060] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:28:37.137859 15123 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:37.139192  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.141450  6308 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 12:28:37.119824 15271 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:28:37.146929  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.077279 14254 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:37.154203  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 79127us
I20260709 12:28:37.083376  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.155565  6307 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:37.081916  6304 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:37.086272  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.159368  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:37.155030  6275 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:37.076849 15255 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:28:37.047932 15116 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.120391 15208 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.161759 14254 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.163597  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.140470 15011 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.175253  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.175920  6300 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:37.175962  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:37.177642  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 105033us
I20260709 12:28:37.224783 14254 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.183555  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:37.159256 15008 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.073738 15194 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:37.180295  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.238145  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.239056  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.159050 14214 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.242028  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:37.244377  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.169111 15252 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.247092  6304 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:37.248420  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:37.249019  6302 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:37.181236 15123 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:37.263644  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.273172  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.067093 15058 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.278359  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:37.507018  6301 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:37.252099 14214 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.185093 15011 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.247217 15208 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:28:37.237361 15042 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.242419 15008 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:37.353916  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.266603 15252 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:37.350800  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 138114us
W20260709 12:28:37.419364  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 177400us
I20260709 12:28:37.180202 15174 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:37.423137 15120 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:37.424741 15202 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:37.706952  6304 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:37.424783  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3405) took 1481 ms. Trace:
I20260709 12:28:37.707859  6385 rpcz_store.cc:276] 0709 12:28:35.943675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.912816 (+969141us) service_pool.cc:224] Handling call
0709 12:28:36.913278 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.930994 (+ 17716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.931039 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.931547 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:28:37.053620 (+122073us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:37.053679 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.053826 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:37.424574 (+370748us) raft_consensus.cc:1685] finished
0709 12:28:37.424593 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:37.424727 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":16200,"thread_start_us":104136,"threads_started":1,"wal-append.queue_time_us":247527}
I20260709 12:28:37.428182 15245 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:37.429607  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:37.429924  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:37.431239  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3400) took 1550 ms. Trace:
I20260709 12:28:37.432328  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:37.424736 15166 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:37.699560 15143 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:37.730245 15143 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.732430 15143 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:37.247967 15006 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:37.443389  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.442677 15039 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.446435  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3932) took 1907 ms. Trace:
I20260709 12:28:37.460117  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.734810  6454 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:37.735759 15039 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.735929  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.461324 15210 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.741816 15210 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.743770 15202 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:37.744632 15210 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:37.461427  6457 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:28:37.746412  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.747388  6457 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:37.748236  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:37.463589  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.749712  6457 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:37.750320  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.751376  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:37.741014  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:37.464324  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:37.465063  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.462852  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:37.447590 15167 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.443430  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:37.768553  7316 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:37.472162 15168 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.016s
I20260709 12:28:37.456369  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3401) took 1572 ms. Trace:
I20260709 12:28:37.462078  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:37.457072  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 742056us
I20260709 12:28:37.775074  6380 rpcz_store.cc:276] 0709 12:28:35.883615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:35.956316 (+ 72701us) service_pool.cc:224] Handling call
0709 12:28:37.456196 (+1499880us) spinlock_profiling.cc:243] Waited 255 ms on lock 0x7b54001950d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:37.456310 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":585385728}
W20260709 12:28:37.775887  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 314172us
I20260709 12:28:37.776667  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:37.462167  6456 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:37.463284  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.777827  6452 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
W20260709 12:28:37.456686  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 814630us
I20260709 12:28:37.457520  7317 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:37.456288  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:37.448552 15159 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.667197  6302 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:37.456810 15162 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:37.457248 15206 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.128s
W20260709 12:28:37.564028  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52878.1us
I20260709 12:28:37.783752  6454 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:37.506778 15212 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.455614 15116 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:37.523942  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:37.485229 15142 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:37.448521 15194 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:37.450609 15013 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:37.711638  6381 rpcz_store.cc:276] 0709 12:28:35.880452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.896491 (+1016039us) service_pool.cc:224] Handling call
0709 12:28:36.909853 (+ 13362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.946807 (+ 36954us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.431023 (+484216us) spinlock_profiling.cc:243] Waited 36.2 ms on lock 0x7b54003d0c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:37.431185 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83258112}
I20260709 12:28:37.515197 15125 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.151s
I20260709 12:28:37.582378 15123 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:37.719656  6309 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:37.721048  6305 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:37.728303 15252 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.460811  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.736584  7319 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:37.733714  6451 rpcz_store.cc:276] 0709 12:28:35.538477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.084205 (+545728us) service_pool.cc:224] Handling call
0709 12:28:36.166336 (+ 82131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:36.166621 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:36.379379 (+212758us) log.cc:844] Serialized 14 byte log entry
0709 12:28:36.593015 (+213636us) log.cc:844] Serialized 14 byte log entry
0709 12:28:37.445307 (+852292us) spinlock_profiling.cc:243] Waited 2.97 ms on lock 0x7b1800327a00. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:37.445730 (+   423us) log.cc:844] Serialized 14 byte log entry
0709 12:28:37.445931 (+   201us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:37.445980 (+    49us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:37.446042 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.446139 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:37.446358 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":850279,"prepare.queue_time_us":332109,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2041,"raft.queue_time_us":869773,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":8815,"spinlock_wait_cycles":16317184,"thread_start_us":400324,"threads_started":2,"wal-append.queue_time_us":442157}
I20260709 12:28:37.737568  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:37.739131 15041 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.683s	user 0.005s	sys 0.008s
I20260709 12:28:37.722273 15008 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.462448  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:37.754015  6453 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:37.756107  6458 pending_rounds.cc:85] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 18
I20260709 12:28:37.811143  6458 pending_rounds.cc:107] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 27.19
I20260709 12:28:37.811492  6458 raft_consensus.cc:2889] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:37.753648  6274 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:37.768985  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 325628us
I20260709 12:28:37.769531  7331 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:28:37.778162 13772 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:37.782330 15039 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:37.796399  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:37.819309  6302 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:37.722254 14214 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.699178 15174 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:37.824602  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71026.3us
I20260709 12:28:37.827020  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:37.697005 15010 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 12:28:37.871300 13772 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.885583 13772 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.877997  6455 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:37.809166  6459 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:37.872835 15039 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:37.791148 15143 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
I20260709 12:28:37.962894  6305 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:28:37.963763  6454 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:37.871376 15118 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:37.804888 15210 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:37.828895 15174 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:37.947734 13772 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:37.908885 15180 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:37.950470  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 123530us
I20260709 12:28:37.975437  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:37.978325  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.008s	sys 0.012s
I20260709 12:28:37.797048 15212 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:37.936084 15202 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.953747  6302 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:37.956950  6301 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:37.979442  6457 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
W20260709 12:28:37.976518  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 239045us
I20260709 12:28:37.987684 15042 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.987977 15212 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.989768 15167 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:37.990450 15167 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:37.991899 15011 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:37.808820 15013 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.999567 15118 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:37.984082 15180 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:37.804545 15142 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.000629 15118 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.002646 15208 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:37.948715 14214 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:37.967783 15162 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.019021  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:37.984630  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.009s	sys 0.012s
I20260709 12:28:38.009531  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:37.943528 15039 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:37.892006 15116 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
I20260709 12:28:38.018101  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:38.025228  6457 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:38.045773  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:38.122846  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:38.124672  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 115248us
I20260709 12:28:38.125708  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:38.043380  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.126892  6302 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:38.044404  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:37.969424 15121 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:38.130244  6385 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 33: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:38.128955  6377 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:28:38.132592 15272 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.516s	user 0.003s	sys 0.003s
I20260709 12:28:38.132951  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:38.133002  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:38.007632 15143 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.138808  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:38.124603  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 78954.9us
I20260709 12:28:38.021837 15206 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:38.145340 15206 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.018296 15208 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:38.155808  6378 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:38.159054 15174 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.129875 15262 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:38.042861  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:38.162775 15262 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:38.163298 15262 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:38.027562 15276 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.223s	user 0.000s	sys 0.005s
I20260709 12:28:38.141705 15252 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:38.167680  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1382) took 1446 ms. Trace:
I20260709 12:28:38.168064  6303 rpcz_store.cc:276] 0709 12:28:36.721357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.726138 (+  4781us) service_pool.cc:224] Handling call
0709 12:28:36.726562 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:37.075291 (+348729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:37.075326 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:37.130859 (+ 55533us) log.cc:844] Serialized 26 byte log entry
0709 12:28:37.336372 (+205513us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:37.336415 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.336520 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:38.167491 (+830971us) raft_consensus.cc:1685] finished
0709 12:28:38.167512 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:38.167627 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":360324,"raft.queue_time_us":418851,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":3193,"thread_start_us":205038,"threads_started":1,"wal-append.queue_time_us":433786}
W20260709 12:28:38.146919 15273 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.437s	user 0.000s	sys 0.001s
I20260709 12:28:38.170083 15215 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:37.985695 15210 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.170995 15008 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:38.171411 15008 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:38.171718 15215 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:38.171802 15008 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:38.172454 15210 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:38.147929  6451 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:38.173229 15210 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:38.173600 15210 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:38.174006  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.022120 15010 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:38.018699 14214 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:38.149004  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:38.149979  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.133356 15058 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.147647  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.183548  6453 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:38.145773 15206 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:38.147291 15280 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:28:38.007992 15194 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.163343  6381 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:38.172528 15215 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.174197 15177 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.021394 15162 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:38.170828  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1299) took 1293 ms. Trace:
I20260709 12:28:38.200076  6303 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:28:38.146198 15143 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.204912  6301 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:38.200270  6384 rpcz_store.cc:276] 0709 12:28:36.877786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.977755 (+ 99969us) service_pool.cc:224] Handling call
0709 12:28:36.978089 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.979993 (+  1904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.980051 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.981067 (+  1016us) log.cc:844] Serialized 26 byte log entry
0709 12:28:37.180006 (+198939us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:37.247500 (+ 67494us) log.cc:844] Serialized 14 byte log entry
0709 12:28:37.247745 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.247825 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:38.170633 (+922808us) raft_consensus.cc:1685] finished
0709 12:28:38.170651 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:38.170774 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":74433,"prepare.run_cpu_time_us":20347,"prepare.run_wall_time_us":156697,"raft.queue_time_us":8315,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":10267,"spinlock_wait_cycles":19102080,"thread_start_us":198440,"threads_started":1,"wal-append.queue_time_us":531331}
I20260709 12:28:38.210219 15143 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:38.211740 15143 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.212347 15143 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.176875 15010 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:38.219398  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.199283 15058 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:38.221484  6451 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:37.999119 15013 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:38.178001  6456 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:38.130950 15142 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:38.185496  6302 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:38.156020 15208 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:38.003566 15123 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:38.223973  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78968.3us
I20260709 12:28:38.233803  6381 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:38.236506  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.220731 15159 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:38.218982 15010 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:38.174366 15210 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.215713 15215 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.248731  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:38.249379  6451 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:38.178648  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 394283us
I20260709 12:28:38.208081 15194 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.223652 15174 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:38.242182 15288 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.246031  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.250574  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.251201  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.259011  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:38.259528  6378 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:38.251610  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:38.260308  6385 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:38.261921  6450 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:38.262704 15143 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:38.263448  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:38.264276  6378 raft_consensus.cc:2368] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:38.265429  6378 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:38.266911  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.252498  6377 raft_consensus.cc:1240] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:38.276942  6385 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:38.329051  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:38.329811  7318 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:38.294209 15143 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.336877 15143 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.283003 15210 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:38.341847  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1542) took 1166 ms. Trace:
I20260709 12:28:38.343945 15281 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.344720 15281 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.348162 15208 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.349666 15170 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:38.245260 15013 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
W20260709 12:28:38.277983 15166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.029s
I20260709 12:28:38.264098  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.280784  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.354506  6451 raft_consensus.cc:2393] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:38.354632  6384 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:28:38.259605  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.278198  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.356154  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:38.357443  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:38.224913 15058 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.297065  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:38.293375  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:38.301843 15194 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.222865 14214 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.307967 15215 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.310199 15206 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.283978 15288 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.312605  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:38.370755  6301 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:38.313423  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:38.317903 15162 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.319777 15262 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.288757 15174 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:38.291810 15168 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.000s	sys 0.033s
I20260709 12:28:38.351436  7319 rpcz_store.cc:276] 0709 12:28:37.175449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.175655 (+   206us) service_pool.cc:224] Handling call
0709 12:28:38.341525 (+1165870us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b5400a41fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:38.341735 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1286295936}
I20260709 12:28:38.345255  7318 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
W20260709 12:28:38.352804  6425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.014s	sys 0.018s
I20260709 12:28:38.350220 15208 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.358878  7316 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:38.360019  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63043.9us
I20260709 12:28:38.364709  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.371969  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:38.382465  6425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.014s	sys 0.018s
W20260709 12:28:38.383601  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 606993us
I20260709 12:28:38.383687  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:38.384697  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:38.385437  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:38.354151 15166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.004s	sys 0.029s
I20260709 12:28:38.289525 15008 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.386343  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.388561 15206 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:38.362601 15142 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:38.350442 15170 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.400915  6381 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:38.400098  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:38.401719 15285 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:28:38.374116 15252 consensus_queue.cc:1175] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" responder_term: 25 status { last_received { term: 24 index: 13 } last_committed_idx: 13 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:38.384514 15283 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:28:38.389437 15262 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:38.384801 15168 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.000s	sys 0.034s
I20260709 12:28:38.404438  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:38.407682  7316 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 33.
I20260709 12:28:38.538825  6451 raft_consensus.cc:2368] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:38.390053 14214 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:38.495239  7285 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.67:45591 timed out after 2.023s (SENT) [suppressed 13 similar messages]
W20260709 12:28:38.543078  6458 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.730s	user 0.000s	sys 0.038s
I20260709 12:28:38.503047  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.545957  6450 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:38.401871 15290 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.421692 15143 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:38.548090  6349 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.010s	sys 0.028s
W20260709 12:28:38.554896  6349 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.011s	sys 0.028s
I20260709 12:28:38.391317 15166 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:38.552762  6194 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.697s	user 0.003s	sys 0.020s
I20260709 12:28:38.390900 15174 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:38.555634  6456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:28:38.560850  6456 raft_consensus.cc:606] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:38.561328  6456 raft_consensus.cc:614] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:38.374440 15039 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:38.548590  6458 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.000s	sys 0.038s
I20260709 12:28:38.539881 15206 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:38.398509 15287 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.161s
I20260709 12:28:38.570309 15288 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.557731 15290 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.571517 15170 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.572098 15170 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:38.572605 15170 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.573078 15290 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:38.573628 15290 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:38.555469 15281 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:38.558045 15168 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:38.565845 15118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.035s	sys 0.031s
I20260709 12:28:38.574074 15290 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:38.590240 15245 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.027s	sys 0.032s
W20260709 12:28:38.580817 15118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.036s	sys 0.031s
W20260709 12:28:38.593676 15245 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.027s	sys 0.032s
I20260709 12:28:38.591584 15170 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:38.622151  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 365813us
I20260709 12:28:38.627708  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:38.630039  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 373798us
I20260709 12:28:38.591367 15288 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:38.584223 15168 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.638907  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:38.639736  7316 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:38.641860  7316 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33. Ops: []
I20260709 12:28:38.645015  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:38.633118  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 375681us
I20260709 12:28:38.668040  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:38.671002  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 286519us
I20260709 12:28:38.709564  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:38.711876  7319 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 21. (term mismatch)
I20260709 12:28:38.712438  7319 pending_rounds.cc:85] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 20
I20260709 12:28:38.712517  6425 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
I20260709 12:28:38.712833  7319 pending_rounds.cc:107] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 28.21
I20260709 12:28:38.713129  6425 leader_election.cc:403] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:38.702809  6274 leader_election.cc:400] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:38.625010 15170 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:38.716200  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48267.7us
I20260709 12:28:38.707245  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.716887  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:38.717738  6381 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:38.707245  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:38.714689  6274 leader_election.cc:403] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:28:38.667215 15274 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.548s	user 0.003s	sys 0.006s
I20260709 12:28:38.634117 15288 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:38.719036  6307 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 18.
W20260709 12:28:38.722641  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45813.7us
I20260709 12:28:38.719352  6425 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:38.721657  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 226465us
W20260709 12:28:38.720569  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 322141us
I20260709 12:28:38.665748 15058 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:38.724988  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:38.725379  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:38.733889  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:38.734993  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:38.723670  7319 raft_consensus.cc:2889] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:38.745815 15123 consensus_queue.cc:1059] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:38.746505 15013 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:38.747192 15013 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:38.747427 15125 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:38.747732 15013 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:38.748556 15125 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:38.749737 15125 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.751474 15125 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:38.723745 15285 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:38.761171 15285 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.758090 15013 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:38.769243  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:38.702505 15118 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:38.670051 15291 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:38.734123  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:38.774678  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48128.5us
I20260709 12:28:38.717104 15281 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:38.799728  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.721354 15142 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25.
W20260709 12:28:38.782612  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3796) took 1320 ms (client timeout 1619 ms). Trace:
W20260709 12:28:38.804534  6454 rpcz_store.cc:269] 0709 12:28:37.461760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.740698 (+278938us) service_pool.cc:224] Handling call
0709 12:28:38.782316 (+1041618us) spinlock_profiling.cc:243] Waited 30.4 ms on lock 0x7b54002c8b54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:38.782437 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70033152}
I20260709 12:28:38.784580  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3653) took 1543 ms. Trace:
W20260709 12:28:38.787221  6305 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3637) took 1708 ms (client timeout 1901 ms). Trace:
W20260709 12:28:38.808616  6305 rpcz_store.cc:269] 0709 12:28:37.079350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.093004 (+ 13654us) service_pool.cc:224] Handling call
0709 12:28:38.787003 (+1693999us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b54002e8454. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:38.787144 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1290236800}
W20260709 12:28:38.788301  6304 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3654) took 1538 ms (client timeout 1546 ms). Trace:
W20260709 12:28:38.810015  6304 rpcz_store.cc:269] 0709 12:28:37.250069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.255377 (+  5308us) service_pool.cc:224] Handling call
0709 12:28:38.788100 (+1532723us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54002ddcd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:38.788231 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465813376}
I20260709 12:28:38.731557 15112 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:38.814895  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:38.817912  6455 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 32: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:38.807647  6309 rpcz_store.cc:276] 0709 12:28:37.241185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.429697 (+188512us) service_pool.cc:224] Handling call
0709 12:28:38.784293 (+1354596us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b54002de1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:38.784497 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17839488}
I20260709 12:28:38.815582  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:38.816711  6457 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:38.665985 15166 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:38.824007  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:38.824085  6304 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:38.772563 15143 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:38.726320 15170 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.772631 15298 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.773752 15289 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:38.770758 15008 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.780750 15291 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:38.831748  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1300) took 1859 ms. Trace:
I20260709 12:28:38.837859  6376 rpcz_store.cc:276] 0709 12:28:36.972007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.982544 (+ 10537us) service_pool.cc:224] Handling call
0709 12:28:36.982911 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:36.983247 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:36.983277 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:36.983713 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:28:37.010869 (+ 27156us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:37.010916 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:37.010985 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:38.555949 (+1544964us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:38.822462 (+266513us) raft_consensus.cc:1685] finished
0709 12:28:38.822514 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:38.831667 (+  9153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3869,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":2933,"spinlock_wait_cycles":98745088,"thread_start_us":26754,"threads_started":1,"wal-append.queue_time_us":76373}
I20260709 12:28:38.729892 15010 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.845142 15303 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:38.800926 15118 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:38.845535 15039 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.842408 15291 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:38.848388 15174 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:38.843737 15298 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:38.978009  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 262010us
I20260709 12:28:38.833503 15142 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:28:38.837209 15245 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:38.841225 15058 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 12:28:38.876263  7320 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1540) took 2045 ms. Trace:
I20260709 12:28:38.989830  7320 rpcz_store.cc:276] 0709 12:28:36.830628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.830769 (+   141us) service_pool.cc:224] Handling call
0709 12:28:36.831015 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:37.062357 (+231342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.872226 (+1809869us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b5400508554. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:38.876191 (+  3965us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500162944}
I20260709 12:28:38.822264 15281 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:38.887921 15299 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:38.772912 15296 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.112s
I20260709 12:28:38.870082  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1262) took 2193 ms. Trace:
I20260709 12:28:38.887486 15283 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:39.008752  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:38.926702 15293 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:28:38.982139  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:38.980643  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:39.013466  6309 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:38.887363 15289 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:38.888725  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:38.980706 15295 log.cc:927] Time spent T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
I20260709 12:28:39.004122  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.019207  6450 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:39.020304  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.004329  6456 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:39.003532  7331 rpcz_store.cc:276] 0709 12:28:36.676227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:36.676342 (+   115us) service_pool.cc:224] Handling call
0709 12:28:36.676668 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:37.814710 (+1138042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.869658 (+1054948us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540070e4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:38.869983 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2512529152}
I20260709 12:28:39.004416  7285 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:39.005244  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:39.012734  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 277822us
I20260709 12:28:39.013697 13772 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:28:38.988797 15239 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 2.438s	user 0.013s	sys 0.008s
I20260709 12:28:39.014694  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.016031  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.005239 15303 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:39.024515  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:39.022228  6455 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:39.001416 15299 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.025568  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.023553 13772 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.026213  6450 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 20: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:39.026990  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.026702  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:39.028074  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 359426us
I20260709 12:28:39.029772 13772 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:39.028323  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 252852us
I20260709 12:28:38.998070 15245 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.029508  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.005016 15305 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:39.015136 15142 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:39.030092  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 305184us
I20260709 12:28:39.030157  6454 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:39.022423 15112 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:39.018630 15301 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:39.022006 15290 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.019912 15283 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:39.026885 15039 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:39.038017  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3802) took 1571 ms. Trace:
I20260709 12:28:39.027827 15303 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.040148  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:39.043044  6453 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:39.035221  6424 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:39.036604 15112 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:39.033730 15013 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.044826 15301 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.060909 15290 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:39.031291 15299 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:39.044173 15162 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.062649 15289 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.063519 15298 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:39.066282 15118 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:39.066634 15168 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.048668 15319 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:28:39.067796 15112 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.049320 15305 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:28:39.069090 15112 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:39.043077 15142 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.046849 15304 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:39.071012 15304 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:39.071111 15311 log.cc:927] Time spent T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:28:39.072029 15304 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.046234 15303 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:39.073184 15283 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.049628 15170 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.080734 15303 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.030588 15041 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:39.055532 13772 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.042604 15010 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:39.072554 15302 log.cc:927] Time spent T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.182s	user 0.001s	sys 0.001s
W20260709 12:28:39.078327 15210 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.739s	user 0.009s	sys 0.001s
I20260709 12:28:39.069922 15299 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.096782 15291 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.101522 15291 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:39.102679 13772 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.113293 15282 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:39.113994 15282 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:39.115165 15282 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.116605 15282 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:39.141134 15299 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.112345 15303 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.154186 15290 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:39.229939  6194 maintenance_manager.cc:643] P 08c731ab128f4ef881e4c3c73ff48bd8: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 3.038s	user 0.043s	sys 0.028s Metrics: {"bytes_written":5802,"delete_count":0,"lbm_write_time_us":5839,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":5,"reinsert_count":0,"spinlock_wait_cycles":35265280,"thread_start_us":68827,"threads_started":1,"update_count":225,"wal-append.queue_time_us":203880}
W20260709 12:28:39.238462  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:39.239377  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 362411us
I20260709 12:28:39.240617  6195 maintenance_manager.cc:419] P 08c731ab128f4ef881e4c3c73ff48bd8: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.259145
I20260709 12:28:39.260936  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:39.038054  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:39.277408  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:39.278339  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 240215us
I20260709 12:28:39.280402  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:39.280947  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:39.286381  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.287832  6304 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 19.
I20260709 12:28:39.290474  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:39.296975  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.298043  6309 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:39.299052  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:39.301578  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.305712  6309 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:28:39.308944  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 273639us
I20260709 12:28:39.310792  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:28:39.319072  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 267390us
I20260709 12:28:39.321861  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.323376  6309 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:39.337509  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:39.338446  6305 raft_consensus.cc:2376] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25. Re-sending same reply.
I20260709 12:28:39.045949  6458 rpcz_store.cc:276] 0709 12:28:37.466580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.755686 (+289106us) service_pool.cc:224] Handling call
0709 12:28:37.755944 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:37.812104 (+ 56160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:37.812149 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:37.812650 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:28:38.565504 (+752854us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:38.565558 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:38.565732 (+   174us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:39.037771 (+472039us) raft_consensus.cc:1685] finished
0709 12:28:39.037797 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:39.037947 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":752435,"threads_started":1,"wal-append.queue_time_us":862453}
I20260709 12:28:39.340464  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:39.342294  6309 raft_consensus.cc:2410] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
W20260709 12:28:39.357546  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51835.7us
I20260709 12:28:39.359042  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:39.361121 14254 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:39.362818 14254 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:39.363375 14254 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:39.367591  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:39.371018  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.372311  7319 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:39.375206 15008 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:39.376513 15010 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:28:39.420188  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.429665 15291 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:39.430433  7335 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:39.431130 15294 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:39.433847 15294 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:39.434758 15013 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:39.435506 15013 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:39.435173 15297 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:39.442534 15323 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:39.437371  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.453635  7331 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:39.446441 15303 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.412683 15174 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:39.104759 15142 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.451365 15305 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:28:39.452452  7320 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:39.453385 15323 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
I20260709 12:28:39.464341 15283 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.465474 15283 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:39.467208  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 148697us
I20260709 12:28:39.468446  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.470754 15210 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:39.471004 15305 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:39.471586 15210 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:39.472398  7316 raft_consensus.cc:1240] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:28:39.471036  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.473930 15305 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:39.467094  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3659) took 1176 ms. Trace:
I20260709 12:28:39.475029 15305 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:39.450850 15304 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:28:39.453222 15166 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:39.475755  6302 rpcz_store.cc:276] 0709 12:28:38.290753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.290963 (+   210us) service_pool.cc:224] Handling call
0709 12:28:38.300510 (+  9547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:38.300897 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:28:38.677165 (+376268us) log.cc:844] Serialized 14 byte log entry
0709 12:28:39.233898 (+556733us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:39.236002 (+  2104us) log.cc:844] Serialized 26 byte log entry
0709 12:28:39.236363 (+   361us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:28:39.236480 (+   117us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.236690 (+   210us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:39.466578 (+229888us) raft_consensus.cc:1685] finished
0709 12:28:39.466607 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:39.466832 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":533227,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":890,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":4241,"thread_start_us":680795,"threads_started":2,"wal-append.queue_time_us":371127}
I20260709 12:28:39.478660 15278 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:28:39.449111  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 90368.8us
I20260709 12:28:39.478832 15305 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.483939  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.484208  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.485188  6383 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 21.
I20260709 12:28:39.484269  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:39.475983  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.473387 15210 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.491761 15208 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.490532  6376 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:39.493075  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.493511  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.473265 15303 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:39.430441 15291 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.479550 15278 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:39.456063 15297 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:39.450949 15162 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:39.534157 15245 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.534523 15129 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:39.532673 15013 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.537467 15013 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:39.538146  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:39.546008 15283 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:39.539201  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.535079 15129 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.559067 15129 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:39.559825  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.560833  6383 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:39.542820 15166 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.543722  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:39.561806 15278 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.563021  6381 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:39.564543 15166 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.550758  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.533576 15297 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.561854  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.526347 15291 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:39.533754 15162 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:39.553205  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.568985 15162 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:39.546066 15208 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.530561 15174 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:39.554813 15006 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.571911 15006 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.572508 15174 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.533206  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.538702  6275 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:39.575162  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 51450.5us
I20260709 12:28:39.556217  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.576462 14756 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:39.577121  6385 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:39.551255  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.557789 15336 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:39.582938 15336 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:39.592003  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 150775us
I20260709 12:28:39.587200  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.595654  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.592657 15336 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:39.590364 15208 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.602804  6425 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:39.603037  6384 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:39.609611  6309 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 30: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:39.609450 15006 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.620848  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:39.600723 15303 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.635850  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:39.636979  6383 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:39.640568 14254 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.670981  6381 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:28:39.673178  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:39.679608  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 63387us
I20260709 12:28:39.666150 15303 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:39.688640  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:39.691848 15303 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:39.705276  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.695796 15303 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:28:39.712791 15285 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:39.714395 15285 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:39.714941 15285 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:39.716609 15143 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:39.716434 15285 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.717310 15143 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:39.717742 15285 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:39.722754 15338 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:39.731072 15338 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:39.734717 15210 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:39.735561 15210 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.735785 15338 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.732878 15323 consensus_queue.cc:1059] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:28:39.739647 15212 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:39.740120 15212 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:39.741612 15212 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.742710 15281 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.071s
W20260709 12:28:39.746459 15282 consensus_meta.cc:287] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.629s	user 0.016s	sys 0.008s
W20260709 12:28:39.717098 15112 consensus_meta.cc:287] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.647s	user 0.000s	sys 0.021s
I20260709 12:28:39.695444 15301 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.725172 15288 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:39.756397 15288 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:39.694377 15297 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.758716 15129 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.758888 15323 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.762872 15129 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.763476 15323 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.704166 15206 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Committing config change with OpId 18.9: config changed from index -1 to 9, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:28:39.759339 15281 raft_consensus.cc:3055] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:39.767139 15281 raft_consensus.cc:740] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:39.768523 15281 consensus_queue.cc:260] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.770174 15281 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:39.738008 15210 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:39.805377  6379 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:39.760560 15297 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.811805  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.704811 15291 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.801434  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:39.826776  6383 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
W20260709 12:28:39.834962  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 184695us
W20260709 12:28:39.844676  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:39.856101  6459 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:28:39.858503  7314 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:39.859913  7331 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:39.858956  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.857059  7288 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:39.858258  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:39.867185  6451 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:39.851532 15212 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:39.860335  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.857389  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:39.848234 15112 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.856446 15323 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:39.876459  7316 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:28:39.887153  7319 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 33: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:39.886270  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.900439  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:39.896150  6272 connection.cc:256] RPC call timeout handler was delayed by -1.1209130310003275s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2195s for a response
I20260709 12:28:39.902169 15299 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:39.902715  6194 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.297s	user 0.014s	sys 0.005s
I20260709 12:28:39.907169 15296 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.903813 15351 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.924115  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.925096  6453 raft_consensus.cc:2393] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:39.926071  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.895295 15112 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.927951 15143 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:39.928493 15112 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:39.929215  6451 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:39.930369  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.932895  6453 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29.
I20260709 12:28:39.933369 15323 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:39.935415 15323 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:39.934311  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:39.940582 15350 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:39.941205 15350 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:39.941632 15350 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:39.947543  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.949200  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.950078  6453 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:39.939836  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:39.951612 15210 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:39.938122 15323 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:39.960356  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1558) took 1223 ms. Trace:
I20260709 12:28:39.960965  6457 rpcz_store.cc:276] 0709 12:28:38.736803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:38.799411 (+ 62608us) service_pool.cc:224] Handling call
0709 12:28:39.959840 (+1160429us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b54003b92d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:39.960080 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36968320}
I20260709 12:28:39.963194  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.967631  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:39.993271  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:39.997534  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:39.970606 15143 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:40.008793  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 151868us
I20260709 12:28:40.012524  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.019814  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:40.023926  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:40.024864  6381 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:40.026122  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.026597  6383 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:40.027346  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.027987  6381 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:39.998721 15008 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:40.028169 15177 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:40.028752 15008 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.028944 15177 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.031018  6457 raft_consensus.cc:2410] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:40.031371 15177 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.032086  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.032737  6308 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:40.037084  6424 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.038304  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.039429 15286 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:40.039989 15286 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:40.040591  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.041551  6381 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:40.041103 15286 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.042570  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.031281 15008 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.043942  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:40.044266  6381 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:40.045212  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.045776  6381 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:40.046715 15322 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:28:40.047447  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.033682  6378 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:40.033639  6457 raft_consensus.cc:1275] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
W20260709 12:28:40.061358 15352 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:28:40.061699  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:39.990303 15291 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.053625  6457 pending_rounds.cc:85] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 12
I20260709 12:28:39.932013 15351 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.047212 15322 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:40.010848 15118 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:40.065742  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 167508us
I20260709 12:28:40.069612  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:40.065048  6457 pending_rounds.cc:107] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:28:39.993625 15296 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.075248  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.084617  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.085318  6309 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:40.084172 15118 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.086213  6378 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:40.087894  6378 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:40.088207 15118 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.088213 15351 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:40.088936 15177 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:40.091328 15177 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:40.092274 15177 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:40.092644 15177 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:40.093791 15177 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.097939  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:40.099105  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:40.066143 15245 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:28:40.100067  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:40.101056 15292 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:40.101212  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.101584 15292 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.102053 15292 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:40.104750  6458 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:40.105682 15180 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.106377 15303 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:40.106614 15013 consensus_meta.cc:287] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.000s
I20260709 12:28:40.096527 15177 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:40.107071 15303 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.110327 15296 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.110813  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.111714 15351 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:40.112123 15351 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.112411  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.112685  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.113059  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.113459  6310 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:28:40.101085 15245 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:40.114303 15245 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:40.114385  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.114832 15245 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:40.115254  6304 raft_consensus.cc:2468] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 19.
I20260709 12:28:40.116293  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:40.116855  6301 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:28:40.116935  7316 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:40.118921  6308 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:40.119253  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.119381  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:40.120031  6307 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:40.120685 15291 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:40.121133 15291 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.121531 15291 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:40.121559 15351 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:40.122756  6305 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: [21.12-23.13]
I20260709 12:28:40.122860  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:40.123227  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.124038  6309 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:40.124135  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:40.113869  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:40.125864  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:40.121156  7329 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:28:40.128683  7285 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:40.113406 15210 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:40.131454 15210 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.131961 15210 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:40.130748 15299 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:40.133021 15299 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:40.133507 15299 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:40.135670  6198 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:40.137444  6198 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 21, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 21 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:40.141712 15339 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.551s	user 0.002s	sys 0.000s
I20260709 12:28:40.143273  6382 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:40.143478  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:40.144537  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.145795  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:39.922503  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:40.152366  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.152449 15322 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.153231  6377 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:40.153704 15322 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:40.153846 15342 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:40.155923 15342 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:40.157351 15289 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:40.158656 15289 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:40.159029 15289 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:40.144563 14254 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.160825 15305 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:40.110951 15143 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:40.167852  6457 raft_consensus.cc:2889] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:40.130110  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:40.090306  6378 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:40.130394  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 275) took 38104 ms (client timeout 15000 ms). Trace:
I20260709 12:28:40.146385  6382 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
W20260709 12:28:40.202608  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:40.204625  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:40.167305  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68337.4us
I20260709 12:28:40.118240  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:40.212988 14254 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.211648  7316 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:40.215996  6383 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:40.218798 14254 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:40.175699  6202 rpcz_store.cc:269] 0709 12:28:02.026046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:07.641576 (+5615530us) service_pool.cc:224] Handling call
0709 12:28:40.130110 (+32488534us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:29.343068 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:29.343759 (+   691us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:29.343830 (+    71us) write_op.cc:435] Acquired schema lock
0709 12:28:29.343874 (+    44us) tablet.cc:662] Decoding operations
0709 12:28:29.394477 (+ 50603us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:29.410738 (+ 16261us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:29.410775 (+    37us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:28:29.438002 (+ 27227us) tablet.cc:701] Row locks acquired
0709 12:28:29.438063 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:28:29.438330 (+   267us) write_op.cc:270] Start()
0709 12:28:29.438551 (+   221us) write_op.cc:276] Timestamp: P: 1783600109438238 usec, L: 0
0709 12:28:29.438612 (+    61us) op_driver.cc:348] REPLICATION: starting
0709 12:28:29.456901 (+ 18289us) log.cc:844] Serialized 7330 byte log entry
0709 12:28:30.220801 (+763900us) op_driver.cc:464] REPLICATION: finished
0709 12:28:30.454948 (+234147us) write_op.cc:301] APPLY: starting
0709 12:28:30.455073 (+   125us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:30.706294 (+251221us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:30.836433 (+130139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:31.400407 (+563974us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b180023a160. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba522 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:31.413200 (+ 12793us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:31.413634 (+   434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=0,mrs_lookups=0
0709 12:28:31.413679 (+    45us) write_op.cc:312] APPLY: finished
0709 12:28:31.513355 (+ 99676us) log.cc:844] Serialized 172 byte log entry
0709 12:28:31.678990 (+165635us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:31.700778 (+ 21788us) write_op.cc:454] Released schema lock
0709 12:28:31.711209 (+ 10431us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:31.711740 (+   531us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:31.711772 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15495,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":219353,"apply.run_cpu_time_us":42258,"apply.run_wall_time_us":1256754,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83585,"num_ops":19,"prepare.queue_time_us":154039,"prepare.run_cpu_time_us":13588,"prepare.run_wall_time_us":161281,"raft.queue_time_us":118561,"raft.run_cpu_time_us":21263,"raft.run_wall_time_us":221307,"replication_time_us":742717,"spinlock_wait_cycles":9206656,"thread_start_us":330525,"threads_started":5,"wal-append.queue_time_us":379507,"wal-append.run_cpu_time_us":9866,"wal-append.run_wall_time_us":2644263}]]}
I20260709 12:28:40.285485  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:40.214784 15162 consensus_meta.cc:287] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.005s
I20260709 12:28:40.290132  6451 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:28:40.212407 15301 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:40.209749 15303 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.253790  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:40.291167  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.311582  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.312739 14254 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.315455  6382 raft_consensus.cc:2368] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20.
I20260709 12:28:40.312218  6378 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:40.315439  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:40.319375  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 108921us
I20260709 12:28:40.291667 15118 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:40.328400  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.329272 14254 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:40.330572  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:40.331550  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:40.314505  6198 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 20, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 20 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:40.308530 15301 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:40.217724 15350 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.332847  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.353510  7316 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:28:40.344666  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:40.354916  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:40.355517  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.356036  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.360299  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.307179 15162 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.345247 15323 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:40.372414  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.376499  6450 raft_consensus.cc:2368] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:28:40.354885 15301 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:40.359936  6198 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:40.376075  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3812) took 1316 ms. Trace:
I20260709 12:28:40.379415 15328 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:40.379304  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.363502 15338 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:40.382872  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.383576 15045 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:28:40.384621  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.373154 15303 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:40.387223  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 302) took 17909 ms (client timeout 15000 ms). Trace:
I20260709 12:28:40.401701  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:40.376860 15212 consensus_meta.cc:287] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.003s
I20260709 12:28:40.395334  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.382647 15142 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:40.398135  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:40.405270  6424 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:40.406425  6202 rpcz_store.cc:269] 0709 12:28:22.478052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.363168 (+17885116us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:40.387101 (+ 23933us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:28:40.389107 15162 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:40.403964  6307 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
W20260709 12:28:40.410382  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 280388us
W20260709 12:28:40.401207 15336 consensus_meta.cc:287] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.000s
I20260709 12:28:40.418046  6377 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:40.412983  6378 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 29.
I20260709 12:28:40.397634  6452 rpcz_store.cc:276] 0709 12:28:39.015883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.016015 (+   132us) service_pool.cc:224] Handling call
0709 12:28:39.016338 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:39.016723 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:39.016756 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:39.023443 (+  6687us) log.cc:844] Serialized 26 byte log entry
0709 12:28:39.475102 (+451659us) spinlock_profiling.cc:243] Waited 1.29 ms on lock 0x7b180025c960. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:39.535263 (+ 60161us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:39.535376 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:39.535534 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:40.140844 (+605310us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:40.319806 (+178962us) raft_consensus.cc:1685] finished
0709 12:28:40.319830 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:40.332415 (+ 12585us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14382,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":5491,"spinlock_wait_cycles":18690560,"thread_start_us":59373,"threads_started":1,"wal-append.queue_time_us":554229}
I20260709 12:28:40.417629  6198 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 19, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:40.422900  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57988.6us
I20260709 12:28:40.426913  6305 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:40.411329 15303 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.411757 15142 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.436511  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.441651  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.379343 15319 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:28:40.449717  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.450621  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.444671 15337 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:40.451473 15337 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:40.452023 15337 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:40.419343 15045 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:40.377866 15118 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.451508  6377 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:40.419582  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 164724us
I20260709 12:28:40.455471  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.457434  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.448865  6198 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:40.456131  6305 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:40.469427 15299 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.472244 15299 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:40.475004  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:40.475991  7335 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:40.479092 15318 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:40.475152  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.479779 15318 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:40.480018  6305 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:40.422209 15328 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:28:40.444977 15303 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:40.481161  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.446455 15142 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:40.481491  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.482288  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.499511  6305 raft_consensus.cc:2410] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 33 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 21.
I20260709 12:28:40.501353  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.501823  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.422508 15336 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.510627  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.513547  6308 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
W20260709 12:28:40.512228  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 167955us
I20260709 12:28:40.520766  6307 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:40.523115  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.524650  6377 raft_consensus.cc:2393] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:40.526095  6351 connection.cc:256] RPC call timeout handler was delayed by -1.0344418509998832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2396s for a response
I20260709 12:28:40.528945  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:40.532145  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.532766  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.534757  6305 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:40.535964  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.536135 15304 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:40.536599  6305 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:40.535530  6198 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 22 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:40.494829 15299 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:28:40.480902 15318 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.542676  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.534871  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.543501  6385 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:40.546013  6198 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:28:40.546820  6274 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.573s (SENT)
I20260709 12:28:40.568341  6307 raft_consensus.cc:2410] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
W20260709 12:28:40.570662  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 127256us
I20260709 12:28:40.595858  6382 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:40.596879  6308 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:40.595934  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.597499  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:40.536633 15304 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:40.561632 15278 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:40.603371  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 166829us
I20260709 12:28:40.612252  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.613138  6307 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 30 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:28:40.621848  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:40.623589  6310 consensus_meta.cc:287] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.005s
I20260709 12:28:40.625523  6308 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:40.625547  6310 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:40.621632  6198 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 24. New cstate: current_term: 24 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.631701  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:40.627961  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 86623.6us
W20260709 12:28:40.635151  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95724.5us
I20260709 12:28:40.621508 15013 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.642345  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.643005  6303 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:40.644884  6382 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:40.644327 15362 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.054s	user 0.001s	sys 0.001s
I20260709 12:28:40.660806  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:40.566390 15045 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.662241  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:40.663764  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.636566 15162 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:40.641090 15008 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.627952 15292 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.627274 15278 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.654402 15338 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:40.681296  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:40.685529  6385 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25. Ops: []
W20260709 12:28:40.677248 15305 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.001s
I20260709 12:28:40.630162 15289 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.680528  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.670888 15304 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.701206 15304 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:40.696123 15336 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:40.705341 15301 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.684624 15008 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.709095 15008 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.709802 15008 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:40.696707 15371 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:40.711692 15371 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:40.713215 15371 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.716393  6380 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:40.692203  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51854 (request call id 3420) took 1094 ms. Trace:
I20260709 12:28:40.698580  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57670 (request call id 3648) took 3395 ms. Trace:
I20260709 12:28:40.765429  6300 rpcz_store.cc:276] 0709 12:28:37.227244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:37.230424 (+  3180us) service_pool.cc:224] Handling call
0709 12:28:40.584580 (+3354156us) spinlock_profiling.cc:243] Waited 1.47 s on lock 0x7b5400276854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:40.622825 (+ 38245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3369652096}
I20260709 12:28:40.718004  6376 rpcz_store.cc:276] 0709 12:28:39.599458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.666340 (+1066882us) service_pool.cc:224] Handling call
0709 12:28:40.692134 (+ 25794us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:40.772854  7329 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:40.775290  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:40.773804  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80610us
I20260709 12:28:40.779229  6377 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:28:40.780336  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.683065  6198 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 23, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:40.789438  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:40.790349  7317 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:40.790737  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:40.671063 15013 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.797113  6385 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:40.811540  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 147751us
I20260709 12:28:40.813715  6382 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:28:40.680398 15373 consensus_queue.cc:1048] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.121s
I20260709 12:28:40.820027  6382 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:28:40.815769  6198 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:40.785370 15371 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:28:40.808317 15322 consensus_meta.cc:287] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.643s	user 0.001s	sys 0.014s
I20260709 12:28:40.719033 15008 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:40.811334 15364 log.cc:927] Time spent T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:28:40.826503  6382 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:40.798308 15118 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.831521  6310 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:40.830698  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 165204us
I20260709 12:28:40.695633 15338 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.809592 15162 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:40.825587 15291 consensus_meta.cc:287] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.703s	user 0.001s	sys 0.011s
I20260709 12:28:40.846745  6303 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:40.851536  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:40.854084  6198 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 22 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:40.860646  7316 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:40.836014 15118 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.876266  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.858124 15006 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.894724  6309 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:40.894557 15006 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.895987  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:40.889010 15366 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.151s	user 0.001s	sys 0.001s
I20260709 12:28:40.856511 15291 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.907034  6198 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 19, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:40.922423 15387 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:28:40.922775 15006 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:40.924032  6309 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:28:40.929265 15381 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:28:40.944075  6378 raft_consensus.cc:2368] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:40.952358  6380 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:40.896261 15118 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:40.953708  6380 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:40.967316  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.971096  6384 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:40.973094  6384 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:40.943931 15390 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:40.977281  6309 raft_consensus.cc:2376] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22. Re-sending same reply.
I20260709 12:28:40.988852  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:40.983886  6198 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:40.993242 15008 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.002954 15008 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.001031 15390 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.003669 15008 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.006088 15008 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.975741 15301 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:41.017464 15375 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:28:40.948886 15162 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:40.991509 15013 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:40.994433 15289 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.028744  6198 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 19, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:41.032485  6380 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
W20260709 12:28:41.033082  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.041042  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.015780 15008 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.043035 15008 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.044585  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3419) took 1492 ms. Trace:
I20260709 12:28:41.045054 15008 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.045018  6379 rpcz_store.cc:276] 0709 12:28:39.552588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.565786 (+ 13198us) service_pool.cc:224] Handling call
0709 12:28:39.566107 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:39.806088 (+239981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.044329 (+1238241us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54003d27d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:41.044497 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":549640448}
W20260709 12:28:41.045964 15210 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.913s	user 0.009s	sys 0.001s
I20260709 12:28:41.046152  6198 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 24, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 24 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:41.047901  6198 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 24, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 24 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:41.049923  6198 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 19, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:41.052456  6198 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 22 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:28:41.057137 15351 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.935s	user 0.008s	sys 0.005s
I20260709 12:28:41.057670 15278 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.058087  6198 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 28, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 28 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:41.060056  6459 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:41.060009  6198 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:41.060832  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 3962) took 1195 ms. Trace:
W20260709 12:28:41.062306 15379 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 12:28:41.063378  6450 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3961) took 1234 ms. Trace:
I20260709 12:28:41.064249  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.065047  7331 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 33.
I20260709 12:28:41.063795  6450 rpcz_store.cc:276] 0709 12:28:39.829056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.871049 (+ 41993us) service_pool.cc:224] Handling call
0709 12:28:40.143235 (+272186us) spinlock_profiling.cc:243] Waited 2.34 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa826d0a0 00007f6fa826c984 00007f6fa17e76d4 00007f6fa17e7662
0709 12:28:41.063156 (+919921us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b54002c7c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:41.063311 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529739264}
I20260709 12:28:41.006295 15395 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.061s
I20260709 12:28:41.067334 15301 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:41.068174 15118 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.067051  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:41.061550  6198 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 18, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 18 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:41.061247  6459 rpcz_store.cc:276] 0709 12:28:39.865925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.875040 (+  9115us) service_pool.cc:224] Handling call
0709 12:28:41.059429 (+1184389us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:28:41.059655 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:41.060564 (+   909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.060770 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5318400}
W20260709 12:28:41.092476  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 64289.1us
I20260709 12:28:41.094190  6384 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:41.092751  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.092557  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.100741  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:41.101589  6384 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:41.102875  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:41.064193 15338 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.107012  7329 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 33.
I20260709 12:28:41.110669  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:41.116518 15337 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.664s	user 0.005s	sys 0.014s
I20260709 12:28:41.116986 15337 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.118633 15006 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:41.119017  6455 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:41.124205 15292 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:41.132287  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 91415.7us
I20260709 12:28:41.095537 15289 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:41.073359  6194 maintenance_manager.cc:643] P 08c731ab128f4ef881e4c3c73ff48bd8: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 1.831s	user 0.116s	sys 0.051s Metrics: {"cfile_cache_miss":7,"cfile_cache_miss_bytes":25633,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":322875,"dirs.run_cpu_time_us":759,"dirs.run_wall_time_us":30421,"lbm_read_time_us":2125,"lbm_reads_lt_1ms":23,"lbm_write_time_us":2255,"lbm_writes_lt_1ms":11,"peak_mem_usage":277331,"reinsert_count":0,"thread_start_us":565022,"threads_started":3,"update_count":225,"wal-append.queue_time_us":248788,"wal-append.run_cpu_time_us":776,"wal-append.run_wall_time_us":1034785}
I20260709 12:28:41.138569 15323 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:41.139329 15323 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:41.057709 15351 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:41.143517 15351 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.144641 15338 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:41.111732 15390 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.101958 15206 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.151211 15206 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.154057 15206 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.108486 15392 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.156962 15392 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.157641 15337 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:41.159829 15303 consensus_meta.cc:287] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.644s	user 0.005s	sys 0.006s
I20260709 12:28:41.160277 15303 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:41.161613 15142 consensus_meta.cc:287] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.001s
I20260709 12:28:41.162000  6307 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
W20260709 12:28:41.163091  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53411.1us
I20260709 12:28:41.046422 15210 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:41.166448  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62883.1us
I20260709 12:28:41.165824 15303 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.175400  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.171207 15373 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.190006  6459 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:41.198568 14254 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:41.167269 15006 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:41.193189  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:41.204726 14254 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:41.205117 14254 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:41.167181 15289 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:41.209619 15118 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:41.209379 15006 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.209789 15403 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:28:41.210175 15118 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.210624 15118 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:41.210784  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.215454 15291 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:41.216522  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.209657  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:41.221252  6384 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:41.206593 15401 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:28:41.239902 14254 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.240777  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.241438 14254 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:41.242790 15008 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:41.244772 15008 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:41.246125 15403 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:41.246512 15403 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:41.247620 15403 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.249253 15403 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:41.172072 15142 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:41.250721  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 68131.6us
I20260709 12:28:41.252986  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.253706 15142 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.254442  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.255481  6453 raft_consensus.cc:2393] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:41.259744 15210 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.261868 15401 raft_consensus.cc:2764] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
W20260709 12:28:41.268476  6274 leader_election.cc:336] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.664s (SENT)
I20260709 12:28:41.269019  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.269070 15118 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.269675 15323 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.271004 15392 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.272274 15118 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.273064 15118 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:28:41.276506  6456 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 24. Ops: []
I20260709 12:28:41.278402  6456 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:28:41.278805  6456 pending_rounds.cc:85] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 11
I20260709 12:28:41.279093  6456 pending_rounds.cc:107] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:28:41.279002  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.279536  6456 raft_consensus.cc:2889] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:41.279938  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.280987  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.281441  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.282255  6457 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:41.282500 15401 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.283609  6454 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:41.283215 15401 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.284678  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.285270  6458 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:41.285782 15401 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:41.287774 15245 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 1.170s	user 0.011s	sys 0.003s
I20260709 12:28:41.288311 15245 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.290745  7314 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:41.207562 15373 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:41.296083  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 86495.5us
W20260709 12:28:41.270300  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.280864  6452 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:41.294117 15323 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.310272 15402 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:41.310911 15402 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:41.311095 15388 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.490s	user 0.008s	sys 0.000s
I20260709 12:28:41.311316 15402 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:41.313390  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:41.314008  7329 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:41.315685  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 73197.3us
W20260709 12:28:41.319151  6451 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38272 (request call id 1566) took 1392 ms (client timeout 1519 ms). Trace:
W20260709 12:28:41.319593  6451 rpcz_store.cc:269] 0709 12:28:39.926410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:39.930087 (+  3677us) service_pool.cc:224] Handling call
0709 12:28:41.318903 (+1388816us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b54000c21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:41.319079 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":822109952}
W20260709 12:28:41.320785  6274 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.698s (SENT)
I20260709 12:28:41.321277  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.322837 15245 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.323285  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.324193  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:41.325137 15323 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.325649 15008 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:41.325815 15323 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.328888 15323 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.330749 15289 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.334017 15392 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.334568 15392 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.335628  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.336817  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.337595  6456 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:28:41.337510  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.339412  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.340430  6457 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:28:41.340481  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.341210 15142 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.325187  6308 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 30 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:28:41.341655 15142 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.318252  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:41.342239  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:41.349475  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.350117  6451 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:41.351454  6459 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:28:41.352133  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 99266.6us
I20260709 12:28:41.354804  6452 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:41.355449 15342 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.359072 15342 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.361860  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.363077 14254 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:41.363564 14254 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:41.363977 14254 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:41.365854  7329 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 34.
W20260709 12:28:41.370088  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.362581  6457 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:41.366959  6275 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:41.380044  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.352910  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.383486  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.385418  6453 raft_consensus.cc:2393] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 28: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:41.339118 15392 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.387624  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:41.388357  7316 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:41.389020  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.390295  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:41.398342  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 75131us
I20260709 12:28:41.355916 15013 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:28:41.415218  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52690us
I20260709 12:28:41.416241  7320 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:41.393342  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:41.398705  7287 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.419024  6348 leader_election.cc:400] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29.
I20260709 12:28:41.428028  6348 leader_election.cc:403] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:28:41.430789  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:41.386983 15401 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:41.387066 15118 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.393036 15371 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:41.393056 15303 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:41.437927  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46256us
I20260709 12:28:41.415166 15045 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:41.443439  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:41.421044 15342 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.477780  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.489795  7329 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 21.
I20260709 12:28:41.421299 15013 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:41.431952 15299 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:41.443401  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.487008 14254 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:41.492992  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52273.9us
I20260709 12:28:41.434672 15323 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:41.438297 15401 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.497835  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:41.441259 15392 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:41.499279  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 181116us
I20260709 12:28:41.506383 14254 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:41.495369 15363 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:28:41.514858 14254 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:41.515900 14254 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:41.495464 15013 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:41.518530 14254 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:41.526898  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83567.3us
I20260709 12:28:41.522851 15006 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:41.477751 15371 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.440387 15118 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.498345 15278 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:28:41.524279 15323 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.524964 15159 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29.
I20260709 12:28:41.495014 15395 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:41.535086 15390 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:41.538921  6272 leader_election.cc:400] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 20. Current term is 21.
I20260709 12:28:41.540823 15392 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:41.509536 15401 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:41.523442  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.546051  6451 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:41.508167 15245 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.520277 15299 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:41.489462 15303 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.576565 15142 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.583885 15118 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:41.593837  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 226977us
I20260709 12:28:41.600538  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:41.610236  7316 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:41.564186 15342 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.568921 15392 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.624398 15402 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.631764 15303 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.634368  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3440) took 1528 ms. Trace:
I20260709 12:28:41.634842  6377 rpcz_store.cc:276] 0709 12:28:40.105716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.587712 (+481996us) service_pool.cc:224] Handling call
0709 12:28:40.600525 (+ 12813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:40.779845 (+179320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.634104 (+854259us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54003f25d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:41.634304 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395807616}
W20260709 12:28:41.625595 15368 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.007s	user 0.001s	sys 0.006s
I20260709 12:28:41.643091  6302 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3310) took 1528 ms. Trace:
I20260709 12:28:41.643705  6302 rpcz_store.cc:276] 0709 12:28:40.164034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.164631 (+   597us) service_pool.cc:224] Handling call
0709 12:28:40.164999 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:40.165378 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:40.165405 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:40.165913 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 12:28:40.415054 (+249141us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:40.415123 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:40.415228 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:41.624961 (+1209733us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:41.642730 (+ 17769us) raft_consensus.cc:1685] finished
0709 12:28:41.642762 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:41.643012 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5151,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":6166784,"thread_start_us":241670,"threads_started":1,"wal-append.queue_time_us":365768}
I20260709 12:28:41.645164 15323 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.625893 15342 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.642755 15299 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.629643 15392 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:41.631834  7314 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1556) took 1516 ms. Trace:
W20260709 12:28:41.649238  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.651073  7314 rpcz_store.cc:276] 0709 12:28:40.115298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.115459 (+   161us) service_pool.cc:224] Handling call
0709 12:28:40.115790 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:41.291222 (+1175432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.631498 (+340276us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540043a3d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:41.631740 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701606144}
I20260709 12:28:41.570322 15245 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.614398 15395 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:41.658244  6459 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
W20260709 12:28:41.666723  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 178639us
I20260709 12:28:41.650997 15008 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:41.659540 15291 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:41.659533 15322 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.680661 15278 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:41.656703 15373 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.666059 15402 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:41.695151 15411 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.148s	user 0.001s	sys 0.001s
W20260709 12:28:41.714294  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 175586us
W20260709 12:28:41.716375  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.727025 15371 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.729624 15350 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:41.733227 15408 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:41.736461 15245 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.737598  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:41.739769  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:41.739984 15337 raft_consensus.cc:1064] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:28:41.740540 14254 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:41.740933 14254 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.741242 14254 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:41.741361  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:41.743455  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 305802us
I20260709 12:28:41.756827 15177 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:41.759872  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3446) took 1048 ms. Trace:
I20260709 12:28:41.703241 15322 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:41.764537  7288 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.618s (SENT)
I20260709 12:28:41.766196  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.760815  6380 rpcz_store.cc:276] 0709 12:28:40.699640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:41.039992 (+340352us) service_pool.cc:224] Handling call
0709 12:28:41.096792 (+ 56800us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:41.102172 (+  5380us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:41.401720 (+299548us) log.cc:844] Serialized 14 byte log entry
0709 12:28:41.408324 (+  6604us) log.cc:844] Serialized 14 byte log entry
0709 12:28:41.627999 (+219675us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b18004249c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa2e5dd70 00007f6fa2e51293 00007f6fa2e46cea 00007f6fa2e4723a 00007f6fa32b1a2a 00007f6fa32c10bf 00007f6fa32c1038
0709 12:28:41.747811 (+119812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:41.747868 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:41.747918 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.748001 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:41.748126 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27633,"prepare.queue_time_us":65343,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":71,"raft.queue_time_us":52280,"raft.run_cpu_time_us":272,"raft.run_wall_time_us":13146,"spinlock_wait_cycles":21636608,"thread_start_us":403224,"threads_started":2}
I20260709 12:28:41.742497 15342 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:41.785969 14254 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:41.787093  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 45824.7us
I20260709 12:28:41.794755  6272 leader_election.cc:400] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:28:41.758570 15402 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:41.788265  6274 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.795279  6272 leader_election.cc:403] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
W20260709 12:28:41.788942  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 265574us
I20260709 12:28:41.799523  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:41.802078  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:41.807776 14254 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:41.808468  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 148327us
I20260709 12:28:41.802052 15045 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 12:28:41.811625  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 264167us
I20260709 12:28:41.815431  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:41.842918 14254 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:41.842782  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48197.3us
I20260709 12:28:41.847818  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:41.851650  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.853405  6451 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:28:41.854671  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.799160 15422 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.044s
I20260709 12:28:41.850996  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:41.860261  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:41.861660  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.865025  6450 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
W20260709 12:28:41.866178  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101472us
W20260709 12:28:41.867473  7288 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.628s (SENT)
I20260709 12:28:41.811522 15426 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:41.868912  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.871739  6452 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:41.861032  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:41.874270  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:41.878170  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 90043.1us
I20260709 12:28:41.878448  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.879822  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.878039  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.882608  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.883919  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.884800  6452 raft_consensus.cc:2410] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:28:41.856956 15431 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:28:41.889683  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.892997  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.885967 15401 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.894217  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.903543  6459 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:41.856835 15402 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:28:41.910388 14254 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:28:41.919011 15008 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:41.919370  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51956.2us
I20260709 12:28:41.921424  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.922005  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.922139  7319 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:28:41.889343 15342 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:41.922953  7320 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:41.896338 15390 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.892103 15432 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:28:41.889134 15431 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:41.896561 15045 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:28:41.930071 15390 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.934017  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:41.934147 15013 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.935268 15396 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:41.837471 15337 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 9, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:41.938005  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:41.961530  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:41.962647  6300 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:28:41.963240  6300 pending_rounds.cc:85] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 14
I20260709 12:28:41.963581  6300 pending_rounds.cc:107] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:28:41.963863  6300 raft_consensus.cc:2889] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:41.962927  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:41.964641  6307 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:28:41.956945  7314 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:41.968263  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:41.927788  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:41.966020  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:41.974066  6301 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:41.927714 15433 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:41.975257  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49501.9us
I20260709 12:28:41.976130  7320 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:41.990506  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.992057  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.936641 15431 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:42.015786 15401 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:41.982895 15006 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:42.018774 15433 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.020079 15433 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:41.993011  6453 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:42.023667  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:42.024327  6453 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:42.028255  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.029265  6450 raft_consensus.cc:2410] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 23 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:28:41.984345  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:41.995200  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:41.990113  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:41.998271  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:41.999300  6456 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:28:42.037706  6308 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:28:42.036092  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.000124  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.001070  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.034442  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.047150  6303 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:42.010615 15434 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:42.011225 15389 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.074s
I20260709 12:28:41.943563 15396 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:42.062683  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79046.2us
I20260709 12:28:42.047266  6305 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:41.999385  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.008421 15390 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.049423  6454 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:42.030417  6450 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: []
W20260709 12:28:42.046800  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62533.5us
I20260709 12:28:42.054656  6307 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:42.065294  6302 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:42.033300  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:42.059826  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 69787us
I20260709 12:28:42.030309 15177 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:42.057578 15434 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:41.973596 15373 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:41.962567 15377 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:42.088364 15377 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.076962  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:42.090730  6309 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:28:42.091594  6309 pending_rounds.cc:85] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 13
I20260709 12:28:42.091907  6309 pending_rounds.cc:107] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 12:28:42.092239  6309 raft_consensus.cc:2889] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:42.083878  6452 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:41.974843 15402 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:41.996323 15432 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:41.997301 15395 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:42.074344 15006 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:42.090790  6459 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:42.032240 15403 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:42.094329  6377 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:28:42.098873  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.031877 15013 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:42.098865  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 130662us
I20260709 12:28:42.103227  6453 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:28:42.082073  6303 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:42.083925 15429 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:42.062099 15245 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:42.074065 15396 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:42.112937  6380 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:42.114362  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 221514us
I20260709 12:28:42.120525  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.116516  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.122712  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.076946 15434 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:42.117230  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:42.121553  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 278989us
I20260709 12:28:42.123184  6302 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:42.126724  6309 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:42.127415  6452 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:42.127342  6382 raft_consensus.cc:2393] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:42.129611  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.129972  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.130276  6377 raft_consensus.cc:3060] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:42.127211  6309 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:42.107661 15395 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.099678 15373 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.143283  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.143886  6381 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:42.145072 15373 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:42.146867 15425 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.271s	user 0.000s	sys 0.001s
I20260709 12:28:42.111905 15403 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:42.119830 15389 raft_consensus.cc:2764] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:28:42.121284 15245 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.113835 15432 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.153003  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.156790  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.172793  6452 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:42.123466 15006 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.183073  6380 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:42.183174  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.172179  6302 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:42.184501  6382 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:42.159365  6381 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:42.131793 15350 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:42.152716  6309 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.188931  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.162073  7314 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:42.162757  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:42.167375 15436 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:42.107534 15402 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.169021  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.178848  7320 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 34.
I20260709 12:28:42.150409 15443 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:42.187609  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.187964  6377 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:28:42.192518 15441 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 12:28:42.192852  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.152173 15395 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:42.211247  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.140797 15180 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:28:42.199483  6309 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:42.157501 15430 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:42.200165  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:42.183427 15245 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:42.133085 15427 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.062s
I20260709 12:28:42.228506  6303 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:42.230137  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.230474  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.230913  6302 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:42.231454  6307 raft_consensus.cc:2410] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 16.
I20260709 12:28:42.202649 15401 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.247380  6305 raft_consensus.cc:2468] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:28:42.249040  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 92646.3us
I20260709 12:28:42.268895  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.231739 15429 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:42.273219  6382 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:42.231096 15377 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.280345  6305 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
W20260709 12:28:42.281294  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76759.9us
W20260709 12:28:42.281708  6274 connection.cc:256] RPC call timeout handler was delayed by -1.5198573980001129s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
I20260709 12:28:42.274354 15429 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.298126  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.279546 15430 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:42.290716 15350 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.307099  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.247577 15431 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:42.311066 15042 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.295740 15377 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:28:42.311599 15042 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.314116 15042 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.316895  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.317545  6452 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:42.302129 15443 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.318521  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.319052 15443 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.319368  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:42.320274 15289 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:42.318608  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 33 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.321421  6379 raft_consensus.cc:2410] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 21.
I20260709 12:28:42.323000  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.320842 15289 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.324630 15289 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:42.326993 15415 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.645s	user 0.002s	sys 0.001s
I20260709 12:28:42.330052  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.330711 15443 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:42.333228  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94946.6us
I20260709 12:28:42.334339  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.335644  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.336215 15392 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:42.336635 15392 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.336931  6351 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.337702  6309 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:28:42.338057  6309 pending_rounds.cc:85] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 14
I20260709 12:28:42.337517 15392 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.338282  6309 pending_rounds.cc:107] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:28:42.338948 15392 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:42.343663  6309 raft_consensus.cc:2889] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:42.335561 15328 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:42.346315  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.341544 15402 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.271198 15403 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.289002 15006 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.291754 15432 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.343232  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.303274 15445 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:42.309072  6274 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.510s (SENT)
I20260709 12:28:42.349381  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.359381  6304 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:42.300253 15429 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:42.351414  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.351953 15444 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:42.346827 15328 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:42.352753 15291 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.353271 15433 raft_consensus.cc:2764] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
W20260709 12:28:42.354249 15446 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
W20260709 12:28:42.355830 15414 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.675s	user 0.004s	sys 0.000s
I20260709 12:28:42.356186  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.302938 15322 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:42.350185  7314 raft_consensus.cc:2393] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:42.387151  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.387948  6377 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:42.345971 15342 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
I20260709 12:28:42.390609  6377 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:28:42.363082  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.358301  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:42.363734  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.363942  6309 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29. Re-sending same reply.
I20260709 12:28:42.366091  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.369748  6379 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:42.375168  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.370874  7332 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:28:42.380328  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.396194  6305 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:42.396766  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.396682  6304 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:42.340282 15434 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.399925  6380 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:42.400105  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.401099  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.401418 15291 raft_consensus.cc:2764] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:28:42.406706  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:42.348233 15350 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.371054 15430 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:42.409837  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.410960  6304 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
W20260709 12:28:42.412739  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 54513us
I20260709 12:28:42.413901  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:42.359123 15013 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.040s
I20260709 12:28:42.415843  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.419675  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:42.420959  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 119414us
I20260709 12:28:42.421372  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.386335 15245 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.433629  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.415131 15430 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:42.408762 15428 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:42.417276 15402 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:42.417070 15322 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:42.446650  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.442914  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.449047  6304 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 30 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:28:42.447674  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.367918 15445 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:42.455475  6305 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:42.427932 15337 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:42.461853  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.462074  6379 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:42.466094  6303 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:42.468120  6307 raft_consensus.cc:2376] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23. Re-sending same reply.
I20260709 12:28:42.470225 15434 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.470355  6379 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:28:42.471032  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.471236  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.471511 15342 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:42.472429  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.473066  6304 raft_consensus.cc:2376] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29. Re-sending same reply.
I20260709 12:28:42.414997 15396 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.473855  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.479465  6305 raft_consensus.cc:2410] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:28:42.480813  6451 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
W20260709 12:28:42.479758  6274 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.193s (SENT)
I20260709 12:28:42.483402  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.475042  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.451139 15428 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.446645 15430 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:42.477484 15350 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.486888  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.481423  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:42.488253 15006 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.488729  6453 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:42.488833 15006 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.490437  6302 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 30 because replica has last-logged OpId of term: 29 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:28:42.491194 15006 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:42.492707  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.493777 15428 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.495882  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.474915  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.498569  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.498869  6304 raft_consensus.cc:2393] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:42.499388  6452 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:42.499857  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.500728  6424 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:42.501519 15342 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:42.501921 15447 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.224s	user 0.002s	sys 0.000s
I20260709 12:28:42.502220 15342 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:42.502628  6307 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 21. (term mismatch)
I20260709 12:28:42.502754 15342 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.503084  6307 pending_rounds.cc:85] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 20
I20260709 12:28:42.503829  6307 pending_rounds.cc:107] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 29.21
I20260709 12:28:42.504079  6307 raft_consensus.cc:2889] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:42.504009 15342 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:42.506217  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.434595 15328 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.507773 15118 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.508376 15328 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:42.485160 15371 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:42.510298 15371 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.512428  6424 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.484265  6303 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:42.478530  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.449954 15322 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.505339 15457 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:42.498606  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.514837  6310 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:42.483870  6309 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:42.435458 15389 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:28:42.519761  6300 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:28:42.527371 15118 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:42.528014 15118 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.522593  7332 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:42.529464  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3826) took 1368 ms. Trace:
I20260709 12:28:42.533666  6379 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:42.535056  6384 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:28:42.535975  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.536871  6384 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:42.466352 15337 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.538018  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:42.538507 15337 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:42.538717  6459 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:28:42.546036  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.546641  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.547655  6382 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:42.549453  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.549340  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.554030  6459 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:42.557375  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.557847  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.558308  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.558164  6381 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:42.561829  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.562680  6380 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:28:42.529929  6455 rpcz_store.cc:276] 0709 12:28:41.161428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:41.166421 (+  4993us) service_pool.cc:224] Handling call
0709 12:28:41.190956 (+ 24535us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:41.254688 (+ 63732us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:41.254735 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:41.255477 (+   742us) log.cc:844] Serialized 52 byte log entry
0709 12:28:41.547140 (+291663us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:41.547213 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:41.547317 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:42.355241 (+807924us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:42.520334 (+165093us) raft_consensus.cc:1685] finished
0709 12:28:42.520358 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:42.529393 (+  9035us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4678,"raft.queue_time_us":45502,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"spinlock_wait_cycles":4088064,"thread_start_us":291142,"threads_started":1,"wal-append.queue_time_us":425281}
I20260709 12:28:42.567950  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:42.569248  6303 raft_consensus.cc:2410] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:28:42.575031  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.577313  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.579087  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:42.521302 15322 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:42.579831  6383 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:42.522708 15350 raft_consensus.cc:2804] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:42.582093  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:42.582870  6459 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 32.
I20260709 12:28:42.583051  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:42.547415 15006 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:42.589021  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 120688us
I20260709 12:28:42.589784  6456 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:42.585381  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.547533 15457 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.601194  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:42.560513 15210 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:28:42.606307  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:42.607250  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:42.613976  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 147684us
I20260709 12:28:42.586619 15434 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:42.586937 15431 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.621601  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.578904 15427 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:28:42.605121 15323 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:42.634943  6275 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.598033 15245 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.607131 15442 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.630909  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.590910 15291 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:28:42.604841 15350 raft_consensus.cc:697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.644932 15350 consensus_queue.cc:237] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.602382 15006 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.641270  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.649684  6303 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29.
I20260709 12:28:42.643415 15454 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.122s
I20260709 12:28:42.643704 15291 raft_consensus.cc:3055] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:42.645398 15177 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:42.591138 15433 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:28:42.616238 15401 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:42.624470  6383 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:28:42.619650 15377 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
W20260709 12:28:42.655169  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97411.1us
I20260709 12:28:42.618646 15210 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.691632  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.691154 15210 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.692490 15450 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:42.693181 15450 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.661202  6457 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:42.693604 15450 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:42.697974  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:42.691325 15323 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 12:28:42.630191 15443 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:28:42.703007 15323 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:42.664217  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.704361 15342 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:42.666579  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.678994  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.705899  6310 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:42.705698 15389 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:42.648901 15006 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.679468 15118 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.679733 15403 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:42.682850 15457 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:42.687994 15459 log.cc:927] Time spent T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:28:42.688861 15396 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.649490 15212 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:28:42.630764 15278 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:42.703469 15443 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:42.704775 15342 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.711784  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.732270  6303 raft_consensus.cc:2376] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21. Re-sending same reply.
I20260709 12:28:42.734745  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.735584  6303 raft_consensus.cc:2410] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:28:42.660621 15291 raft_consensus.cc:740] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.719242 15401 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.694053 15289 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.751917  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.752930  6310 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:42.754362  6303 raft_consensus.cc:1240] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
W20260709 12:28:42.742384  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 157105us
W20260709 12:28:42.743115  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 170855us
I20260709 12:28:42.745055  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:42.746522  6309 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:28:42.751888  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.757418  7314 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:42.756208  6302 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:42.730407 15434 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:42.731176 15443 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:42.763638  7314 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:42.756222  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:42.750175  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:28:42.773435  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:42.778100  7332 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:42.780247  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.748464 15278 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.732288 15177 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:28:42.730870 15395 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.786358  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.784392  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:42.795037  7314 raft_consensus.cc:2393] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 33: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:42.761276 15434 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:42.803236  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:42.800372  6310 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:42.801575  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.806385  6309 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:42.760339 15402 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:28:42.811985  6384 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:42.814248  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:42.814937  6310 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:42.783360 15466 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:42.808199  7287 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:42.818713  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:42.823565  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 197574us
I20260709 12:28:42.825006  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.798686 15445 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.804085 15427 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.844496  6310 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:42.846803  6310 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:42.849884  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.855842  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.858968  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.789536 15430 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:42.870769  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65304.9us
I20260709 12:28:42.826906 15428 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:42.874429  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.874516 15450 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.876384  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:42.827695 15465 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:42.882835 15486 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:28:42.861892 15350 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:42.887728 15473 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:42.834559 15429 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.787588 15342 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.888466 15482 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:42.888492 15456 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:28:42.889423 15480 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:42.810096 15245 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.057s
I20260709 12:28:42.895679  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:42.810303 15337 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.887607 15395 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.824543 15443 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.910118 15328 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:42.873983 15401 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:42.911937 15430 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.912472 15468 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:28:42.897045 15402 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:28:42.913959 15289 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:42.811406 15291 consensus_queue.cc:260] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.916592 15450 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:42.953414 15450 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:42.953847 15450 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:42.954236 15450 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:42.955672 15337 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:42.956525  6458 raft_consensus.cc:2368] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:28:42.946988 15473 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:42.958158 15473 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.917498 15428 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:42.958511 15473 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:42.958690 15445 consensus_queue.cc:1059] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:28:42.920555 15013 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:42.909157  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:42.862030 15466 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:42.902024 15486 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:42.907977 15350 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:42.967949  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 58866.4us
I20260709 12:28:42.972626  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:42.947894 15392 consensus_meta.cc:287] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.002s
I20260709 12:28:42.957131 15482 consensus_queue.cc:1059] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:28:42.983814  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88217.7us
I20260709 12:28:42.946856 15245 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:42.892359 15278 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:42.968814 15013 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:42.973500 15480 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:42.976565 15350 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:43.018206  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.020046  6382 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:43.008385  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:43.008596  6310 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:43.012845 15429 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.001935 15177 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:43.044880  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:43.046181  6310 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:43.002424 15013 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:43.041859 15444 consensus_meta.cc:287] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.001s
I20260709 12:28:43.058861  6308 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:43.060988  6310 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:43.067345  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59070us
I20260709 12:28:43.028441 15480 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:43.074172  6382 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:43.078892  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:43.045879 15177 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:43.075809  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:43.076110 15468 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
W20260709 12:28:43.078248 15463 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.154s	user 0.000s	sys 0.003s
W20260709 12:28:43.098285  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.074046 15337 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:28:43.127961  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.072634 15444 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:43.130004  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:43.083398 15461 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.113s	user 0.000s	sys 0.003s
I20260709 12:28:43.137378  6457 raft_consensus.cc:2410] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
W20260709 12:28:43.098752 15476 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:28:43.085397 15177 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:43.097689 15322 consensus_meta.cc:287] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.503s	user 0.000s	sys 0.011s
I20260709 12:28:43.103863 15377 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:43.099196 15494 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.102246 15371 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.094533 15484 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:43.157348  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:43.158389  6452 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:43.148149  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.150108  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:43.151855  6376 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:43.103576 15499 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:43.152762  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.160224  6457 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:43.160612  6379 raft_consensus.cc:2368] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:43.132618 15337 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:43.164981  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.054031 15428 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.163796  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.166813  6380 raft_consensus.cc:2368] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
W20260709 12:28:43.166857  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.054805 15486 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.169988  6382 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:43.153878 15377 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.131995 15210 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:28:43.094720 15485 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.160s
I20260709 12:28:43.096202 15490 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.093s
I20260709 12:28:43.176656  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.161484 15484 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:43.182360  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.182111  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.167906 15337 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:43.190178 15453 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.729s	user 0.002s	sys 0.000s
I20260709 12:28:43.192355 15484 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:43.152089 15322 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.209314  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.214154  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.222102  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:43.222815  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:43.223701  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 147983us
I20260709 12:28:43.200160  6272 leader_election.cc:400] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:28:43.227562  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:43.228690 15483 log.cc:927] Time spent T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:28:43.231124  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:43.232079  6305 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 31 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 20.
I20260709 12:28:43.234714 15322 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:43.236925  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.237550  6385 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:43.238009  6459 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 19. Current term is 20. Ops: []
I20260709 12:28:43.084372 15466 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.226975  6272 leader_election.cc:403] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:43.242951  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.243331  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.244789  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:43.213227 15480 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.175032 15377 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:43.252003  6380 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:28:43.206064 15013 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:43.250664  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50726.9us
I20260709 12:28:43.251044  6452 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:43.253346  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.255208  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.256254  6304 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:43.259548  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.260211  6379 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:28:43.256801  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.260574  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.262032  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.263332  6274 leader_election.cc:400] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:43.264041  6274 leader_election.cc:403] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:43.263525  7319 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:43.261080  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.267771  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.268541  6301 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:43.269940  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:43.261621  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:43.270841  7332 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:43.250511  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.272192  6382 raft_consensus.cc:2368] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:28:43.272104  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.273203  6380 raft_consensus.cc:2393] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 20: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:43.162871 15494 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.275451 15499 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:43.256626 15485 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:43.251691 15486 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:43.305598  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:43.305982  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44456.6us
I20260709 12:28:43.306895  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.273370  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.311089  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.310734  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.313647  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.278620 15389 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:43.275610 15482 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:43.315156  6380 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:43.318297  6380 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:43.315174  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51976us
I20260709 12:28:43.316538  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:43.320350  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.321357  6351 leader_election.cc:400] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29.
I20260709 12:28:43.321080  6382 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 28: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:43.323357  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.319064  6380 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.324445  6380 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:43.326114  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3454) took 1208 ms. Trace:
I20260709 12:28:43.308147 15488 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 23. Current term is 24.
I20260709 12:28:43.330526 15488 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:43.330667 15469 consensus_queue.cc:1175] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 34 status { last_received { term: 31 index: 21 } last_committed_idx: 21 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 33. Current term is 34. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:43.331422 15491 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:43.332214 15504 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:43.332635 15504 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:43.332979 15504 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:43.335202  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.335801 15434 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.336534  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.337052 15434 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:43.338835 15501 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:43.339567 15501 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:43.330139  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.341058  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:43.342089  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.342801 15500 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:43.343559 15500 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:43.345140 15177 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.338408  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.351082  7332 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:43.353524  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:43.317953 15389 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:43.327641  6378 rpcz_store.cc:276] 0709 12:28:42.118095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:42.118233 (+   138us) service_pool.cc:224] Handling call
0709 12:28:42.140162 (+ 21929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:42.189219 (+ 49057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:42.189253 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:42.198111 (+  8858us) log.cc:844] Serialized 26 byte log entry
0709 12:28:42.357715 (+159604us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:42.357759 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:42.357824 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:43.189543 (+831719us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:43.319671 (+130128us) raft_consensus.cc:1685] finished
0709 12:28:43.319694 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:43.326055 (+  6361us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26792,"raft.queue_time_us":40125,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":6449,"spinlock_wait_cycles":15630592,"thread_start_us":152853,"threads_started":1,"wal-append.queue_time_us":225256}
I20260709 12:28:43.352655 15322 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:43.278026 15499 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.358130  6301 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
W20260709 12:28:43.369586  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.313205 15444 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.037s
I20260709 12:28:43.362195  6382 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:43.362617  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:43.355265 15278 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:43.374193  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.363113 15389 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:43.382222  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:43.387742  6380 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:28:43.328905 15473 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.443938 15488 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:43.426111 15504 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.394642 15490 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:28:43.449501 15488 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:43.403905  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 65554.9us
I20260709 12:28:43.371701 15511 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:43.456547  6305 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:28:43.458966  6351 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:43.455087  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 131797us
I20260709 12:28:43.463075  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.459976  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.394819 15210 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:43.400951 15322 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:28:43.397464 15466 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:43.468053  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:43.475286  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 76014.9us
I20260709 12:28:43.460592 15444 consensus_queue.cc:1059] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 12:28:43.484520  6274 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.476924  6385 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 34.
W20260709 12:28:43.470054 15505 log.cc:927] Time spent T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 12:28:43.499823  6275 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 1.835s (SENT) [suppressed 42 similar messages]
W20260709 12:28:43.494243 15502 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:28:43.494696 15501 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:28:43.515126 15501 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:43.516355  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.516126 15501 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:43.506712  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.517652  6300 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:43.509164  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.512832 15402 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:43.518844 15485 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:28:43.519330 15473 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.518987 15402 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.519996 15515 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:43.521765 15402 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.472003 15431 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:43.525749  6307 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:43.469595 15177 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:43.489279 15494 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.525532 15350 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.529587  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.487249 15504 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:43.491711 15118 raft_consensus.cc:3055] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:43.488672 15488 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.462455 15507 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:28:43.493665 15491 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:43.466080 15210 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:43.531008  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 46595.5us
I20260709 12:28:43.548221  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:43.549530  6310 raft_consensus.cc:1275] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:43.549629 15518 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.542455 15118 raft_consensus.cc:740] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:43.539947 15322 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:43.552055 15444 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:43.563067 15210 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:43.582784 15450 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.628s	user 0.009s	sys 0.003s
I20260709 12:28:43.565846  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.569370  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.569626  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.570703  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:43.571707  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:43.596460  6385 raft_consensus.cc:2393] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 28: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:43.597158  6382 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:43.598037  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.572431  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:43.564378 15350 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:43.577306 15006 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:43.579480  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.609297  6310 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:43.607357  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.610733  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.608518 15402 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:43.635963  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:43.636912 15515 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:43.637635 15515 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:43.637981 15515 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.638310 15515 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:43.640504  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.641011  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.641921  6305 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:43.608194 15396 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:43.644678 15402 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:43.645763 15402 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.579912 15518 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.599447 15488 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:43.642131  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:43.648818  6310 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:43.585022 15322 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.649518  6303 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 20.
I20260709 12:28:43.648581  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.615195 15433 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:43.659868  6274 leader_election.cc:400] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23.
I20260709 12:28:43.608695 15177 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:43.659044 15445 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:43.664862 15518 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:43.701565  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.704084  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:43.646440 15512 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 12:28:43.674152 15513 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.298s	user 0.001s	sys 0.002s
I20260709 12:28:43.674440  6376 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:43.650056 15006 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:43.710119 15006 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.684165 15337 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.688383  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:43.714052  6303 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:43.594966 15450 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.717828 15450 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:43.700235  6310 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:43.721148  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.721674  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.722939  6300 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:43.721964 15337 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:43.733348  6305 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:43.736033  7331 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:43.590340 15118 consensus_queue.cc:260] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:43.748405  7288 connection.cc:256] RPC call timeout handler was delayed by -1.1138909500000409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
W20260709 12:28:43.749037  7288 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Timed out: RequestConsensusVote RPC to 127.5.4.65:37395 timed out after 1.519s (SENT)
I20260709 12:28:43.749432  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:43.630213 15484 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.725781 15433 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:43.734920 15488 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:43.739653 15473 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:28:43.785658  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 285855us
I20260709 12:28:43.756131 15518 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:43.787667 15401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.835s	user 0.030s	sys 0.030s
I20260709 12:28:43.760545 15118 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:43.762034 15450 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.762115 15507 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:28:43.787571 15328 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:28:43.795616 15401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.030s	sys 0.031s
I20260709 12:28:43.818051 15396 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:43.822749 15396 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:43.827062 15430 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:43.827524 15430 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:43.833917  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.820204 15013 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.842839  6305 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:43.762892 15445 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 20, Time since last communication: 0.046s
I20260709 12:28:43.860332  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:43.795406 15518 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.868749  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:43.863591 15521 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:28:43.863390 15445 consensus_queue.cc:1243] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 33.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:28:43.866210 15392 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:28:43.900671 15328 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.901355 15328 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.875077 15337 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:43.903156 15337 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:43.904330 15337 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.912177 15507 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:43.912330 15490 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:43.913022 15490 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:43.807922 15450 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:43.915263 15377 consensus_meta.cc:287] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.645s	user 0.012s	sys 0.000s
I20260709 12:28:43.915743 15377 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.860144  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:43.916802  6384 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:43.917207  6384 raft_consensus.cc:614] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:43.919160 15401 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:43.919270  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:43.919641  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53970 (request call id 1419) took 1449 ms. Trace:
I20260709 12:28:43.920296 15401 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:43.920770 15401 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:43.921682 15401 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.886075 15350 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:43.881780  6425 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:43.870812 15433 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.922686 15494 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:43.877190 15389 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:43.922369 15434 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:43.884989 15210 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.929271  6305 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:43.957742  6300 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [20.11-22.12]
I20260709 12:28:43.920056  6308 rpcz_store.cc:276] 0709 12:28:42.470441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:42.470811 (+   370us) service_pool.cc:224] Handling call
0709 12:28:43.919425 (+1448614us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54000b3954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:43.919535 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350203264}
W20260709 12:28:43.933779  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.923596 15350 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:43.947995  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 66311.7us
W20260709 12:28:43.956626  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 223475us
W20260709 12:28:43.948330  7288 connection.cc:256] RPC call timeout handler was delayed by -1.0193909500003429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 12:28:43.964604  7288 connection.cc:256] RPC call timeout handler was delayed by -1.0130909500003327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17400000000000002s for a response
I20260709 12:28:43.949548  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:43.950409  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 160399us
I20260709 12:28:43.877640 15515 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.963563  6274 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:43.970106  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:43.971554  6384 raft_consensus.cc:606] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:43.971913  6384 raft_consensus.cc:614] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:43.970278  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.972841  6303 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 22: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:43.958222 15434 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:43.962450 15328 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:43.961730 15392 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:43.964695 15350 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:43.980681  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:43.983464  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:43.983636  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:43.988394  7332 raft_consensus.cc:3060] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:43.988976  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:43.989661  6300 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:43.990878  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:43.912935 15507 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:43.972184 15401 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:43.992415  6301 raft_consensus.cc:2410] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 25 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:28:43.914275 15490 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:43.999930  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:43.973764 15532 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:44.001777  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:43.978713 15434 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:44.011021  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.014340  7319 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:43.966550 15511 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:28:44.016498  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:44.017526  7320 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:44.018535  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:43.972069 15006 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:28:44.025911 15299 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.027001 15299 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.028009 15531 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.028558 15531 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.033497  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4e0e521cb08c420f981a29576eee90a6"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:44.005247 15473 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:44.003115 15392 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:44.039894  7320 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:28:44.040129  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.013279 15322 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.042153  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:44.041841  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:44.049427  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.037915 15532 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.042423  6384 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
W20260709 12:28:44.052564  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.053803  6385 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:44.054920  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:44.057732  7319 raft_consensus.cc:2376] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 34. Re-sending same reply.
I20260709 12:28:43.999801 15396 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.033146 15488 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:44.060513  7332 raft_consensus.cc:2468] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:44.038193 15490 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:44.043757 15450 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:44.057440 15531 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.062708 15428 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:44.065542 15488 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:44.067510 15473 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:44.109378  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.106280  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.065783 15532 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.117087  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:44.090799 15488 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:28:44.100968 15432 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.147s	user 0.006s	sys 0.024s
I20260709 12:28:44.123363  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.130146  6384 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:44.138077  6384 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 31.
W20260709 12:28:44.138583 15432 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.006s	sys 0.024s
I20260709 12:28:44.143805 15396 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.146915 15392 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:44.125203 15537 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:44.152154  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3477) took 1382 ms. Trace:
I20260709 12:28:44.152336 15473 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.152671  6377 rpcz_store.cc:276] 0709 12:28:42.770198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:42.777209 (+  7011us) service_pool.cc:224] Handling call
0709 12:28:42.791895 (+ 14686us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:43.043024 (+251129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:43.043105 (+    81us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:43.236754 (+193649us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b48001f0158. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:43.921338 (+684584us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:43.921924 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:28:44.012651 (+ 90727us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:44.012707 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:44.012810 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:44.151850 (+139040us) raft_consensus.cc:1685] finished
0709 12:28:44.151882 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:44.152080 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17700,"raft.queue_time_us":180013,"raft.run_cpu_time_us":2454,"raft.run_wall_time_us":261092,"spinlock_wait_cycles":21803264,"thread_start_us":314327,"threads_started":2,"wal-append.queue_time_us":138950}
I20260709 12:28:44.120952 15301 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:28:44.154210 15486 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.000s
I20260709 12:28:44.155296  6309 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:44.158106 15434 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.090714 15428 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.162942 15428 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.103098 15494 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.129415  6300 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:44.171030 15301 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:44.176836 15301 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.177075 15507 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:44.176926  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59955.8us
I20260709 12:28:44.178747 15328 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:44.169212 15389 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:44.171350 15210 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.178767 15515 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.180172 15342 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:44.192133  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.201287 15428 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:44.213148  6301 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:44.215059  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.220139 15299 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.229637 15507 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:44.230603 15543 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:28:44.233446 15468 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:44.235807  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:44.236792 15350 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.175247 15494 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:44.238909  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46860.9us
I20260709 12:28:44.240631  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.244285  7319 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:28:44.265522  7332 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:44.265755  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:44.269207  7288 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.278870  7331 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:44.277994  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:44.283896 15494 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.285758 15350 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.287117  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.277351 15434 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.245962 15507 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.260279 15118 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:44.294812 15118 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.295177 15118 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:44.296191 15507 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.245015 15468 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:44.299546  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.300257 15468 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.300761 15487 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:44.301275 15487 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:44.301520 15468 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:44.302286 15487 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.303493 15487 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:44.280206 15299 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:44.280393 15536 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:44.309340 15536 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:44.280045 15395 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.311154 15395 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.311584 15403 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.312237 15403 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.313295 15469 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.313628 15395 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:44.316637 15546 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:28:44.319653 15403 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.319900 15494 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.320266 15508 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:44.320835 15494 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.321599 15508 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:44.323113 15006 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.323760 15006 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.324327  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3706) took 1069 ms. Trace:
I20260709 12:28:44.324774  6309 rpcz_store.cc:276] 0709 12:28:43.254699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.254842 (+   143us) service_pool.cc:224] Handling call
0709 12:28:43.274001 (+ 19159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:44.155666 (+881665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:44.324086 (+168420us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54002d1ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:44.324267 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2026035456}
W20260709 12:28:44.311682  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.310186 15536 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.331851  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:44.333122  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.333748  6308 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 21.
I20260709 12:28:44.333902  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:44.331817  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.334770  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.335057  6300 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
W20260709 12:28:44.342222  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64375.9us
I20260709 12:28:44.313956 15469 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.346684  6309 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: [21.12-24.13]
I20260709 12:28:44.347203  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:44.348987  7319 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:28:44.354379  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.356280  6453 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:44.349364  7319 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 10
I20260709 12:28:44.358074  7319 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 19.11
I20260709 12:28:44.358358  7319 raft_consensus.cc:2917] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 FOLLOWER]: Aborting config change with OpId 19.11: Aborted: Op aborted by new leader
I20260709 12:28:44.359266  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.358767  7319 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:44.359802  6453 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:44.361932  6456 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:44.368285  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.368614  6459 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:44.347712 15428 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:44.344846 15403 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:44.376010  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.352361 15454 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.383361  6455 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:28:44.361263 15006 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.357342 15450 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:44.391405  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.391885  6453 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:44.365067 15328 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.032s
I20260709 12:28:44.387091 15395 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:44.409664  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.410547  6453 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:44.436280 15484 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:44.385661 15403 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.395478 15454 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.443694 15396 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:44.444286 15469 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.403569 15450 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.449574  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:44.433749 15118 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:44.452699  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.473079  6308 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:44.456983  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.474699  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:44.475021  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:44.475540  6308 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:44.475941  6310 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:44.480248  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:44.442353 15299 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:44.489495  6454 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:44.492002  6455 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:28:44.497431  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.501246  6384 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:44.507400  6304 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:28:44.494818 15480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.225s	user 0.000s	sys 0.012s
W20260709 12:28:44.511013  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 147172us
W20260709 12:28:44.514040  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56743.6us
I20260709 12:28:44.468947 15328 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:44.513648  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.515144  6310 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:44.515718  6450 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:44.519706  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.520422  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.522734  6458 raft_consensus.cc:2410] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:28:44.467360 15396 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:44.524011  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.525732  6308 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:44.484972 15544 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:44.451710 15395 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:44.529287 15548 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:28:44.531038 15443 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:44.526961 15402 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.533476 15328 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:44.519285 15428 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:28:44.534966  6425 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:44.536473 15013 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:44.539237  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.561807  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:44.566813  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.567545  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:44.541771 15450 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.543632 15403 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.544006 15118 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.573645 15490 consensus_queue.cc:1048] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:44.575464  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.576882  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.578022  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.578603 15465 consensus_queue.cc:1059] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:44.583181  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.584426  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.585043  6301 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:44.587874 15396 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.589165 15396 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:44.590827 15328 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:44.591521  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3781) took 1384 ms (client timeout 1730 ms). Trace:
W20260709 12:28:44.591974  6376 rpcz_store.cc:269] 0709 12:28:43.207473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.225184 (+ 17711us) service_pool.cc:224] Handling call
0709 12:28:44.591322 (+1366138us) spinlock_profiling.cc:243] Waited 404 ms on lock 0x7b54001f40d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:44.591436 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":929878784}
I20260709 12:28:44.593127  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.566352 15402 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.595404 15402 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.572058 15484 raft_consensus.cc:2804] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:44.597455  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.600157  6451 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:44.600621  6457 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:28:44.545645 15454 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:44.547261  6459 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:44.549288  6456 raft_consensus.cc:1240] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:44.549453  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.550729  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.603026  6380 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:44.604624  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.552197  6300 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:44.555233  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.561842  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.612426  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.594238  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.613615  6376 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:44.614951  6376 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
W20260709 12:28:44.615908  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:44.508940 15480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.244s	user 0.000s	sys 0.012s
I20260709 12:28:44.571664  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.578768  6272 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.623986  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.625119  6450 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:44.601980  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.622061  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.622129  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:44.571584 15457 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:44.565773  6384 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:44.621593  6376 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:44.550189 15544 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:44.633051  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54384.6us
I20260709 12:28:44.635076  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.611217 15342 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:44.571771 15013 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.637776 15484 raft_consensus.cc:697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:44.569309 15473 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.061s
I20260709 12:28:44.641265  6378 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:44.643133  6380 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:44.644500  6376 raft_consensus.cc:1217] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Deduplicated request from leader. Original: 29.20->[31.21-34.23]   Dedup: 31.21->[32.22-34.23]
I20260709 12:28:44.645314  6376 pending_rounds.cc:85] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 21
I20260709 12:28:44.645654  6376 pending_rounds.cc:107] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 33.22
I20260709 12:28:44.646003  6376 raft_consensus.cc:2889] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:44.648409  6272 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.649502  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.652138  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.654654  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.654336  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.657460  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.606076 15488 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.638964 15177 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:44.666841  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.667527  7319 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:44.661136  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:44.662695  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 59827.1us
I20260709 12:28:44.668412  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:44.669337  7314 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:44.662686  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.664136  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.660064  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.670468  6384 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:44.670683  6385 raft_consensus.cc:2410] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:44.639904 15457 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:44.671741  6300 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:44.673452  7320 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:44.683583  6384 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:44.678819  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.672112  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:44.686255  7319 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:44.686609  6300 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:44.686647  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:44.688042  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.688910  6301 raft_consensus.cc:2410] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:28:44.689028  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.673812 15457 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.693729  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.667366 15557 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.694985  7314 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:44.695345 15557 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.697710 15557 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.697763  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.698077  7332 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:44.679884 15467 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:28:44.698963 15432 raft_consensus.cc:2764] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 32: won
I20260709 12:28:44.651716 15480 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:44.699584  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.702241 15484 consensus_queue.cc:237] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.702809 15544 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.705988 15013 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.709335  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.710618  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:44.711041  6301 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:44.711184  6303 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:44.711241 15395 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.713785  6302 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:44.715099  6455 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:44.716351  6455 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:44.716688  6455 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:44.711149  7319 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:44.717366  6455 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.718344 15508 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:44.718817 15467 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.718914 15508 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.719410 15467 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.720042 15013 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:44.726657  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:44.736513  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 54575.3us
I20260709 12:28:44.656981 15433 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:44.739120  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:44.740144  6307 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 21: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:44.740579  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.659443 15402 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:44.757104  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.759512  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.758240  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:44.681013 15533 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:44.764473  6300 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:44.765301 15323 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:44.665480 15535 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.095s
I20260709 12:28:44.770252  6303 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:28:44.770663  6303 pending_rounds.cc:85] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 16
I20260709 12:28:44.770871  6303 pending_rounds.cc:107] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 26.17
I20260709 12:28:44.771145  6303 raft_consensus.cc:2889] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:44.762750  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.761688 15337 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.760788 15487 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:44.753377 15434 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:28:44.794852 15434 consensus_queue.cc:1243] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:44.788980 15402 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.795845 15402 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.798048 15402 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:44.781608  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44225.7us
I20260709 12:28:44.767455 15507 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:44.763293 15006 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:28:44.800217  7319 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
W20260709 12:28:44.767515  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 162047us
I20260709 12:28:44.808585  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:44.810056  6302 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:28:44.773727 15508 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.761173 15433 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:28:44.812407  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49766.7us
I20260709 12:28:44.774266 15557 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:44.799338 15337 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.799155 15535 consensus_queue.cc:1243] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 27.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:44.813380  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:44.822850  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:44.864806  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.807883 15507 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.866142  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:44.811890 15450 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:44.777184 15401 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:28:44.907370 15401 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:44.907743 15401 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:44.908496 15401 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.909703 15401 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:44.814275 15533 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.099s
I20260709 12:28:44.910589  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:44.912798 15457 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:44.868093 15535 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.915207 15535 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:44.916149 15522 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.281s	user 0.005s	sys 0.000s
I20260709 12:28:44.917452 15535 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:44.917711  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3475) took 2162 ms. Trace:
I20260709 12:28:44.872195 15484 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:44.829403 15395 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:28:44.920598  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 109789us
I20260709 12:28:44.918079  6383 rpcz_store.cc:276] 0709 12:28:42.755662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:42.776084 (+ 20422us) service_pool.cc:224] Handling call
0709 12:28:42.776455 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:42.783812 (+  7357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:42.783864 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:43.521995 (+738131us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:28:43.522374 (+   379us) log.cc:844] Serialized 26 byte log entry
0709 12:28:43.601967 (+ 79593us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:43.602002 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:43.602066 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:44.915859 (+1313793us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b2400297a58. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:28:44.917453 (+  1594us) raft_consensus.cc:1685] finished
0709 12:28:44.917474 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:44.917630 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":705,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":11957,"spinlock_wait_cycles":35396736,"thread_start_us":79185,"threads_started":1,"wal-append.queue_time_us":112637}
I20260709 12:28:44.875150  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:44.818928 15566 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.069s
I20260709 12:28:44.884476 15488 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.887755 15291 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:44.888273  6305 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:28:44.929495  6305 pending_rounds.cc:85] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 13
I20260709 12:28:44.929780  6305 pending_rounds.cc:107] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:28:44.930037  6305 raft_consensus.cc:2889] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:44.930374  6305 pending_rounds.cc:107] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:28:44.930632  6305 raft_consensus.cc:2889] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:44.871519 15507 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:44.893858 15323 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.895294 15432 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:44.897704 15337 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.767666 15278 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:28:44.880219 15467 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:44.905565  7329 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:28:44.820590 15557 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:44.918628  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:44.918000 15556 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.308s	user 0.001s	sys 0.000s
I20260709 12:28:44.923779  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:44.873250 15558 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:44.951189  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:44.932299  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 57271.7us
I20260709 12:28:44.926028 15431 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:28:44.964956 15431 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:44.965334 15431 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:44.966012  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:44.966813  6379 raft_consensus.cc:2410] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 29 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:28:44.921249 15484 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:44.967432 15484 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:44.967974 15442 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.968549 15442 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.969192  6380 raft_consensus.cc:1217] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Deduplicated request from leader. Original: 23.18->[24.19-30.21]   Dedup: 24.19->[27.20-30.21]
I20260709 12:28:44.969756  6380 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 30. Current term is 31. Ops: [24.19-30.21]
I20260709 12:28:44.904153 15433 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.970753  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.958451 15396 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:44.959530  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:44.972982 15480 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.973505 15480 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:44.959324  6385 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:44.958241 15558 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:44.974923 15558 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:44.975677 15480 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:44.918855 15560 consensus_queue.cc:1175] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4e0e521cb08c420f981a29576eee90a6" responder_term: 25 status { last_received { term: 23 index: 13 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:28:44.948107 15432 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.948099 15434 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:28:44.944136 15323 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.985884 15432 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:44.963972  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.966140 15431 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:44.920636 15395 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:44.987624 15395 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:44.987741 15323 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:44.988687  6453 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:28:44.961968  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:44.934537 15515 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:44.993083  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:44.940838 15457 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:45.012274  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:45.013531  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 205029us
I20260709 12:28:45.013667  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:45.013806  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.013044  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.015524  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:45.024868  7329 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:45.015710  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:44.948354 15557 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:45.003072 15323 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:45.029006  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.034973  6303 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:45.006219 15392 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:45.032761  7335 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:45.035919  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.029474  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:45.037261  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.042101  6424 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.024029 15337 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:45.049191  6385 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:45.042299  6308 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:45.041342  6303 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:44.984091 15484 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.063449  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.003189 15515 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.053268  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.023440 15457 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:45.053732 15490 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:45.054207 15433 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:44.998015 15565 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.046s
I20260709 12:28:45.065891  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.044957 15467 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:45.046021 15566 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:45.070849  6456 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:28:45.048195 15337 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:45.034410 15291 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:28:45.027311 15549 consensus_queue.cc:1059] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:28:45.050184 15392 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:45.072046 15467 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:45.076901 15433 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:45.077805 15518 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:45.079629 15430 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:45.099948  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 57922.1us
I20260709 12:28:45.100942  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.076292 15337 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:45.078302  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.082424  6302 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:28:45.083695  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:45.087709 15481 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:45.085137 15565 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:45.090080 15532 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:45.090755 15457 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.081341  6456 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:28:45.110155  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:45.113178  6308 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:28:45.116079  6304 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:45.124420  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:45.107553 15549 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:28:45.131023  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 118087us
I20260709 12:28:45.134233  6301 raft_consensus.cc:1240] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:28:45.114802 15573 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:45.113694 15481 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:45.133589  6424 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:45.132026 15569 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:28:45.100274 15558 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.129614 15006 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:45.129213 15432 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:45.150408 15006 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:45.150916 15006 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:45.155246  6458 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 3992) took 1931 ms (client timeout 2083 ms). Trace:
I20260709 12:28:45.155496 15481 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.157080 15481 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:45.148813 15571 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:28:45.140000 15549 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:45.165323  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.167276  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.167984  6452 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 20.
I20260709 12:28:45.168862  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:45.170006  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.171475  6348 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:45.172422  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:45.155730  6458 rpcz_store.cc:269] 0709 12:28:43.223372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.223518 (+   146us) service_pool.cc:224] Handling call
0709 12:28:45.155005 (+1931487us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400344254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:45.155182 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2957182848}
I20260709 12:28:45.173730 15508 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:45.173911  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.175535  6451 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:28:45.175956  6451 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 10
I20260709 12:28:45.176188  6451 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 19.11
I20260709 12:28:45.172761  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.176793  6451 raft_consensus.cc:2917] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Aborting config change with OpId 19.11: Aborted: Op aborted by new leader
I20260709 12:28:45.177232  6451 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:45.170650  6450 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:45.137233 15573 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.175535  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.182864  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:45.174670 15508 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:28:45.185593  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:45.186241  7318 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:45.121577 15452 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:28:45.189268 15558 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.140892 15392 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.191068 15573 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:45.192026 15484 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:45.191082  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.198668  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:45.131737 15518 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:28:45.200969  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:45.146677 15467 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.209481 15560 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:45.210956 15586 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:45.179528 15490 raft_consensus.cc:3055] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:45.194304 15487 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:45.199061 15401 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:28:45.268206 15578 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:28:45.202538 15488 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:45.202533 15480 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:45.262048 15531 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:45.262717 15490 raft_consensus.cc:740] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:45.265528 15487 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.211433 15560 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.066s
I20260709 12:28:45.259488 15289 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:28:45.277562 15586 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:45.290000 15531 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.324199 15580 consensus_queue.cc:1059] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.155s
I20260709 12:28:45.335669 15342 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:45.336433  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 126164us
I20260709 12:28:45.308223 15487 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:45.322927 15433 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:45.288828 15507 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.330091 15289 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 31.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:28:45.341351 15531 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:45.359052  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:45.360078  6378 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:28:45.361244  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:45.361487  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.323625 15508 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:28:45.366904  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3784) took 2158 ms. Trace:
I20260709 12:28:45.374516  6379 rpcz_store.cc:276] 0709 12:28:43.208342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:43.310814 (+102472us) service_pool.cc:224] Handling call
0709 12:28:45.366651 (+2055837us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b5400037d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:45.366828 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3802392320}
I20260709 12:28:45.375902  7318 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:45.341720 15342 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:45.381894  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:45.383179  6383 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:45.386317  6379 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 34.
I20260709 12:28:45.388664  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.370100  6305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57670 (request call id 3717) took 1037 ms. Trace:
I20260709 12:28:45.390967  6305 rpcz_store.cc:276] 0709 12:28:44.332381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.332736 (+   355us) service_pool.cc:224] Handling call
0709 12:28:44.333038 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:44.931110 (+598072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:45.369771 (+438661us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54001b35d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:45.370019 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275993856}
I20260709 12:28:45.332653 15586 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.399952  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:45.400914  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:45.412051  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57418us
I20260709 12:28:45.413585  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:45.413911  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.380453 15566 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:45.398010 15469 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.311036 15006 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.352049 15490 consensus_queue.cc:260] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.420001 15565 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:45.417114 15566 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:45.419445 15558 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:45.468849  7318 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEARNER]: Advancing to term 24
I20260709 12:28:45.471143  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:45.473392  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58224.5us
I20260709 12:28:45.487319  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:45.487951  6308 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:45.487916 15322 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:45.495654 15480 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.496236 15480 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.497313 15557 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.498350 15480 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.498756  6380 raft_consensus.cc:2393] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:45.501053  6308 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:28:45.505542  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:45.506932 15592 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:28:45.453577 15469 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.513360 15469 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.503012 15507 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:45.475386 15566 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:45.503477 15588 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:28:45.514539  6378 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:45.514938 15598 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:45.476233 15533 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:45.500819  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.527367  6457 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:45.418830 15395 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.487139  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:45.528921  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:45.533293 15533 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.539227  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:45.540407  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53342.7us
I20260709 12:28:45.458832 15587 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.240s
I20260709 12:28:45.555531  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:45.579633  6308 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:45.486326 15452 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 12:28:45.523501 15490 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:45.597330  6385 raft_consensus.cc:1240] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:45.525662 15599 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:45.552930 15322 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:45.541414 15488 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:45.602579  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.603763  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.611217  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.603998  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.605679  6456 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:45.609735  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:45.536396 15337 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.608445  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.608922  6458 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:45.552209 15586 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:45.552760 15533 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:45.631353  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73635.6us
I20260709 12:28:45.553680 15491 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:45.536193 15487 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.644488  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.651640  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:45.652405  6452 raft_consensus.cc:2376] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23. Re-sending same reply.
I20260709 12:28:45.618537 15322 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.611855 15488 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:45.655074  6452 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:45.655048  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.615602 15599 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:45.586334 15602 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 12:28:45.646950 15586 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:28:45.666927  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50926.3us
I20260709 12:28:45.671362  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:45.672986  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:45.673573  6305 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:45.661052 15322 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:45.660719 15488 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:45.665472 15601 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:45.648000 15491 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:45.698016  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50015.3us
I20260709 12:28:45.663130 15392 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:28:45.669258 15586 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:45.698863  6425 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.663179 15507 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:28:45.683071 15377 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:45.708519 15377 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:45.708956 15608 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:28:45.712476 15597 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:28:45.692773 15006 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.725837  6305 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:45.726665  6302 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
W20260709 12:28:45.717801  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53762.7us
I20260709 12:28:45.707141 15586 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:45.750025  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.756603 15604 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.762701 15491 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.763916  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:45.764806 15507 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:45.763952 15557 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:45.762537 15322 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.767010 15342 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.767652 15337 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:45.714427 15610 consensus_queue.cc:1059] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:28:45.768687 15610 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:45.768718  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:45.769217 15342 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:45.771557 15322 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:45.772090 15487 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.773029  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:45.773599  6452 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:45.777015 15531 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:45.777794 15573 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.586s	user 0.012s	sys 0.000s
I20260709 12:28:45.780716  6452 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:45.782454  6453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3868) took 1168 ms. Trace:
I20260709 12:28:45.784747  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1438) took 1056 ms. Trace:
I20260709 12:28:45.796260  6300 rpcz_store.cc:276] 0709 12:28:44.728241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.735650 (+  7409us) service_pool.cc:224] Handling call
0709 12:28:44.736019 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:44.766548 (+ 30529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:45.784476 (+1017928us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b54000a9e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:45.784686 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64810240}
I20260709 12:28:45.793406  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:45.797191  6303 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:45.796609  6272 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:45.797411  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:45.803406  6300 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:45.795346  6453 rpcz_store.cc:276] 0709 12:28:44.614062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.911429 (+297367us) service_pool.cc:224] Handling call
0709 12:28:44.911713 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:44.989046 (+ 77333us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:45.782168 (+793122us) spinlock_profiling.cc:243] Waited 76.6 ms on lock 0x7b5400344c54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:45.782383 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176234880}
I20260709 12:28:45.769273 15610 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:45.816146  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.819141  6300 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:45.830168  6300 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:28:45.827761  6425 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:45.757279 15604 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.812855 15491 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:45.769939 15599 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.804769 15558 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.838548 15491 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:45.839468 15491 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.843943 15322 raft_consensus.cc:2804] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:45.844487 15322 raft_consensus.cc:697] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:45.845261 15322 consensus_queue.cc:237] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.846342  6303 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:28:45.851886 15533 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.852432 15533 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:45.854151 15566 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.857306  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:45.862061  7318 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:28:45.863142  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 113221us
I20260709 12:28:45.864228  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.862902  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:45.866952  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.813117 15289 consensus_queue.cc:1059] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.046s
I20260709 12:28:45.862188 15533 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:45.876636  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 205298us
I20260709 12:28:45.804172 15573 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:45.859336 15558 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.893718  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:45.901580  6381 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:28:45.905520 15430 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:45.961460 15494 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:45.918418  7331 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:45.963701  7331 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:45.964689  7331 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.965744  7331 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:45.968279 15499 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:45.968863 15499 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:45.978376 15442 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:46.004674  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1349) took 1041 ms. Trace:
I20260709 12:28:46.057166  6380 rpcz_store.cc:276] 0709 12:28:44.963025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.970463 (+  7438us) service_pool.cc:224] Handling call
0709 12:28:46.004346 (+1033883us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b5400195ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:46.004571 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1212129152}
I20260709 12:28:46.013022  6450 raft_consensus.cc:1240] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:46.017791 15452 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.064016 15452 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.023543  6454 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:28:46.006441 15579 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:46.068295 15579 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:46.069402 15579 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:46.071391 15609 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.234s	user 0.001s	sys 0.002s
W20260709 12:28:46.072072 15615 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.237s	user 0.002s	sys 0.000s
I20260709 12:28:46.076455  6302 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:46.024837  6458 raft_consensus.cc:1240] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:46.076988 15565 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:46.077472 15565 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:46.026088  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1350) took 1062 ms. Trace:
I20260709 12:28:46.078094  6385 rpcz_store.cc:276] 0709 12:28:44.963285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.068513 (+105228us) service_pool.cc:224] Handling call
0709 12:28:45.068798 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:45.606252 (+537454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.025737 (+419485us) spinlock_profiling.cc:243] Waited 528 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:46.026005 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1214946560}
I20260709 12:28:46.074858  6425 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:46.082558  6455 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:46.083724 15612 raft_consensus.cc:2804] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:46.084414 15612 raft_consensus.cc:697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:46.085418 15612 consensus_queue.cc:237] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.088508  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1347) took 1126 ms. Trace:
I20260709 12:28:46.088958  6383 rpcz_store.cc:276] 0709 12:28:44.962445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.039845 (+ 77400us) service_pool.cc:224] Handling call
0709 12:28:45.040084 (+   239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:45.385688 (+345604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.088232 (+702544us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400109fd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:46.088433 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788535552}
I20260709 12:28:46.091032  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3394) took 1400 ms. Trace:
I20260709 12:28:46.089676  6302 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:28:45.959928 15590 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.506s	user 0.001s	sys 0.000s
I20260709 12:28:46.091804 15565 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.091451  6301 rpcz_store.cc:276] 0709 12:28:44.690859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.711835 (+ 20976us) service_pool.cc:224] Handling call
0709 12:28:45.133891 (+422056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:45.135977 (+  2086us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.090835 (+954858us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b5400063bd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:46.090967 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969876352}
I20260709 12:28:46.094738  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1620) took 1057 ms. Trace:
I20260709 12:28:46.030406 15549 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:46.010910  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:46.054153  6303 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:46.095568 15549 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.098011 15549 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:46.095247  6459 rpcz_store.cc:276] 0709 12:28:45.037013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.101498 (+ 64485us) service_pool.cc:224] Handling call
0709 12:28:45.101908 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:45.106122 (+  4214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:45.106167 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:45.106879 (+   712us) log.cc:844] Serialized 52 byte log entry
0709 12:28:45.274825 (+167946us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:28:45.308819 (+ 33994us) log.cc:844] Serialized 14 byte log entry
0709 12:28:45.309120 (+   301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:45.309206 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:45.959300 (+650094us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:45.972298 (+ 12998us) spinlock_profiling.cc:243] Waited 6.78 ms on lock 0x7b1800346a20. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:46.094537 (+122239us) raft_consensus.cc:1685] finished
0709 12:28:46.094557 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.094682 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3788,"prepare.queue_time_us":35593,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":4932,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":13559,"spinlock_wait_cycles":31071232,"thread_start_us":138119,"threads_started":1,"wal-append.queue_time_us":256359}
I20260709 12:28:46.093247 15454 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:46.100775 15531 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:46.029629 15431 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:46.101109 15565 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:46.030215  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:45.914201  6378 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:46.103902  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:46.106123  6309 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:46.095075  6300 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:46.131328 15499 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:46.132390 15499 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:46.112111  6383 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:28:46.134945  6380 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:28:46.135800  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1352) took 1001 ms. Trace:
I20260709 12:28:46.136212  6383 raft_consensus.cc:1240] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:28:46.136188  6380 rpcz_store.cc:276] 0709 12:28:45.134189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.058394 (+924205us) service_pool.cc:224] Handling call
0709 12:28:46.058870 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.135465 (+ 76595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.135729 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:28:46.137913  6274 connection.cc:256] RPC call timeout handler was delayed by -1.1632724689998213s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2904s for a response
I20260709 12:28:46.116977 15465 consensus_queue.cc:1059] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:46.118988 15581 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:46.139067 15465 raft_consensus.cc:3055] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:46.119724 15468 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:28:46.139465 15465 raft_consensus.cc:740] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:46.141633  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:45.949568  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:46.055023 15395 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:45.955497 15596 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.477s	user 0.005s	sys 0.000s
I20260709 12:28:46.100775  6303 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:46.162704 15484 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:46.165732 15579 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:46.178577  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:46.179474  6453 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:46.125476  6301 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:46.106061  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3510) took 1123 ms. Trace:
I20260709 12:28:46.196074  6378 rpcz_store.cc:276] 0709 12:28:44.982146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.913243 (+931097us) service_pool.cc:224] Handling call
0709 12:28:45.913709 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.105709 (+192000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.105987 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:46.080713  6385 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:46.198014  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3512) took 1026 ms. Trace:
I20260709 12:28:46.198469  6385 rpcz_store.cc:276] 0709 12:28:45.171147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.080260 (+909113us) service_pool.cc:224] Handling call
0709 12:28:46.080524 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.197694 (+117170us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.197944 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:46.199911 15549 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.120814  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:46.203121 15565 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:46.203758 15565 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.206231 15565 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:46.208009 15484 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.224429 15549 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.235884 15465 consensus_queue.cc:260] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.237650 15465 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:46.241117 15486 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:46.242467 15457 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.004s
W20260709 12:28:46.246870  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:46.248515  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 106972us
I20260709 12:28:46.264547  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:46.140846 15515 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:46.525779  7288 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.011s	sys 0.009s
W20260709 12:28:46.588178  7288 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.012s	sys 0.010s
I20260709 12:28:46.588212  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3484) took 2088 ms. Trace:
W20260709 12:28:46.484937  6307 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.022s	sys 0.092s
I20260709 12:28:45.891680 15450 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.483s
I20260709 12:28:45.888351 15602 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:46.508456  6376 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.008s	sys 0.007s
W20260709 12:28:46.598223  6376 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.008s	sys 0.007s
I20260709 12:28:45.878116 15610 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.569897  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:46.621922 15432 raft_consensus.cc:1064] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:28:46.622141  6450 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
W20260709 12:28:46.622475 15432 raft_consensus.cc:1068] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Unable to promote non-voter 561a4e21d8b344a79af44568ca80a3c5: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:28:46.421262 15617 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.623391 15617 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:46.624411 15624 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:28:46.574600  6456 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.878s	user 0.000s	sys 0.010s
W20260709 12:28:46.626084  6456 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.000s	sys 0.011s
I20260709 12:28:46.626874  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3882) took 1004 ms. Trace:
I20260709 12:28:46.627307  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3807) took 1983 ms. Trace:
I20260709 12:28:46.627224  6456 rpcz_store.cc:276] 0709 12:28:45.623028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.623209 (+   181us) service_pool.cc:224] Handling call
0709 12:28:45.623671 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:45.624197 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:45.624238 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:45.624962 (+   724us) log.cc:844] Serialized 26 byte log entry
0709 12:28:45.674074 (+ 49112us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:45.694736 (+ 20662us) log.cc:844] Serialized 14 byte log entry
0709 12:28:46.626468 (+931732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.626572 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:46.626623 (+    51us) raft_consensus.cc:1685] finished
0709 12:28:46.626641 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.626802 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4537,"prepare.queue_time_us":21612,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":44,"raft.queue_time_us":15652,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":13096,"thread_start_us":979945,"threads_started":2,"wal-append.queue_time_us":75992}
W20260709 12:28:46.575567  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.628007  6376 rpcz_store.cc:276] 0709 12:28:44.643468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.643645 (+   177us) service_pool.cc:224] Handling call
0709 12:28:44.644189 (+   544us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:28:44.646965 (+  2776us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:44.647017 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:44.647806 (+   789us) log.cc:844] Serialized 52 byte log entry
0709 12:28:45.911342 (+1263536us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b18004bd540. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:46.598705 (+687363us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:46.598758 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.598848 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:46.626995 (+ 28147us) raft_consensus.cc:1685] finished
0709 12:28:46.627014 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.627248 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3699,"raft.queue_time_us":143,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6085,"spinlock_wait_cycles":7089024,"thread_start_us":687130,"threads_started":1,"wal-append.queue_time_us":1976296}
I20260709 12:28:46.629767 15452 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:46.576028  6304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.007s	sys 0.004s
W20260709 12:28:46.630692  6304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.007s	sys 0.004s
I20260709 12:28:46.576992  6274 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:46.524202 15445 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.000s	sys 0.021s
W20260709 12:28:46.589589  6307 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.022s	sys 0.097s
I20260709 12:28:46.588703  6377 rpcz_store.cc:276] 0709 12:28:44.499996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.500158 (+   162us) service_pool.cc:224] Handling call
0709 12:28:44.500460 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:44.568903 (+ 68443us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:44.568948 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:44.568995 (+    47us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:44.569039 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.581152 (+2012113us) spinlock_profiling.cc:243] Waited 68.2 ms on lock 0x7b5400065254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:46.587971 (+  6819us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.588140 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156940672}
I20260709 12:28:46.591857  6453 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:28:46.588848  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3508) took 1607 ms. Trace:
W20260709 12:28:46.603116 15618 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.704s	user 0.000s	sys 0.011s
I20260709 12:28:46.555132 15531 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:46.524551 15557 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.000s	sys 0.005s
I20260709 12:28:46.612571 15558 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:46.555639 15118 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:46.616755 15586 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.827s	user 0.013s	sys 0.005s
W20260709 12:28:46.616905 15431 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.000s
W20260709 12:28:46.617331 15488 consensus_meta.cc:287] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.921s	user 0.009s	sys 0.011s
I20260709 12:28:46.597661 15613 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:46.650738 15612 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.027s	sys 0.039s
I20260709 12:28:46.619460 15486 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:46.620216  7331 consensus_meta.cc:287] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.000s
I20260709 12:28:46.620314  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3506) took 1639 ms. Trace:
W20260709 12:28:46.620770  6455 consensus_meta.cc:287] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.004s
I20260709 12:28:46.621246 15572 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:46.621217 15430 consensus_meta.cc:287] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.659s	user 0.004s	sys 0.006s
W20260709 12:28:46.630515  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:46.631906  6458 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:46.630514 15629 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:46.618280  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:46.708097  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:46.560566 15620 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:46.644819  6275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.004s	sys 0.011s
I20260709 12:28:46.560689 15490 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:28:46.655226  6300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.008s	sys 0.024s
W20260709 12:28:46.716720  6300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.008s	sys 0.028s
I20260709 12:28:46.685984 15601 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:46.656457  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:46.639446 15487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.009s	sys 0.005s
W20260709 12:28:46.682126  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 114620us
W20260709 12:28:46.637701 15445 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.000s	sys 0.022s
I20260709 12:28:46.640851  6379 rpcz_store.cc:276] 0709 12:28:44.981677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.753757 (+772080us) service_pool.cc:224] Handling call
0709 12:28:45.754192 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:45.754461 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:45.791650 (+ 37189us) log.cc:844] Serialized 14 byte log entry
0709 12:28:46.588090 (+796440us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:46.588368 (+   278us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:46.588442 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.588570 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.588764 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":689393,"prepare.queue_time_us":38453,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":722247,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"thread_start_us":106526,"threads_started":1,"wal-append.queue_time_us":574506}
I20260709 12:28:46.618249 15566 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:46.691606  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.011s	sys 0.011s
I20260709 12:28:46.692675  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1356) took 1175 ms. Trace:
I20260709 12:28:46.695801  7331 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:46.696079  6455 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 12:28:46.650537 15557 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.759s	user 0.001s	sys 0.005s
I20260709 12:28:46.695693  6381 rpcz_store.cc:276] 0709 12:28:44.981183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.492175 (+510992us) service_pool.cc:224] Handling call
0709 12:28:45.492506 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:45.902114 (+409608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.619921 (+717807us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540023d7d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:46.620223 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":940114176}
I20260709 12:28:46.672348 15433 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.681742 15118 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.685223 15586 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.691609 15488 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.704703  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:46.708940  6382 raft_consensus.cc:606] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Received request to transfer leadership to TS 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:46.691461 15612 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.027s	sys 0.039s
W20260709 12:28:46.712544  6275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.004s	sys 0.011s
I20260709 12:28:46.673659  6302 raft_consensus.cc:1240] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
W20260709 12:28:46.695191 15328 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.014s	sys 0.008s
I20260709 12:28:46.691047 15642 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.672037 15531 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:46.740022  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:46.777818  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 922166us
W20260709 12:28:46.754802  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.011s	sys 0.016s
I20260709 12:28:46.776793  6380 rpcz_store.cc:276] 0709 12:28:45.517440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.137627 (+620187us) service_pool.cc:224] Handling call
0709 12:28:46.236145 (+ 98518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.236590 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:46.238206 (+  1616us) log.cc:844] Serialized 14 byte log entry
0709 12:28:46.692293 (+454087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:46.692354 (+    61us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:46.692407 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.692482 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:46.692608 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":447092,"prepare.queue_time_us":469,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":425861,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":5008}
I20260709 12:28:46.776783  6382 raft_consensus.cc:614] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:46.709272 15536 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.003s	sys 0.023s
I20260709 12:28:46.700318 15278 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:46.804790  6300 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:28:46.715956 15604 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:46.806033  6384 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:46.801492  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1303) took 2104 ms. Trace:
I20260709 12:28:46.703635 15573 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:46.747637 15579 consensus_meta.cc:287] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 12:28:46.828995  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:46.786056  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:46.830277  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 44330.9us
I20260709 12:28:46.777772  6455 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3866) took 2168 ms. Trace:
I20260709 12:28:46.830966  6455 rpcz_store.cc:276] 0709 12:28:44.609633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.715981 (+106348us) service_pool.cc:224] Handling call
0709 12:28:44.716242 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:46.082069 (+1365827us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b580011d8d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa329dfe7 00007f6fa329f7de 00007f6fa32a9b49 00007f6fa32aa416 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4
0709 12:28:46.777283 (+695214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.777708 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2880825088}
I20260709 12:28:46.832880  7319 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
W20260709 12:28:46.803733 15210 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
W20260709 12:28:46.776568 15487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.011s	sys 0.009s
I20260709 12:28:46.826517  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:46.781989 15328 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.016s	sys 0.012s
I20260709 12:28:46.780948 15620 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:46.807794 15536 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.003s	sys 0.024s
I20260709 12:28:46.734046 15572 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:46.834961  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:46.835255  6302 raft_consensus.cc:2393] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 30: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:46.828320  7331 rpcz_store.cc:276] 0709 12:28:44.672143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.672291 (+   148us) service_pool.cc:224] Handling call
0709 12:28:44.672652 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:45.918277 (+1245625us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54003f3ca0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa321edc0 00007f6fa32a91c0 00007f6fa32aa231 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c
0709 12:28:46.776247 (+857970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:46.776658 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441390848}
I20260709 12:28:46.729554 15628 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:46.783607 15427 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:28:46.763159 15601 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.729765 15616 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.202s
I20260709 12:28:46.730070 15289 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:46.783213 15433 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.842087  6385 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:46.714347 15629 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.842399  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:46.841248  6377 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:46.962257  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:46.963696  7318 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:46.964824  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:46.967159  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:46.842519  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:46.875460  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:46.803725 15430 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:46.807946 15469 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.750972 15641 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:47.007905  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:47.010385  7329 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:47.013103 15581 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:28:47.014550  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 1.12083e+06us
I20260709 12:28:47.015864 15432 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:46.968016  6379 raft_consensus.cc:2393] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 21: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:46.841303  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:47.017352 15396 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:47.017753 15396 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:47.019191 15612 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.018973 15396 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.019893 15612 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.021401 15465 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:47.023243 15612 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:46.838551  6381 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 34 index: 23. (index mismatch)
I20260709 12:28:46.734292 15291 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:46.979140 15614 log.cc:927] Time spent T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:28:46.961732  6378 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:46.841090  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:46.836239 15508 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:46.812320 15507 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:46.839118 15644 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:47.002029 15566 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:47.002511 15579 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:47.004601  7318 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:28:47.005671  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:46.831178 15278 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:46.957928  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:46.882926 15557 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.117s
I20260709 12:28:47.038883  6376 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:46.836203 15631 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.961544 15454 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.780427 15602 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:47.044911  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:46.805673 15642 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.830471 15401 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.066250  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:46.990970 15289 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.075570  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:47.087682  6453 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:46.871167 15627 consensus_queue.cc:1059] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.191s
I20260709 12:28:46.999092 15430 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.073364  6385 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:47.059464 15644 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.062100 15431 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:28:46.979310 15628 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:46.981539 15620 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.095659  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:47.095873  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.101421 15644 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:47.040253 15291 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.099783  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:47.026336 15469 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.066160 15328 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.108198  6377 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:47.071483 15587 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:28:47.112715  6457 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:28:47.096796  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.106093 15433 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.097568 15557 consensus_queue.cc:1243] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 21.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:28:47.092036 15401 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.133814 15617 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:47.136835 15467 raft_consensus.cc:3055] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:28:47.138507  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.139289 15377 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:47.145790 15396 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:47.151382 15581 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.142191 15210 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:28:47.076452 15507 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.059631 15508 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:47.166059  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.087045 15454 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.100723 15616 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:28:47.106329 15465 consensus_queue.cc:1059] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:28:47.156790 15531 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:47.120174  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 869847us
I20260709 12:28:47.218199  6274 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:47.177565 15599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.009s	sys 0.021s
I20260709 12:28:47.178809 15490 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:47.153712 15586 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:47.198496  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.089417 15631 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.104660 15602 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.088106 15572 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:47.202755  6348 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.175107 15467 raft_consensus.cc:740] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:47.183810 15634 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:47.278748  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60668.4us
I20260709 12:28:47.288774  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:47.171376 15328 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.286509 15433 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.297914  7319 raft_consensus.cc:1275] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:47.276583 15490 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:47.203797 15581 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.276075 15395 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:47.289399 15627 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:47.309115 15602 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.308761 15467 consensus_queue.cc:260] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.309396 15565 consensus_queue.cc:1059] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:47.220917 15612 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:28:47.282048 15536 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:47.318869 15484 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:47.296280 15629 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.329351 15536 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:47.331421 15628 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.331977 15536 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.311465 15645 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:47.299631 15531 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:47.299598 15601 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.348498  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:58396 (request call id 60) took 3321 ms. Trace:
I20260709 12:28:47.348951  6382 rpcz_store.cc:276] 0709 12:28:44.026797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.030579 (+  3782us) service_pool.cc:224] Handling call
0709 12:28:47.348253 (+3317674us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b54003d36d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a1853 00007f6fa826f60a 00007f6fa17f57d4 00007f6fa17f5762 00007f6fa17f5691 00007f6fa17f560c 00007f6fa17f48c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:47.348391 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6150910208}
I20260709 12:28:47.351042 15487 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.351596 15565 raft_consensus.cc:3055] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:47.327734 15467 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:47.319520 15535 raft_consensus.cc:3055] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:47.340648 15620 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.318933 15278 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.341800 15291 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.326421 15627 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:47.344267 15612 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:47.307343 15433 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.345009 15610 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:47.332839 15633 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.255s	user 0.003s	sys 0.001s
I20260709 12:28:47.352103 15565 raft_consensus.cc:740] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:47.360582 15430 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.339361 15395 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.366783  7332 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 21. (term mismatch)
I20260709 12:28:47.369319  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:47.382020  7332 pending_rounds.cc:85] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 20
I20260709 12:28:47.382462  7332 pending_rounds.cc:107] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 30.21
W20260709 12:28:47.381841  6423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.014s	sys 0.024s
I20260709 12:28:47.338773 15289 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:47.356863 15535 raft_consensus.cc:740] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:47.379375  7320 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:47.303790 15634 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.383948  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b776445e68c14533bcde1f784caa703a"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:47.397686  6382 raft_consensus.cc:606] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:28:47.398212  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 103588us
I20260709 12:28:47.365589 15612 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:47.351625 15487 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:47.403512  6304 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.001s	sys 0.009s
W20260709 12:28:47.403906  6304 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.001s	sys 0.009s
I20260709 12:28:47.382819  7332 raft_consensus.cc:2889] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:47.395581  6423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.015s	sys 0.024s
I20260709 12:28:47.221433 15480 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:47.400856  7320 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35. Ops: []
I20260709 12:28:47.405253  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:47.407207  7317 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.038s
W20260709 12:28:47.407562  7317 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.004s	sys 0.038s
I20260709 12:28:47.408794  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:47.400631 15631 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:47.409677  6310 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.015s	sys 0.009s
W20260709 12:28:47.410086  6310 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.016s	sys 0.009s
I20260709 12:28:47.412627 15491 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:47.413537 15491 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:47.413846 15491 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:47.414677 15491 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.415758 15536 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:47.415695 15655 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:47.416393 15536 consensus_queue.cc:1243] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 26.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:47.417022 15491 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:47.420437 15536 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:47.384473 15433 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.426838  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:47.398252  6382 raft_consensus.cc:614] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:47.430778  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:47.436609  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:47.443358  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:47.226312 15599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.010s	sys 0.021s
I20260709 12:28:47.422590 15508 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.427510 15118 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.411779 15480 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:47.459012  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.464222  6274 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:47.433390 15652 consensus_queue.cc:1059] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.029s
I20260709 12:28:47.429942 15602 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:47.433493 15507 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.476715  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.486510  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.460698 15617 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.482008  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.487766  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:47.499110 15652 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.037s
I20260709 12:28:47.445292 15210 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:47.456082 15499 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:47.494771  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:47.494983  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:47.496546  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.556818  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.544469 15536 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:47.558748 15536 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.561079 15536 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.553867 15617 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.557437  6382 raft_consensus.cc:606] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:47.501550  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:47.564474  6382 raft_consensus.cc:614] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:47.564929  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63618us
W20260709 12:28:47.472726  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 357327us
I20260709 12:28:47.426362 15565 consensus_queue.cc:260] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.508286  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.515278  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.569721  6452 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:47.521760  6458 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:47.526021 15651 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:28:47.510462 15599 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.526937  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.490727 15602 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.575394  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:47.470770 15664 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:28:47.541673 15395 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.427646 15627 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:47.542289 15581 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:47.455622 15535 consensus_queue.cc:260] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.543133 15480 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.488549 15118 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:28:47.492105 15612 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:47.461050 15484 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:47.565903  6302 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 27: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:47.503713  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 63811.5us
I20260709 12:28:47.563687 15634 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.540278 15490 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.578648  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.577888 15663 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.002s
I20260709 12:28:47.551213 15644 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.583009  6383 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:47.580139  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.700721  6459 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 4016) took 1525 ms. Trace:
I20260709 12:28:47.702121  7285 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:47.564128 15210 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.574362 15491 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:47.718351 15631 consensus_queue.cc:1059] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
I20260709 12:28:47.687875 15617 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:47.717182 15549 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.700930 15484 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.682082 15565 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:47.787176 15631 raft_consensus.cc:3055] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:47.787626 15631 raft_consensus.cc:740] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:47.788915 15491 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:47.789460  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:47.788587 15631 consensus_queue.cc:260] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.577986 15669 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.203s
I20260709 12:28:47.792157  6300 raft_consensus.cc:2368] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
W20260709 12:28:47.777644  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75631.8us
I20260709 12:28:47.717451  6459 rpcz_store.cc:276] 0709 12:28:46.175646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.176978 (+  1332us) service_pool.cc:224] Handling call
0709 12:28:46.177426 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:46.210118 (+ 32692us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:28:46.210162 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:46.210818 (+   656us) log.cc:844] Serialized 26 byte log entry
0709 12:28:46.685813 (+474995us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:46.685849 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:47.642604 (+956755us) spinlock_profiling.cc:243] Waited 32.3 ms on lock 0x7b54000eae54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:47.642693 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:47.683687 (+ 40994us) raft_consensus.cc:1685] finished
0709 12:28:47.683716 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:47.700649 (+ 16933us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74404224,"thread_start_us":474647,"threads_started":1,"wal-append.queue_time_us":748309}
I20260709 12:28:47.641938 15301 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:28:47.791528  7318 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.023s	sys 0.036s
W20260709 12:28:47.800937  7318 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.023s	sys 0.036s
I20260709 12:28:47.640497 15536 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:47.789429 15491 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:47.581753 15645 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:47.580715 15499 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:47.686380 15533 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.702785 15480 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:47.756666 15672 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:28:47.682634 15487 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:47.670518 15604 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:47.746706 15395 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:47.702139 15580 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.099s
I20260709 12:28:47.822165  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:47.825507  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.827152  6458 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:47.829074  7320 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:47.746940 15535 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:47.582193 15602 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:47.739140 15490 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.810338 15499 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.736045 15634 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:47.855755  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:47.856396  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.814217 15631 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:47.857275  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:47.862169  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 347011us
I20260709 12:28:47.803539 15301 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:47.858666  6450 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:28:47.863087  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:47.864359  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:47.578066 15599 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.871214  7317 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:47.872467  7317 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:47.864398  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:47.874066  6455 raft_consensus.cc:2393] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 31: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:47.805018 15536 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:47.864825 15467 consensus_meta.cc:287] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.505s	user 0.013s	sys 0.001s
I20260709 12:28:47.892499  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.894866  6309 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:47.810928 15533 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.901453  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.902092  6309 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:47.913443  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.816237 15480 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.819936 15604 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.906450  7317 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.913172  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.913116 15301 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.914208  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:47.829977 15395 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:47.892078 15490 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:47.949970  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:47.918085 15401 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.955230 15480 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.904448 15484 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.963217  6309 raft_consensus.cc:2393] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:47.964736 15599 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.966390  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3430) took 1348 ms. Trace:
I20260709 12:28:47.966818  6308 rpcz_store.cc:276] 0709 12:28:46.618101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.622808 (+  4707us) service_pool.cc:224] Handling call
0709 12:28:46.623127 (+   319us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:47.111713 (+488586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:47.111760 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:47.158588 (+ 46828us) log.cc:844] Serialized 52 byte log entry
0709 12:28:47.651877 (+493289us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:47.651946 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:47.652057 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:47.966145 (+314088us) raft_consensus.cc:1685] finished
0709 12:28:47.966169 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:47.966332 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33070,"raft.queue_time_us":127610,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"thread_start_us":921926,"threads_started":2,"wal-append.queue_time_us":687886}
I20260709 12:28:47.986462  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:47.987342  6303 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
W20260709 12:28:47.988790  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.984789 15621 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:47.991717  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:47.997499  6301 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:47.955236 15644 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.983628 15484 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.993721 15301 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:47.999294 15581 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.991400 15621 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:48.061144  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 483959us
I20260709 12:28:48.019693 15484 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.065213  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:48.066604  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:48.051977 15445 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.023s	sys 0.027s
I20260709 12:28:48.028633 15581 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.106693  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:48.127843  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 250953us
W20260709 12:28:48.129007  6274 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.067394 15676 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.134135  6272 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:48.106329 15621 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:48.106760 15445 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.024s	sys 0.028s
I20260709 12:28:48.071457 15644 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:48.140466  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:48.119083 15210 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.143826 15652 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:48.125300 15499 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.123174 15693 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:48.130577 15627 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:48.155660 15536 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.168346 15536 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:48.169165 15536 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.171453 15536 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.173276  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:48.174052 15323 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:28:48.175092 15686 log.cc:927] Time spent T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:28:48.176076 15323 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:48.176501 15323 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:48.176819 15692 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:28:48.177277  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:48.178324 15499 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:48.178791 15581 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:48.178890 15499 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.137944 15480 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:28:48.179577 15499 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:48.180722 15480 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:48.181243 15480 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.183498 15536 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:48.183987 15536 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.186335 15536 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.145534 15445 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:48.188409 15445 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.188764 15445 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:48.191385  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:48.192597 15674 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:48.139328 15644 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:48.193147 15674 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.193389 15644 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:48.193753 15674 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:48.152729 15627 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.217850 15487 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:48.220630 15487 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.223090 15487 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.176851 15323 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:48.139444 15676 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.205623 15604 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.226966 15491 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.227895  7335 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:48.217224 15627 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:48.232870  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.219484 15395 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.242105  6308 raft_consensus.cc:1240] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:48.243142  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.238725 15604 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.240757 15480 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.259670 15674 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.283085 15674 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:48.283710 15676 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:48.292725  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.293855  6300 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
W20260709 12:28:48.297223  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 101028us
I20260709 12:28:48.299933  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.300534  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.302790  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.303517  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:48.304255  6302 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:48.292771 15688 log.cc:927] Time spent T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:28:48.303812  6300 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:48.306752  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.309445  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51842 (request call id 3821) took 1466 ms. Trace:
I20260709 12:28:48.309900  6377 rpcz_store.cc:276] 0709 12:28:46.842819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:46.966930 (+124111us) service_pool.cc:224] Handling call
0709 12:28:48.309261 (+1342331us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54001090d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:48.309384 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256769408}
I20260709 12:28:48.278854 15604 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.303118 15484 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:48.338527 15697 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:28:48.338124 15693 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:48.379900  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.335948 15499 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.399811 15693 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.401733 15143 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:48.402207 15143 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:48.403156 15143 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:48.410089  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.381640 15484 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.404533 15143 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:48.389354 15674 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:48.422744 15491 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.435081 15674 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.439105 15581 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.443650 15707 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:48.435460 15484 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.455528 15646 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:48.466089  6302 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:48.469903 15674 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.471254 15604 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.472360 15644 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.474813 15484 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:48.479596  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:48.485298 15649 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.176s	user 0.000s	sys 0.004s
I20260709 12:28:48.486897  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3415) took 2700 ms. Trace:
I20260709 12:28:48.483188 15644 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.455420 15707 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:48.488770  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:48.489488  6304 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:48.490286 15707 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.487344  6307 rpcz_store.cc:276] 0709 12:28:45.785988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:45.855695 (+ 69707us) service_pool.cc:224] Handling call
0709 12:28:45.873372 (+ 17677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:46.638124 (+764752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:46.638170 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:46.638745 (+   575us) log.cc:844] Serialized 26 byte log entry
0709 12:28:47.061422 (+422677us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:47.061481 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:47.061586 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:48.484672 (+1423086us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:48.486396 (+  1724us) raft_consensus.cc:1685] finished
0709 12:28:48.486436 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:48.486803 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":206163,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":120,"spinlock_wait_cycles":113756928,"thread_start_us":1133130,"threads_started":2,"wal-append.queue_time_us":621145}
W20260709 12:28:48.495009 15708 log.cc:927] Time spent T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:28:48.454567 15581 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.504695 15581 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.485188 15328 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.506666 15646 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:48.522758 15646 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.523399 15488 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:48.527374 15658 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.202s	user 0.000s	sys 0.005s
I20260709 12:28:48.532517  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:48.534365  6424 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.675s (SENT) [suppressed 9 similar messages]
W20260709 12:28:48.536417  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.537493 15431 raft_consensus.cc:1064] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
W20260709 12:28:48.538024 15431 raft_consensus.cc:1068] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Unable to promote non-voter 561a4e21d8b344a79af44568ca80a3c5: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:28:48.538447 15667 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.995s	user 0.001s	sys 0.001s
I20260709 12:28:48.538959 15465 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:48.539431 15465 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.539774 15465 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:48.540066  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3889) took 1461 ms. Trace:
I20260709 12:28:48.540776 15454 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:48.541023 15499 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:48.540509  6456 rpcz_store.cc:276] 0709 12:28:47.078574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.078739 (+   165us) service_pool.cc:224] Handling call
0709 12:28:47.079121 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:47.079536 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:47.079578 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:47.083120 (+  3542us) log.cc:844] Serialized 26 byte log entry
0709 12:28:47.403959 (+320839us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:28:47.404008 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:47.404147 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:48.537824 (+1133677us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:48.539761 (+  1937us) raft_consensus.cc:1685] finished
0709 12:28:48.539782 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:48.540000 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":126,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":5304,"spinlock_wait_cycles":3844864,"thread_start_us":320419,"threads_started":1,"wal-append.queue_time_us":426327}
I20260709 12:28:48.542405  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:48.543298  6384 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1348) took 3580 ms (client timeout 3256 ms). Trace:
W20260709 12:28:48.543700  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.544727 15457 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:48.545185 15515 raft_consensus.cc:3055] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:28:48.545279 15457 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.545819 15515 raft_consensus.cc:740] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:48.546521 15457 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:48.546909 15515 consensus_queue.cc:260] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:48.548116 15515 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:48.553042 15427 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:28:48.543746  6384 rpcz_store.cc:269] 0709 12:28:44.962772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:44.963733 (+   961us) service_pool.cc:224] Handling call
0709 12:28:48.543083 (+3579350us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b54003d36d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:48.543201 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4183616256}
I20260709 12:28:48.556936 15642 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:48.557184  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.558007  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.558400  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.558774 15445 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.559263  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.559429  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.559744  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.560268  6455 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:48.561101 15445 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:48.530282 15469 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:48.567144  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:48.567291 15710 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:28:48.566959 15457 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:48.567797  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.569416 15457 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:48.533046 15627 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.524096 15646 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:48.570297  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:48.562270  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.637965  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.638119  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:48.642134  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.641288 15322 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:48.666915  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:48.655339  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:48.686160  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:48.670667  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.674913  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:48.665750  6452 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:48.636963 15445 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:48.688504  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:48.689419  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 155726us
I20260709 12:28:48.709712  7332 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:48.689250  7314 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:48.687963  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.651895 15469 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.638890 15707 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:48.561631 15465 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:48.729889  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:48.734067  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:48.644098 15558 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:49.086865  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:48.709205 15491 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:48.887415  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.688928 15616 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.015709 15613 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.715904 15469 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.101655 15616 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.102708  6453 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:48.771819  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:48.838171  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:48.750705  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.104720  6455 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:49.104933  7318 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
W20260709 12:28:48.803664  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.794116  7332 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:48.881199  7319 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1311) took 1878 ms. Trace:
I20260709 12:28:49.092751  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:48.773182  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:48.972141  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:49.014868  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.106437  7319 rpcz_store.cc:276] 0709 12:28:47.002665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.003191 (+   526us) service_pool.cc:224] Handling call
0709 12:28:47.003548 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:47.298343 (+294795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:48.880900 (+1582557us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54006c16d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:48.881129 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675971328}
W20260709 12:28:49.103317  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:48.746287  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:48.671046 15323 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.110567  6302 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 27
W20260709 12:28:49.074970  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:48.789381  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.101883 15613 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.116073  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.117600  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:49.118768  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.115377 15328 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:28:48.973258 15650 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:28:48.880069 15694 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.775s	user 0.002s	sys 0.000s
W20260709 12:28:48.923802 15714 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 12:28:49.125890  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:48.721470 15322 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:49.130915  6453 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:48.876013 15641 raft_consensus.cc:3055] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:49.121711  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.120541  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.127377  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:49.076928 15579 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:49.076768 15427 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.071525 15515 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:49.119887 15450 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:49.133481  6456 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:49.135936  7320 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:48.717767 15445 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:48.774664 15681 log.cc:927] Time spent T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:28:49.145202  6308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3443) took 1249 ms. Trace:
I20260709 12:28:49.142787 15579 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.145634 15515 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.145974  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.163121  6455 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:49.163350  7318 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:28:49.148324 15616 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:49.148859  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59648 (request call id 3891) took 2054 ms. Trace:
I20260709 12:28:49.116127 15558 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.165920  6451 rpcz_store.cc:276] 0709 12:28:47.094793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.094933 (+   140us) service_pool.cc:224] Handling call
0709 12:28:47.105739 (+ 10806us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:47.498059 (+392320us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:47.498110 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:47.565397 (+ 67287us) log.cc:844] Serialized 52 byte log entry
0709 12:28:47.918912 (+353515us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:48.131701 (+212789us) log.cc:844] Serialized 14 byte log entry
0709 12:28:48.148797 (+ 17096us) log.cc:844] Serialized 14 byte log entry
0709 12:28:48.149017 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:48.149090 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:48.879451 (+730361us) spinlock_profiling.cc:243] Waited 18.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:49.148633 (+269182us) raft_consensus.cc:1685] finished
0709 12:28:49.148658 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:49.148793 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1543,"prepare.queue_time_us":511998,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":71,"raft.queue_time_us":58862,"raft.run_cpu_time_us":438,"raft.run_wall_time_us":18233,"spinlock_wait_cycles":41575552,"thread_start_us":845614,"threads_started":3,"wal-append.queue_time_us":469186}
I20260709 12:28:49.167485  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:49.168028 15558 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.168403  6310 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:49.169307  7314 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:49.169486  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.116312 15707 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.116170 15491 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.152861  6458 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 29. Current term is 30. Ops: [29.22-29.22]
I20260709 12:28:49.155308  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.156083  6308 rpcz_store.cc:276] 0709 12:28:47.895386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.969452 (+ 74066us) service_pool.cc:224] Handling call
0709 12:28:47.969699 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:48.242598 (+272899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.144963 (+902365us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b5400140d54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:49.145142 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626043264}
I20260709 12:28:49.157087  6302 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
W20260709 12:28:49.137012 15718 log.cc:927] Time spent T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:28:49.134183 15641 raft_consensus.cc:740] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:49.136765 15627 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.179208  7320 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:49.175549 15518 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.175881  7332 raft_consensus.cc:1240] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:49.176505  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.178292  6310 raft_consensus.cc:2410] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:28:49.178853  7319 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:28:49.180094  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.181339  7331 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35. Ops: []
I20260709 12:28:49.184536  6308 raft_consensus.cc:2393] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:49.153394 15427 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.196748 15328 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:49.181406 15707 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:49.209893 15726 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:28:49.155359 15450 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.151237 15322 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.188974 15518 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.220788  6303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3458) took 1032 ms. Trace:
I20260709 12:28:49.219937 15328 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.221352  6303 rpcz_store.cc:276] 0709 12:28:48.188538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.496492 (+307954us) service_pool.cc:224] Handling call
0709 12:28:48.496817 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:48.537329 (+ 40512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:48.537361 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:48.538003 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 12:28:48.710946 (+172943us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:48.711008 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:48.711076 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:49.220551 (+509475us) raft_consensus.cc:1685] finished
0709 12:28:49.220578 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:49.220726 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40124,"raft.queue_time_us":40465,"raft.run_cpu_time_us":44,"raft.run_wall_time_us":43,"thread_start_us":172526,"threads_started":1,"wal-append.queue_time_us":388807}
I20260709 12:28:49.231796 15579 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.233309 15445 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.210762 15641 consensus_queue.cc:260] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.235742 15323 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:49.235767 15515 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.241688 15613 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.271360 15613 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.271883 15613 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:49.272550 15719 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.084s	user 0.001s	sys 0.001s
I20260709 12:28:49.273983 15613 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.242225 15469 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:49.246819  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:49.247345 15682 log.cc:927] Time spent T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 1.143s	user 0.001s	sys 0.001s
W20260709 12:28:49.248817  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.214658 15491 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.251425 15465 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:49.256076 15450 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.260057 15518 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.260402 15427 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:49.261145 15641 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:49.285009  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:49.287557  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.292935  6307 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.288630  6272 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:49.289610  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:49.290512 15469 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:49.291133 15613 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:49.294579  6303 raft_consensus.cc:2393] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:49.297048  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.308898  6302 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:49.299319  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 192004us
I20260709 12:28:49.317581  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:49.300185  6304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3456) took 1119 ms. Trace:
I20260709 12:28:49.307363  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.302534  6272 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:49.319962  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3445) took 1420 ms. Trace:
I20260709 12:28:49.345269  6309 rpcz_store.cc:276] 0709 12:28:47.899874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.912888 (+ 13014us) service_pool.cc:224] Handling call
0709 12:28:49.319720 (+1406832us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54000a9e54. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:49.319889 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37746432}
I20260709 12:28:49.320948  6301 raft_consensus.cc:1240] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:49.336994  7335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43766 (request call id 1320) took 1888 ms (client timeout 2176 ms). Trace:
I20260709 12:28:49.291812 15612 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:49.338052  6307 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:49.305272 15515 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:28:49.309685 15646 consensus_meta.cc:287] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.003s
I20260709 12:28:49.310947 15427 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:49.346648  7335 rpcz_store.cc:269] 0709 12:28:47.430376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.430529 (+   153us) service_pool.cc:224] Handling call
0709 12:28:49.318920 (+1888391us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b54007a0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:49.319067 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1820827520}
I20260709 12:28:49.346123  6272 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:49.311002 15469 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.342916  6304 rpcz_store.cc:276] 0709 12:28:48.180965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.490108 (+309143us) service_pool.cc:224] Handling call
0709 12:28:48.490349 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:48.502869 (+ 12520us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:28:48.503140 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 12:28:49.099173 (+596033us) log.cc:844] Serialized 14 byte log entry
0709 12:28:49.299819 (+200646us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:49.299878 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:49.299931 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.299998 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:49.300123 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141432,"prepare.queue_time_us":325131,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":2105,"raft.queue_time_us":183790,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":162,"thread_start_us":653942,"threads_started":2}
W20260709 12:28:49.361377  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 43903.6us
W20260709 12:28:49.361789  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.362931  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.365465  6300 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3453) took 1185 ms. Trace:
I20260709 12:28:49.363376  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:49.355253 15480 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:49.367420  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.377673  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.380645  6456 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:49.349493 15515 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:49.379294  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.359858 15646 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.366643  6275 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:49.361006 15693 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:49.364068 15676 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.314314 15613 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.381313  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:49.381603  6451 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:49.387365  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:49.374221  6300 rpcz_store.cc:276] 0709 12:28:48.180285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.433377 (+253092us) service_pool.cc:224] Handling call
0709 12:28:48.433635 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:48.435223 (+  1588us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:48.722300 (+287077us) log.cc:844] Serialized 14 byte log entry
0709 12:28:49.094533 (+372233us) log.cc:844] Serialized 14 byte log entry
0709 12:28:49.231634 (+137101us) spinlock_profiling.cc:243] Waited 15.5 ms on lock 0x7b18004558c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1
0709 12:28:49.232464 (+   830us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b18004558c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:49.244989 (+ 12525us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b18004558c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287 00007f6fa32c2208
0709 12:28:49.365124 (+120135us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:49.365181 (+    57us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:49.365228 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.365302 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:49.365415 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153254,"prepare.queue_time_us":15589,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":82,"raft.queue_time_us":154851,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":2416,"spinlock_wait_cycles":49353472,"thread_start_us":764060,"threads_started":3,"wal-append.queue_time_us":424752}
I20260709 12:28:49.391894  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.391985  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.357229 15427 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.394223  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.441004  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:49.399283  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.441718  6457 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:49.440876  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.394032  6425 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:49.446704  6450 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 28: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:49.450091  6302 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 21. Current term is 22. Ops: []
W20260709 12:28:49.450026  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 62752.4us
W20260709 12:28:49.488636  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 94726us
W20260709 12:28:49.493048  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.491472  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.409686  6304 raft_consensus.cc:1240] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 25. Current term is 27. Ops: []
I20260709 12:28:49.400228  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.497027  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:49.501614  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.491472  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.505208  6302 raft_consensus.cc:2410] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 35 because replica has last-logged OpId of term: 34 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 22.
I20260709 12:28:49.512864  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:49.399766  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.373538 15728 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:49.382179 15515 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:49.515782  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:49.392532 15465 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:49.499589  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:49.518510  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 152745us
I20260709 12:28:49.447165 15427 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.438880 15729 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:49.521029  6455 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:28:49.444214 15450 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:49.524343  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.508029 15676 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.490713 15707 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:28:49.439623 15480 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:49.446358 15613 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.499922 15323 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:49.444856 15693 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:49.532648  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:49.552104  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.551733  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.534744 15445 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:49.566738 15740 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.567101 15733 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:49.449636 15518 consensus_queue.cc:1059] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 29.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 12:28:49.570007 15730 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:49.572590 15533 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.575737 15676 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.575872  7319 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 22. (term mismatch)
I20260709 12:28:49.515532 15491 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:49.611070  7319 pending_rounds.cc:85] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 21
I20260709 12:28:49.611387  7319 pending_rounds.cc:107] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 29.22
I20260709 12:28:49.393098 15646 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:49.591646  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59121.7us
I20260709 12:28:49.585269 15729 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
W20260709 12:28:49.596374  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.580272 15727 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:49.611646  7319 raft_consensus.cc:2889] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:49.588634 15445 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:49.624028 15518 raft_consensus.cc:3055] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:49.624706 15518 raft_consensus.cc:740] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:49.611608  6425 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:49.576503 15728 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:49.614688  7285 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.614861 15613 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.623759 15480 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.627168 15728 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.628194 15613 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.576401 15450 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.630872  7329 raft_consensus.cc:2393] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 26: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:49.631631  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.632107 15450 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:49.625741 15518 consensus_queue.cc:260] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.633461  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.594595 15323 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:49.633985  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.757268  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.634828  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.757681  7335 raft_consensus.cc:2393] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 35: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
W20260709 12:28:49.758502  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.634064  7331 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:28:49.634421  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.608497 15742 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:49.623984 15445 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:49.720009 15747 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.733227 15693 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.778404 15747 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.781649 15747 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.638652 15729 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:49.783335 15739 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:49.784389 15729 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.635195 15491 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:49.832332  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.885207  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53970 (request call id 1453) took 1956 ms. Trace:
I20260709 12:28:49.579353  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:49.836112  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 224373us
I20260709 12:28:49.837328  6307 raft_consensus.cc:1217] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Deduplicated request from leader. Original: 19.12->[20.13-20.13]   Dedup: 20.13->[]
W20260709 12:28:49.838606  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:49.612358  6351 connection.cc:256] RPC call timeout handler was delayed by -1.0820375969997258s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17s for a response
I20260709 12:28:49.885695  6301 rpcz_store.cc:276] 0709 12:28:47.928637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.489875 (+561238us) service_pool.cc:224] Handling call
0709 12:28:49.320736 (+830861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:49.346311 (+ 25575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.884601 (+538290us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54000a9e50. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:49.884916 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1910090496}
I20260709 12:28:49.894322  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:49.836100  6305 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:49.895812  6452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59632 (request call id 4032) took 1337 ms. Trace:
I20260709 12:28:49.896334  6452 rpcz_store.cc:276] 0709 12:28:48.557896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:48.558177 (+   281us) service_pool.cc:224] Handling call
0709 12:28:49.895305 (+1337128us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540033b154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:49.895512 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245139200}
I20260709 12:28:49.855116  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1331) took 1131 ms. Trace:
I20260709 12:28:49.873987 15323 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.900101  7318 rpcz_store.cc:276] 0709 12:28:48.722965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.171730 (+448765us) service_pool.cc:224] Handling call
0709 12:28:49.186226 (+ 14496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:49.384902 (+198676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:49.384949 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:49.439274 (+ 54325us) log.cc:844] Serialized 26 byte log entry
0709 12:28:49.753534 (+314260us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:49.753596 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.753704 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:49.854154 (+100450us) raft_consensus.cc:1685] finished
0709 12:28:49.854215 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:49.854667 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46797,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":119,"spinlock_wait_cycles":1337600,"thread_start_us":342965,"threads_started":2,"wal-append.queue_time_us":265501}
I20260709 12:28:49.902957  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:49.904816  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.904769  6425 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:49.855821 15450 raft_consensus.cc:2804] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:28:49.856041  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.856979 15486 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.907932 15450 raft_consensus.cc:697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:49.908119  7320 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:49.908197 15486 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:49.650271  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.735816 15469 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.911458 15469 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:49.885200 15491 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.912446 15486 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.912379 15469 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.761818 15518 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:49.914805 15465 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:49.914979 15454 consensus_queue.cc:1048] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:49.915586 15465 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:49.630388 15613 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.919389 15613 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:49.713721 15727 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.919952  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:49.910056  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.921504  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:49.611207 15733 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.922372  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.923184  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:49.923346  7320 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:49.924804 15733 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.608712 15740 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.928081 15740 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.929242 15469 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.873471 15745 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.930802 15745 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.924161  7318 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:49.932991 15445 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.892668  6307 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:28:49.917593 15465 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.921536 15322 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:49.630306 15728 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.939646 15322 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.906288 15499 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:49.943517 15322 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:49.943928 15499 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.944484 15499 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:49.945703 15469 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.946619 15469 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.933387 15745 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:49.921769  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.879911 15646 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:49.928941  7317 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:49.949316 15747 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:49.953118 15748 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:49.910126 15450 consensus_queue.cc:237] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:49.954173 15748 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.951001 15486 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.956595 15486 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:49.959832  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:49.953150 15747 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:49.964071  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:49.964629 15747 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:49.965102  6377 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 34.
I20260709 12:28:49.966410  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:49.967424  6384 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:28:49.986361 15743 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:28:49.968111  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:49.971185  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:49.979779  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:49.967362 15515 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.013043 15469 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:50.012696 15445 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.028185  6383 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:50.030306  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.005900 15727 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:50.031235  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.035472 15465 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:50.036173  6303 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:49.975163  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:49.993748 15499 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.052896  6305 raft_consensus.cc:1240] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:50.040192 15465 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:50.055742 15465 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:50.057569  6376 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:50.067201  6301 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:50.068883  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:50.076268  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 101520us
I20260709 12:28:50.084045  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:50.077699 15445 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:50.098752 15445 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.098841  6377 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:50.099285 15445 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:50.110698  6310 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57668 (request call id 3470) took 1406 ms. Trace:
I20260709 12:28:50.110975  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:50.111243  6310 rpcz_store.cc:276] 0709 12:28:48.684886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.169203 (+484317us) service_pool.cc:224] Handling call
0709 12:28:50.072709 (+903506us) spinlock_profiling.cc:243] Waited 7.9 ms on lock 0x7b540010f254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:50.091264 (+ 18555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18178304}
I20260709 12:28:50.112247  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:50.114189 15480 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:50.114615 15480 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:50.114991 15480 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:50.115770 15655 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:50.118196 15655 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:50.116218  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.124953  6384 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 35.
I20260709 12:28:50.126935  6383 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:50.126880 15655 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
W20260709 12:28:50.134326  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 213063us
I20260709 12:28:50.134433 15655 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:50.135598  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.151865 15486 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.155968 15515 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:47.817416  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:50.162389 15748 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:50.167403 15759 log.cc:927] Time spent T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:28:50.175225  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.175716  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:50.176251  6385 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:50.176506  6377 raft_consensus.cc:2376] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 35. Re-sending same reply.
I20260709 12:28:50.176678  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.177692  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.178071  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.178355  6376 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 34.
I20260709 12:28:50.178772  6381 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:50.179062  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.179493  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.179822  6379 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:50.180961  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:50.184185 15617 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:50.181090  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:50.186211  6380 raft_consensus.cc:2376] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31. Re-sending same reply.
I20260709 12:28:50.186470 15515 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:50.186995  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.188104  6385 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:50.188961  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:50.189857  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 138788us
I20260709 12:28:50.190193 15629 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:50.190922 15629 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.191330 15629 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:50.191408  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:50.192412  7331 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 35.
I20260709 12:28:50.192538  6458 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1641) took 1319 ms. Trace:
W20260709 12:28:50.186236  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102934us
W20260709 12:28:50.194666  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 300809us
I20260709 12:28:50.196276  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:50.197048  7335 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:50.197081  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.199927  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:50.192880  6458 rpcz_store.cc:276] 0709 12:28:48.873031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:49.171268 (+298237us) service_pool.cc:224] Handling call
0709 12:28:49.171564 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:49.171970 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:49.175531 (+  3561us) log.cc:844] Serialized 14 byte log entry
0709 12:28:49.903886 (+728355us) spinlock_profiling.cc:243] Waited 2.42 ms on lock 0x7b18003888a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:28:49.956292 (+ 52406us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:49.957203 (+   911us) log.cc:844] Serialized 26 byte log entry
0709 12:28:49.957425 (+   222us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:49.957467 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:49.957612 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:50.191938 (+234326us) raft_consensus.cc:1685] finished
0709 12:28:50.192059 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:50.192274 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4525,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":2128,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":314,"spinlock_wait_cycles":5575040,"thread_start_us":38477,"threads_started":1,"wal-append.queue_time_us":922586}
I20260709 12:28:50.208773  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:50.209116  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:50.211030  7331 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:50.212074  6307 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:50.213305 15748 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:50.213822 15748 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:50.214233 15748 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:50.218788 15323 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:50.219532 15323 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:50.222085  7287 leader_election.cc:336] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.772s (SENT)
W20260709 12:28:50.222612  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 107895us
I20260709 12:28:50.222951  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:50.224259  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.225093  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.225310 15747 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:50.225854 15747 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.226209 15747 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:50.226622 15450 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:50.227607 15450 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:50.228782  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:50.229601  6459 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:28:50.229755  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:50.230914  6458 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:50.222061 15323 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:50.232328  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.271055 15731 consensus_queue.cc:1059] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:50.272716 15748 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.252802 15727 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:50.187322 15515 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:50.275542 15612 consensus_meta.cc:287] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.920s	user 0.010s	sys 0.014s
I20260709 12:28:50.272408 15731 raft_consensus.cc:3055] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:50.276157 15731 raft_consensus.cc:740] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:50.277372 15731 consensus_queue.cc:260] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.258874 15328 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.044s
I20260709 12:28:50.277684  6197 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 23 to 24, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 24 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:50.279088  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.279604 15731 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:50.280778  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:50.281687  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:50.282516  6458 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
W20260709 12:28:50.272336  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50488.2us
I20260709 12:28:50.282184 15480 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:50.290982  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.292148  7335 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:28:50.294905 15748 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.291020  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.300242 15642 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.300999 15642 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:50.301740 15729 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:28:50.302289 15729 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.302779 15729 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:28:50.303061  6454 raft_consensus.cc:2410] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
W20260709 12:28:50.305269  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.277321 15515 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:50.307683  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.308261  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:50.300921  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.320573  7317 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43762 (request call id 1252) took 2499 ms. Trace:
W20260709 12:28:50.321890  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.326185  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.326879  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.327714  6385 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
W20260709 12:28:50.328723 15544 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 22 current term: 22 is higher.
I20260709 12:28:50.329471 15579 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:50.330044 15579 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:50.330590  6454 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:59648 (request call id 3895) took 2836 ms (client timeout 1772 ms). Trace:
W20260709 12:28:50.331434  6454 rpcz_store.cc:269] 0709 12:28:47.494348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.494519 (+   171us) service_pool.cc:224] Handling call
0709 12:28:50.330200 (+2835681us) spinlock_profiling.cc:243] Waited 2.75 s on lock 0x7b54000c0dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:50.330440 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6315251456}
I20260709 12:28:50.311826 15748 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:50.333523 15748 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:50.334008 15748 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:50.326990  6384 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:28:50.293483 15612 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:50.311715 15747 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.345431 15747 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.347054  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:50.347332 15322 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.351179  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:50.353207  6459 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 16. (term mismatch)
I20260709 12:28:50.353662  6459 pending_rounds.cc:85] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 15
I20260709 12:28:50.353943  6459 pending_rounds.cc:107] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 28.16
I20260709 12:28:50.355018  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:50.357095  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:50.360602  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:50.361416  6379 raft_consensus.cc:3060] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:50.362952  6383 raft_consensus.cc:2393] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 30: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:50.369884  6376 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:50.354272  6459 raft_consensus.cc:2889] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:50.336047  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.321094  7317 rpcz_store.cc:276] 0709 12:28:47.821703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:47.821808 (+   105us) service_pool.cc:224] Handling call
0709 12:28:47.861046 (+ 39238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:49.927537 (+2066491us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b58005963d0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32030aa 00007f6fa3285447 00007f6fa329dfe7 00007f6fa329f7de 00007f6fa32a9b49 00007f6fa32aa416 00007f6fa32a6d8f 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4
0709 12:28:50.309714 (+382177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:50.309757 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:50.313864 (+  4107us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:50.313925 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:50.314018 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:50.320471 (+  6453us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3827,"raft.queue_time_us":19987,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":7675,"spinlock_wait_cycles":2975956352}
W20260709 12:28:50.422560  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 121803us
I20260709 12:28:50.375737 15612 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:50.429991  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 110426us
W20260709 12:28:50.465072  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.334043 15579 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:50.314153 15655 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:50.426401 15613 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:28:50.489459  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 115794us
I20260709 12:28:50.501873  6385 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:50.481765 15612 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:50.448597 15515 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:50.512246  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:28:50.513574  6198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 269) took 33470 ms (client timeout 15000 ms). Trace:
W20260709 12:28:50.514803  7287 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.144s (SENT)
I20260709 12:28:50.544518  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:50.515607  6383 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:50.517514  7332 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: []
W20260709 12:28:50.518680  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 179673us
I20260709 12:28:50.524315 15629 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.527819 15480 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:50.531092 15457 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.518719  6377 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:28:50.496609 15613 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:50.554343  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.560068  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:50.510478 15322 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.570525  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:50.543018 15450 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:50.581732  6458 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:50.584885  6379 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:50.580310 15486 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.007s
I20260709 12:28:50.566365 15613 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:50.589063 15629 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:50.568075 15328 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
W20260709 12:28:50.583688 15465 consensus_meta.cc:287] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.527s	user 0.013s	sys 0.000s
I20260709 12:28:50.591030 15465 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.592067 15118 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:28:50.592263  6456 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:50.594444  6377 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:28:50.588866  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:50.543812  6198 rpcz_store.cc:269] 0709 12:28:16.867963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:26.761560 (+9893597us) service_pool.cc:224] Handling call
0709 12:28:50.314399 (+23552839us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:41.554931 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:41.639106 (+ 84175us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:41.639169 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:28:41.639186 (+    17us) tablet.cc:662] Decoding operations
0709 12:28:41.641140 (+  1954us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:41.641240 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:41.641276 (+    36us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:28:41.777309 (+136033us) tablet.cc:701] Row locks acquired
0709 12:28:41.777367 (+    58us) write_op.cc:260] PREPARE: finished
0709 12:28:41.777557 (+   190us) write_op.cc:270] Start()
0709 12:28:41.777708 (+   151us) write_op.cc:276] Timestamp: P: 1783600121777493 usec, L: 0
0709 12:28:41.777749 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:28:41.778733 (+   984us) log.cc:844] Serialized 8437 byte log entry
0709 12:28:42.352453 (+573720us) op_driver.cc:464] REPLICATION: finished
0709 12:28:42.522977 (+170524us) write_op.cc:301] APPLY: starting
0709 12:28:42.548887 (+ 25910us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:43.164857 (+615970us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:43.175242 (+ 10385us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:44.227879 (+1052637us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:44.228527 (+   648us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:28:44.228597 (+    70us) write_op.cc:312] APPLY: finished
0709 12:28:44.229318 (+   721us) log.cc:844] Serialized 196 byte log entry
0709 12:28:44.367326 (+138008us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:44.367586 (+   260us) write_op.cc:454] Released schema lock
0709 12:28:44.367670 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:44.397485 (+ 29815us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:44.397527 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3963187,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":102068,"apply.run_cpu_time_us":49349,"apply.run_wall_time_us":1887857,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96131,"cfile_init":1,"lbm_read_time_us":429,"lbm_reads_lt_1ms":4,"num_ops":22,"prepare.queue_time_us":385051,"prepare.run_cpu_time_us":16546,"prepare.run_wall_time_us":356283,"raft.queue_time_us":157461,"raft.run_cpu_time_us":9575,"raft.run_wall_time_us":72934,"replication_time_us":573117,"thread_start_us":575447,"threads_started":5,"wal-append.queue_time_us":494802,"wal-append.run_cpu_time_us":19946,"wal-append.run_wall_time_us":2152509}]]}
I20260709 12:28:50.601472  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.600500  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.591342  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:50.600507  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.601348  6382 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:50.603017  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:50.602260  6198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 276) took 18573 ms (client timeout 15000 ms). Trace:
I20260709 12:28:50.511195 15445 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:50.606602  6198 rpcz_store.cc:269] 0709 12:28:32.029016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.602023 (+18573007us) service_pool.cc:211] Skipping call since client already timed out
0709 12:28:50.602184 (+   161us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:28:50.608857  7331 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:50.563184 15515 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:50.623920  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.625880  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.629377  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.631872  6381 raft_consensus.cc:3060] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:50.626061  6275 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:50.636345  6376 raft_consensus.cc:2410] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 32 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:28:50.639709  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:50.640059 15118 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:50.640535  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:50.640901 15118 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:50.641376  6304 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:50.567515 15457 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:50.640182  6377 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:50.634929  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.621019 15629 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:50.630419  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:50.650244  7332 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:50.580562 15674 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.206s
I20260709 12:28:50.622637 15499 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:50.656180  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.658969  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.660887  6384 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:50.607730 15729 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.670576  7319 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:28:50.671267  6274 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.892s (SENT)
I20260709 12:28:50.671813  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:50.672528 15515 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.674243  6378 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:50.673095  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94201.3us
I20260709 12:28:50.641990 15118 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:50.675805  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 161094us
I20260709 12:28:50.651401 15629 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:50.696035  6385 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:28:50.696532  6385 pending_rounds.cc:85] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 12
I20260709 12:28:50.696792  6385 pending_rounds.cc:107] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:28:50.697234  6385 raft_consensus.cc:2889] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:50.670634 15544 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:50.698818  6382 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:50.710171  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.711081 15322 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:50.712216 15322 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:50.712597 15322 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:50.715495 15445 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.718806 15642 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:50.722738  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:50.723511 15764 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.217s	user 0.000s	sys 0.002s
I20260709 12:28:50.723837 15736 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:50.687107 15629 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:50.708240  7317 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:50.713687 15322 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.733558  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:50.741716  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:50.707024 15544 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.744287  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:50.750238  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:50.749045  7287 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.178s (SENT)
I20260709 12:28:50.750942  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:50.749429  7331 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:50.736474  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:50.749694  7329 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:28:50.752676  6303 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 20.
I20260709 12:28:50.762835  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.765028  6376 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:50.768149  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:50.753783 15544 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:50.750810 15655 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:50.777020  6310 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:50.780479  6301 raft_consensus.cc:2368] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:50.648815  6197 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 18 to 20, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.791361  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:50.795138  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.799413  6380 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:50.805722  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:50.807590  6382 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:28:50.807964  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:50.822937  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 80101.1us
I20260709 12:28:50.828576  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:50.834803  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.797807 15499 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.018s
I20260709 12:28:50.835215  6305 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:50.845561  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.862946 15742 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:50.867839 15770 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
W20260709 12:28:50.868633  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.864184  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:50.870867  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:50.879577  6300 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:50.870960 15729 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 36 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.871388 15544 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.870883  6310 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:50.874507  6301 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:50.795495 15613 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.886972  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:50.906200  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.907269  6380 raft_consensus.cc:2368] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30.
W20260709 12:28:50.908769 15769 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.123s	user 0.000s	sys 0.000s
I20260709 12:28:50.928521 15613 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:50.941489  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:50.945374  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.946525  6380 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
W20260709 12:28:50.950462  6424 leader_election.cc:336] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.675s (SENT)
W20260709 12:28:50.900787 15748 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.005s
W20260709 12:28:50.952666  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52610.3us
I20260709 12:28:50.960058  6376 raft_consensus.cc:1217] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Deduplicated request from leader. Original: 23.18->[24.19-31.21]   Dedup: 24.19->[27.20-31.21]
I20260709 12:28:50.965354  6309 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:28:50.951121  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:50.963681  6376 pending_rounds.cc:85] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 19
I20260709 12:28:50.969619  6376 pending_rounds.cc:107] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 26.20
I20260709 12:28:50.970044  6376 raft_consensus.cc:2889] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:50.962950 15770 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:28:50.984993  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.987195  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:50.988116  6380 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:50.989284  6382 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:50.989768  6275 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:51.000257  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.000885  6382 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:50.983359  7331 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
W20260709 12:28:51.005975  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:50.963165 15748 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:50.992769 15729 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:28:51.013712 15673 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:51.015902  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:51.019129 15775 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:50.994721 15655 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:28:51.024881  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.031337  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:51.035147  6382 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:51.003754  7332 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:51.039290  6381 raft_consensus.cc:2468] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:28:51.043759  7319 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:51.044129  6300 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:51.047886  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.047964  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:51.076615  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.092161  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60925.2us
I20260709 12:28:51.092386  6377 raft_consensus.cc:1275] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:51.096875  6307 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:51.097456  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:51.098115  7331 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:28:51.109508  6308 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: []
W20260709 12:28:51.111881  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:51.112514  6383 raft_consensus.cc:1240] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35. Ops: [34.23-34.23]
I20260709 12:28:51.112828  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.113466  6378 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:51.113749  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.114445  6384 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:51.114765  6380 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:51.114852  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.115296  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.115751  6381 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:51.115923  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.116405  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.117712  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.119103  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.119756  6383 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:51.120631  7319 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
W20260709 12:28:51.121147  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 170745us
I20260709 12:28:51.043589 15673 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:51.134855 15673 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:51.136158  6424 leader_election.cc:336] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.869s (SENT)
I20260709 12:28:51.136617  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:51.137166  6378 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:51.132840  7331 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 36.
I20260709 12:28:51.141347  7288 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:51.156805  6275 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:51.158001  6275 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:51.164929  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.169164  6378 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:51.103214 15731 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:51.003029 15612 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.103528 15655 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:51.173822  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.174265  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 189141us
W20260709 12:28:51.177911  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 272386us
I20260709 12:28:51.167057 15445 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:51.163954 15727 raft_consensus.cc:2804] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:51.163698 15729 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:28:51.167629 15775 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:51.173266 15731 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:51.186153  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:51.190383  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.194898  6383 raft_consensus.cc:1275] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:28:51.219200  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.207543  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.224610  6383 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:51.228381  6425 leader_election.cc:400] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:51.228824  6425 leader_election.cc:403] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:28:51.230412 15323 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:51.231549  6378 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:51.232563  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.233320 15210 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:51.233857  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.236397 15612 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:51.237896  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.239013  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.239547  6378 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:51.192305 15445 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:51.240803 15445 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.198210 15499 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:28:51.242355 15323 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:51.242861 15499 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:51.243511 15499 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:51.249401  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.250137  6382 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:51.250533 15544 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:51.251204  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.251780  6384 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:51.252199 15322 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:51.253607  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.255072  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.256278  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.256824  6380 raft_consensus.cc:2376] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30. Re-sending same reply.
I20260709 12:28:51.257616  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.258093  6380 raft_consensus.cc:2376] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24. Re-sending same reply.
W20260709 12:28:51.266995 15629 consensus_meta.cc:287] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.522s	user 0.013s	sys 0.000s
I20260709 12:28:51.267467 15629 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.268631  7319 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:51.269706 15629 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:51.196115 15731 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:51.230432  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:51.279417  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:51.198935 15729 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 36 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:28:51.286849  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 56516.4us
I20260709 12:28:51.294520  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.324772  7318 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1335) took 1034 ms. Trace:
I20260709 12:28:51.331775  6378 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:51.332530 15457 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:51.225135 15118 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 12:28:51.332830 15617 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:51.338995  6384 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:51.266327 15673 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:51.356640 15465 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:51.435274  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.498771  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.369624 15773 log.cc:927] Time spent T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:28:51.491132 15727 raft_consensus.cc:697] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:51.499620  6425 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:51.330899  7318 rpcz_store.cc:276] 0709 12:28:50.290705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.307232 (+ 16527us) service_pool.cc:224] Handling call
0709 12:28:50.350409 (+ 43177us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:50.352878 (+  2469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:50.352913 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:50.522357 (+169444us) log.cc:844] Serialized 52 byte log entry
0709 12:28:50.809257 (+286900us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:51.288486 (+479229us) log.cc:844] Serialized 14 byte log entry
0709 12:28:51.324038 (+ 35552us) log.cc:844] Serialized 14 byte log entry
0709 12:28:51.324345 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:51.324444 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:51.324504 (+    60us) raft_consensus.cc:1685] finished
0709 12:28:51.324521 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:51.324687 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":298313,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":100,"raft.queue_time_us":37636,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":18583,"spinlock_wait_cycles":84507264,"thread_start_us":744767,"threads_started":2,"wal-append.queue_time_us":525340}
I20260709 12:28:51.504943  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:51.399065  7287 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.120s (SENT)
I20260709 12:28:51.507977 15642 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:51.508152 15779 raft_consensus.cc:2804] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:51.508498 15642 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:51.508903 15779 raft_consensus.cc:697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:51.509758 15642 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.511694 15642 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:51.514981  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:51.515856  7317 raft_consensus.cc:2368] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:51.466979 15729 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.411984 15748 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:51.399971 15518 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:51.500738 15727 consensus_queue.cc:237] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.523887  7335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1337) took 1203 ms. Trace:
I20260709 12:28:51.524510  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:51.525316  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:51.526475 15748 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.521100 15457 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:51.535174  7318 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:51.531935  7335 rpcz_store.cc:276] 0709 12:28:50.315381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.315579 (+   198us) service_pool.cc:224] Handling call
0709 12:28:50.315856 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:50.487134 (+171278us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:28:50.946444 (+459310us) log.cc:844] Serialized 14 byte log entry
0709 12:28:51.490365 (+543921us) log.cc:844] Serialized 14 byte log entry
0709 12:28:51.504238 (+ 13873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:51.504295 (+    57us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:51.518854 (+ 14559us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:51.518963 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:51.519067 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13518,"prepare.queue_time_us":618679,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":87,"raft.queue_time_us":45435,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":193,"thread_start_us":917822,"threads_started":3,"wal-append.queue_time_us":380190}
I20260709 12:28:51.541421  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:51.542850  6303 raft_consensus.cc:2368] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27.
I20260709 12:28:51.539352  6300 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:28:51.565692  6274 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.503s (SENT)
W20260709 12:28:51.588795  6424 leader_election.cc:336] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.953s (SENT)
I20260709 12:28:51.590893  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.591674  6382 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:51.593333 15731 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:51.594624  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:51.595110  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:51.597545  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.598610  6380 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:51.599663  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.599973  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.600654  6376 raft_consensus.cc:3060] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:51.601012  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.601758  6380 raft_consensus.cc:2376] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27. Re-sending same reply.
I20260709 12:28:51.602982  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:51.606452  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 71458.2us
W20260709 12:28:51.608139  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:51.615015  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.616112  6378 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:51.620014  6425 leader_election.cc:400] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28.
I20260709 12:28:51.553500  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:51.634758  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.535547 15673 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:51.531556 15518 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.510076 15779 consensus_queue.cc:237] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.648490  7317 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:51.648689  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:51.649574  6424 leader_election.cc:336] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.212s (SENT)
I20260709 12:28:51.394424 15655 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.662528  6379 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 33.
I20260709 12:28:51.646370  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.650907  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.651286  6378 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:51.565629 15457 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:51.655081  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 256056us
I20260709 12:28:51.657292  6382 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:51.664371  6384 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 35.
I20260709 12:28:51.563248 15748 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:51.668047  6380 raft_consensus.cc:2368] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30.
I20260709 12:28:51.671510  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:51.672209  6456 raft_consensus.cc:3060] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:51.673097  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.672236  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:51.674077  6454 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:51.661571 15799 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:28:51.676213  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.670961  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:51.677266  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.681051  6451 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:51.678105  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.682000  6384 raft_consensus.cc:2393] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:51.682335  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.662405 15518 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:51.683023  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.684361  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.685025  6384 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:51.685992  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.683081  6378 raft_consensus.cc:2468] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:51.688390  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.689023  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.693572  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.694249  6379 raft_consensus.cc:3060] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:51.695232  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:51.696182  6455 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:28:51.698925  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.699712  6382 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:51.700692  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.701305  6382 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:51.709640  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:51.710685  6455 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:51.702257  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.713711  6456 raft_consensus.cc:2468] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:28:51.715210  6376 raft_consensus.cc:2468] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:28:51.715377  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.716363  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.689843  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.719317  6378 raft_consensus.cc:2376] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:28:51.721823  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.723495  6385 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:51.675721 15792 log.cc:927] Time spent T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.170s	user 0.001s	sys 0.001s
I20260709 12:28:51.725868  6454 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:51.727209  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.728726  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.730897  6377 raft_consensus.cc:2376] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27. Re-sending same reply.
I20260709 12:28:51.644881 15499 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:51.741523  6378 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:51.742453  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.744865  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:51.745968  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.748513  6382 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:51.767915  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.768819  6378 raft_consensus.cc:2376] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27. Re-sending same reply.
W20260709 12:28:51.771555  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.772569  6274 leader_election.cc:336] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.692s (SENT)
I20260709 12:28:51.773118  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:51.774046  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.774686  6377 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:28:51.776235  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.780897  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.676762 15515 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 21, Time since last communication: 0.069s
W20260709 12:28:51.785881  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 148740us
I20260709 12:28:51.786267  6379 raft_consensus.cc:2468] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:51.789548  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.790249  6382 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:51.791961  6454 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:51.794955  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.798617  6378 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:51.799520  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.799889  6457 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:51.800238  6454 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:51.795694  6380 raft_consensus.cc:2376] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23. Re-sending same reply.
I20260709 12:28:51.801784  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.788362 15445 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:28:51.804626  6307 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:28:51.806950  7287 leader_election.cc:336] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.888s (SENT)
I20260709 12:28:51.807420  7287 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:51.809039  7287 leader_election.cc:336] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 3.256s (SENT)
I20260709 12:28:51.809427  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:51.812480  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.813198  6385 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:28:51.815464  6382 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:28:51.810702 15806 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:51.816936 15579 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:51.817770  6424 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.207s (SENT)
I20260709 12:28:51.818068 15748 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:51.823975  6274 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.761s (SENT)
I20260709 12:28:51.825259  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:51.827072  7285 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:51.827888  7287 leader_election.cc:336] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.982s (SENT)
I20260709 12:28:51.828392  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:51.828861 15797 raft_consensus.cc:2804] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:51.829888  6379 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:51.819180 15731 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:51.830355  6379 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:51.832415  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:51.832679  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.833460  6382 raft_consensus.cc:2376] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25. Re-sending same reply.
I20260709 12:28:51.814459 15328 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:51.834518  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.812801 15486 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:51.832295  6380 raft_consensus.cc:3055] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:51.830996  6378 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:51.843003  6385 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:51.842097  6380 raft_consensus.cc:740] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:51.793869 15807 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:28:51.863595  6377 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 36.
I20260709 12:28:51.865099  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.866190  6377 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:51.871603  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.875564  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.835945 15796 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:51.876405  6378 raft_consensus.cc:2376] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22. Re-sending same reply.
I20260709 12:28:51.844143 15486 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:51.843997 15748 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:51.874375  6379 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.829849 15457 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.882957  6380 consensus_queue.cc:260] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:51.866871 15322 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:51.866621 15775 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:51.885923  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66860.5us
I20260709 12:28:51.886540  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.894084  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.881804  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.868283 15731 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:51.899537  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.901875  6382 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:51.905807  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.906745  6381 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 31.
I20260709 12:28:51.911089  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.911892  6377 raft_consensus.cc:2376] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25. Re-sending same reply.
I20260709 12:28:51.915611  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:51.915881  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:51.916743  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92830.7us
I20260709 12:28:51.919636  6377 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:51.920809  6380 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:51.921934  6378 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:28:51.909502 15322 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.925307 15322 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:51.925415 15499 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:51.926179  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:51.922181  6274 leader_election.cc:336] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.843s (SENT)
I20260709 12:28:51.927381  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:51.927434 15518 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:51.929164 15807 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:51.902787 15775 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:51.933159 15775 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:51.934388  6381 raft_consensus.cc:1275] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:28:51.935695  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:51.928630  6274 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.846s (SENT)
W20260709 12:28:51.936365  6424 leader_election.cc:336] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.346s (SENT)
I20260709 12:28:51.936914  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:51.967415  6455 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:28:51.970657  6380 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:28:51.971539 15486 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.977267 15486 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:51.978999 15810 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:28:51.985441 15328 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:51.986925  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.987566  6377 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:51.991075  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.915076 15797 raft_consensus.cc:697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:51.909315 15731 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:51.986274 15328 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.999390  6379 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:51.947750 15796 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:52.005522  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:52.008662  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:51.972532 15807 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:52.008255 15328 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:52.002297 15813 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.075s	user 0.002s	sys 0.002s
I20260709 12:28:52.011929 15796 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.013214  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.009500  6385 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 31.
I20260709 12:28:52.018826  6457 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
W20260709 12:28:52.024842  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96240.9us
I20260709 12:28:52.035763  6379 raft_consensus.cc:1275] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:28:52.042963  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.043629  6459 raft_consensus.cc:2376] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28. Re-sending same reply.
W20260709 12:28:52.047021  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:52.024374 15518 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:52.064103 15328 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:52.067998  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 148680us
W20260709 12:28:52.074750 15814 log.cc:927] Time spent T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:28:52.090862  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:52.098625  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:52.095079  7287 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.217s (SENT)
I20260709 12:28:52.147589  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:52.087862 15797 consensus_queue.cc:237] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.104408  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:52.106251  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:52.149554  6383 raft_consensus.cc:2393] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:52.082043 15815 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:28:52.066205 15731 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:52.110750  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.119626  6451 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:52.154240  6457 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
W20260709 12:28:52.127942  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46733.5us
I20260709 12:28:52.154755  6452 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 30. Current term is 31. Ops: [30.22-30.22]
W20260709 12:28:52.144869  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:52.046166  6274 leader_election.cc:336] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.908s (SENT)
W20260709 12:28:52.150882  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 55871.1us
W20260709 12:28:52.152374  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:52.156873  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:52.152664  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 104999us
I20260709 12:28:52.158295  6425 leader_election.cc:400] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27.
I20260709 12:28:52.161158  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:52.161487  6425 leader_election.cc:403] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:28:52.160351 15118 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:52.154106 15629 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:52.173149  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.173903  6310 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:52.173940  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:52.179073  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.179811  6300 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:52.168622 15793 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:52.174116 15457 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27.
I20260709 12:28:52.190963 15793 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.195364 15793 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:52.210856 15629 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:52.179692 15518 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:28:52.213332  6310 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:52.181233 15499 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:52.214356 15499 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.214776 15499 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:52.215269  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.217625 15793 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:52.223325  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 177173us
W20260709 12:28:52.225310  6274 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.061s (SENT)
I20260709 12:28:52.226979 15775 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:52.227660 15775 consensus_queue.cc:1243] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:52.228833 15775 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:52.230860  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.231473  6309 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:52.231933  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.233523  6300 raft_consensus.cc:2468] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:52.236577  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.237807  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.239087  6303 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:28:52.240449  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:52.240702  6454 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
W20260709 12:28:52.240617  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49097.9us
W20260709 12:28:52.241853  6274 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.765s (SENT)
I20260709 12:28:52.242865  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:52.245853  6310 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:28:52.249678  7331 raft_consensus.cc:1240] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 30. Current term is 31. Ops: [30.20-30.20]
I20260709 12:28:52.243294 15118 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:52.255240 15118 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.256824  6351 leader_election.cc:400] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:28:52.257177  6351 leader_election.cc:403] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:28:52.253345 15736 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:52.261132 15613 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:52.261704 15613 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:52.264786  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.268884 15834 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:28:52.278116  7329 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:28:52.280458  6456 raft_consensus.cc:1217] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Deduplicated request from leader. Original: 19.13->[23.14-23.14]   Dedup: 23.14->[]
I20260709 12:28:52.281185  6456 raft_consensus.cc:1240] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
W20260709 12:28:52.285188 15798 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.532s	user 0.000s	sys 0.004s
I20260709 12:28:52.287753  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.289529  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:52.295508 15796 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:52.296209  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.296530  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.297621  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.297726  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.298017  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.298841  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.297642  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.299345  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.299782 15793 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:52.306310  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.275748 15118 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:52.320349  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102124us
W20260709 12:28:52.334267  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:52.298316  6454 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:52.235832 15779 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.042s
W20260709 12:28:52.366598  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66034.9us
I20260709 12:28:52.356452  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:52.363530  7329 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:52.368156  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.370924  6455 raft_consensus.cc:1240] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:28:52.371258  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.372555  6450 raft_consensus.cc:2368] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
W20260709 12:28:52.373063  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:52.374074  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.357681 15834 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:52.388233 15748 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:28:52.376392 15491 consensus_queue.cc:1059] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.016s
I20260709 12:28:52.391248 15491 raft_consensus.cc:3055] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:52.391731 15491 raft_consensus.cc:740] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:52.391428  7319 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:28:52.392714 15491 consensus_queue.cc:260] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.393733 15729 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:52.394196 15729 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:52.394647 15729 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:52.396042 15322 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:52.398814 15118 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:52.400488  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.400758  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:52.401268  6459 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:52.401322  6456 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:52.406112  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 164267us
I20260709 12:28:52.406457  6272 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:52.407088  6274 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.795s (SENT)
I20260709 12:28:52.414335  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.417124  6309 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:52.410647  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:52.394207 15491 raft_consensus.cc:3060] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:52.425762  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.428614  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:52.433743  6457 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:52.413101 15802 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:52.437161  7288 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:52.446012  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.447549  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.412813 15655 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:28:52.449234  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.410719 15322 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:28:52.452407  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.429492 15736 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:52.452494 15802 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:52.463351  6300 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:52.463239 15802 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:52.464622  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:52.465905 15794 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:52.466562 15817 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:52.467108 15817 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.469749 15817 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:52.441171 15809 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:52.470973  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:52.472777  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 45826.7us
I20260709 12:28:52.500792  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.477748  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.505177  6457 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:52.453088 15727 raft_consensus.cc:2804] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:28:52.509881  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57416.9us
I20260709 12:28:52.511353  6453 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
W20260709 12:28:52.514288  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 107234us
I20260709 12:28:52.516326  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.518980  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.519960  6451 raft_consensus.cc:2368] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
W20260709 12:28:52.518010 15841 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.058s	user 0.003s	sys 0.005s
I20260709 12:28:52.503322 15613 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:52.525537 15727 raft_consensus.cc:697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:28:52.539291  6274 leader_election.cc:336] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.179s (SENT)
I20260709 12:28:52.540956  6459 raft_consensus.cc:2468] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:52.515529 15809 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:52.546823  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.548167  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:52.558246  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47372us
I20260709 12:28:52.550393  6450 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:28:52.549700 15835 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:52.562109  6302 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
W20260709 12:28:52.570330 15837 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:28:52.571382  6303 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:52.578457  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.588840  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:52.583216 15838 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:52.593926 15804 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:52.608065  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.610395  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.611265  6453 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:28:52.612313  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:52.612864  6452 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:28:52.623106 15809 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.626152  7285 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:52.644805  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:52.634202  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 187224us
W20260709 12:28:52.677258  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 51185us
I20260709 12:28:52.679422  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:52.680534 15629 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:52.680720  7285 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:52.681113 15629 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:52.683591  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.684827  6459 raft_consensus.cc:2368] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25.
I20260709 12:28:52.692939  6456 raft_consensus.cc:1275] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:28:52.695518  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:52.691010  6425 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:52.567936 15499 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:52.671919 15804 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:52.704954  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.681643 15629 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:52.682747 15796 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:52.695019 15731 raft_consensus.cc:2804] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:52.592610 15486 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.107s
W20260709 12:28:52.708940  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 116547us
I20260709 12:28:52.713153  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:52.713948  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:52.698374 15832 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:52.682950 15817 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:28:52.722499  6450 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:52.724365  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.745502  6309 raft_consensus.cc:2410] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 31 because replica has last-logged OpId of term: 30 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 19.
I20260709 12:28:52.668005 15729 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:52.728838 15835 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:52.756174  6348 connection.cc:256] RPC call timeout handler was delayed by -3.2270109780001803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:28:52.758935  6348 connection.cc:256] RPC call timeout handler was delayed by -3.1703109780000887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 12:28:52.759413  6348 connection.cc:256] RPC call timeout handler was delayed by -2.938110978000168s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2093s for a response
I20260709 12:28:52.697952 15727 consensus_queue.cc:237] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.760214  6425 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:52.730043 15847 raft_consensus.cc:2804] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:52.745504 15673 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:52.766969 15673 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.767516 15673 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:52.771147  6348 connection.cc:256] RPC call timeout handler was delayed by -2.7995109779999439s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2247s for a response
I20260709 12:28:52.728766 15794 consensus_queue.cc:1048] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.025s
I20260709 12:28:52.773432  6452 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:28:52.775451 15731 raft_consensus.cc:697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:52.778652 15731 consensus_queue.cc:237] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.743552 15845 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:52.807956  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:52.809531  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 104176us
I20260709 12:28:52.813635 15729 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:52.815291  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:52.817137  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 57022.9us
W20260709 12:28:52.817044  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 56822.3us
I20260709 12:28:52.820281  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:52.823868  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:52.787321 15457 consensus_queue.cc:1048] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:28:52.834743 15499 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:52.834944 15457 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:52.825474 15775 consensus_queue.cc:1048] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.013s
I20260709 12:28:52.831194 15847 raft_consensus.cc:697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:52.842680 15842 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:28:52.843802 15847 consensus_queue.cc:237] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:52.832515  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:52.850859 15784 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:52.851671 15784 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:52.852739 15784 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:52.853689  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.854862  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.856282  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.856909 15629 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.857472  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.858134  6310 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:52.858834  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.861557  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.862951 15629 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:52.863013  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:52.864265 15846 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:52.864734 15846 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:52.865090 15846 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:28:52.895946  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 45455.2us
I20260709 12:28:52.824793 15729 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.825264 15779 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:28:52.905129  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.905933  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:52.909432  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 107061us
I20260709 12:28:52.912966  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.899619 15832 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:52.916649  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88238.5us
I20260709 12:28:52.917609  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.916476  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:52.923784  6310 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:52.925278  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:52.926585  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63672.5us
I20260709 12:28:52.927592  6424 leader_election.cc:400] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30.
I20260709 12:28:52.927935  6424 leader_election.cc:403] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:52.928823  6300 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:52.905248 15673 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:52.935423  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:52.916445 15779 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:52.934808  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:52.938202  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:52.939255  6383 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:52.939610  6379 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:52.939796  6301 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:52.937940  6348 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.941411  6385 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:28:52.910882 15729 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:52.943913 15549 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:52.948037 15849 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:52.948472 15863 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.122s	user 0.002s	sys 0.001s
I20260709 12:28:52.956233  6310 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:52.957350  6308 raft_consensus.cc:1240] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:52.940577 15855 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30.
I20260709 12:28:52.972465  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:52.975061 15779 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:52.978562 15673 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:52.980103 15857 log.cc:927] Time spent T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.144s	user 0.002s	sys 0.000s
W20260709 12:28:52.982777 15860 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
I20260709 12:28:52.987674  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:52.988344  6310 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:52.967352 15832 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:52.996150 15673 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:52.998221 15673 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:52.967646 15849 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.003074 15673 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:53.013408  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:53.013319  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:53.015844  6458 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:53.022291  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:53.022907  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:53.023135  7317 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:28:53.023494  6456 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:53.026458 15729 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:53.027643  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 97805.6us
I20260709 12:28:53.028071  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:53.028762  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.029825 15832 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:53.045528  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:53.045678  6310 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:28:53.050874  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.052434 15673 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.050220  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:53.054193 15832 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:53.056517 15642 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:53.073601  6456 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:53.074893  7317 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:53.082064  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:53.087940  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 548653us
W20260709 12:28:53.089627  6274 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.232s (SENT)
W20260709 12:28:53.099948  6274 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.361s (SENT)
I20260709 12:28:53.081558 15673 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:53.102170  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.103204  6383 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
W20260709 12:28:53.104597  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54488.4us
I20260709 12:28:53.105875  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:53.106199  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:53.106966  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:53.109011  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:53.109836  6383 raft_consensus.cc:2393] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 23: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:53.111477  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.112246  6383 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:53.113857 15836 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:53.114063  6383 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:28:53.114289 15836 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:53.115362 15836 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.116225  6383 raft_consensus.cc:3055] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:53.117426  6383 raft_consensus.cc:740] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:53.120839  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.122407  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.122462  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.124162  6425 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:53.125298  6376 raft_consensus.cc:2376] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28. Re-sending same reply.
I20260709 12:28:53.106966  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.116822 15836 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:53.127128  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:53.161270  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 54414.6us
I20260709 12:28:53.169287  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:53.170046  6384 raft_consensus.cc:2376] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 36. Re-sending same reply.
I20260709 12:28:53.173020  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.177403  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:53.179378  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52348.9us
I20260709 12:28:53.182386  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:53.182996  7331 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:53.123507 15457 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:53.159269 15642 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:53.178366 15465 raft_consensus.cc:2804] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:53.180197 15862 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:53.179978 15797 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:28:53.223732  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 116824us
I20260709 12:28:53.224478  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:53.224696  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 102363us
I20260709 12:28:53.183959  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:53.159879  6383 consensus_queue.cc:260] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.225586  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:53.183288 15796 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:53.225967  7319 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 28: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:53.176563 15731 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.055s
W20260709 12:28:53.227662  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 140114us
I20260709 12:28:53.259387  7331 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:28:53.260938  6383 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:53.210464 15846 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.232815 15465 raft_consensus.cc:697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:53.230420 15642 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:28:53.269279  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.242343 15322 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:53.296547  6348 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.242800 15849 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:53.308219  6452 raft_consensus.cc:1240] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 34 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 33. Current term is 34. Ops: []
I20260709 12:28:53.230134 15865 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
W20260709 12:28:53.325065  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60427.5us
I20260709 12:28:53.277855 15878 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:53.304390 15322 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:53.331835 15880 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:53.336957  6383 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:28:53.338658 15322 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:53.321346 15748 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:28:53.358093  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.361514  6300 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:28:53.364897 15875 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:28:53.322520 15796 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.369566 15465 consensus_queue.cc:237] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:53.379034  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53320.8us
I20260709 12:28:53.383678  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:53.352873 15849 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:28:53.336472 15784 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:28:53.360903 15642 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:53.389991 15879 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.173s	user 0.003s	sys 0.000s
I20260709 12:28:53.352432 15883 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:53.391296 15883 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.391736 15883 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:53.349758 15797 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:53.383896 15809 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:53.405977  6309 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 20. (term mismatch)
I20260709 12:28:53.406440  6309 pending_rounds.cc:85] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 19
I20260709 12:28:53.406265  6424 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:53.406870  6309 pending_rounds.cc:107] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 30.20
I20260709 12:28:53.407182  6309 raft_consensus.cc:2889] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:53.408852  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50819.2us
I20260709 12:28:53.399025 15499 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:53.417608  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.423300 15843 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:53.425477 15809 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:53.426240 15809 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:53.426337  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.427591 15729 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:53.429211 15544 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:53.429625  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.430899 15736 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:53.432449 15736 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:28:53.434337  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.435160 15729 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:53.435546 15729 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:53.435889 15729 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:53.444769 15846 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:53.445869 15729 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.402194 15817 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:28:53.448320 15729 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:53.450395 15809 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:28:53.451411  6274 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:53.472810  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 247299us
W20260709 12:28:53.501658  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 50496us
I20260709 12:28:53.487546 15849 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.508584  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:53.502862  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.503543  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:53.504118  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 97944.8us
I20260709 12:28:53.509495  6301 raft_consensus.cc:2368] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:28:53.506359  6272 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.491089 15736 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:53.492559 15862 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:53.511456  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.524844  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:53.534444  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 278764us
I20260709 12:28:53.520812 15748 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:53.534993 15549 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.537837 15549 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.539242 15748 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:53.539744 15748 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:53.540279 15549 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:53.533974 15486 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:53.542965 15486 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:53.543361 15486 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:53.522183 15865 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:53.512430 15849 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.561841  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:53.562187  6452 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:28:53.565034  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54574.9us
I20260709 12:28:53.565462  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.565934  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:53.578081  6455 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:53.509454 15797 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.580063 15836 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:53.575299 15655 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.007s
I20260709 12:28:53.581734 15892 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:28:53.590757  6303 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:53.593114  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:53.597172 15898 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:53.534670 15809 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:28:53.593264  7331 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 20 LEARNER]: Advancing to term 21
I20260709 12:28:53.609762 15845 consensus_queue.cc:1048] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:53.617081 15617 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:28:53.619587 15880 consensus_queue.cc:1059] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:28:53.621373 15322 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.622118 15486 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.623162  6310 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:53.625006 15486 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:53.600924 15802 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.027s
W20260709 12:28:53.629355  6424 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.079s (SENT) [suppressed 72 similar messages]
I20260709 12:28:53.629602  7331 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:28:53.630407 15322 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:53.632576  6452 raft_consensus.cc:1240] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:53.634938  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:53.643263  6450 raft_consensus.cc:1275] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:28:53.647938  6309 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:28:53.642813  7287 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.812s (SENT)
I20260709 12:28:53.653075  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:53.658382  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48987.8us
I20260709 12:28:53.659754  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:53.665998  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 58143.1us
I20260709 12:28:53.670734  7332 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:53.673075  7317 raft_consensus.cc:1217] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Deduplicated request from leader. Original: 21.16->[22.17-22.17]   Dedup: 22.17->[]
I20260709 12:28:53.651762 15876 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.679728  7317 raft_consensus.cc:1240] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:28:53.671720 15882 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:53.679344 15655 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.671011  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:53.680917  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 153679us
I20260709 12:28:53.694715  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:53.695928  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 167765us
I20260709 12:28:53.698390  6351 leader_election.cc:400] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:28:53.698899  6351 leader_election.cc:403] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:53.683774  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:53.708335  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79060.3us
I20260709 12:28:53.711645  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:53.713084  7331 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEARNER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:53.715319  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:53.696285 15613 raft_consensus.cc:1064] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:28:53.665359 15862 consensus_queue.cc:1059] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 33.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.046s
I20260709 12:28:53.679780 15849 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:53.752580 15897 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:53.681633 15655 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:53.754889  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 60249.3us
W20260709 12:28:53.761798  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 118889us
I20260709 12:28:53.693115 15898 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:53.661326 15322 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:28:53.766903  6451 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 23. Current term is 24. Ops: []
W20260709 12:28:53.768985  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85333.7us
I20260709 12:28:53.682541 15836 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:28:53.699581 15865 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:53.757026 15736 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:53.778900 15888 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:53.781540  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:53.754449 15832 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.002s	sys 0.016s
I20260709 12:28:53.706856 15673 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:28:53.778097 15884 consensus_queue.cc:1048] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.070s
W20260709 12:28:53.792912  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.792738  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.808674  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.681807 15854 consensus_queue.cc:1059] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.110s
I20260709 12:28:53.697048 15876 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.756592 15491 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
W20260709 12:28:53.810166  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 94941.4us
W20260709 12:28:53.802323 15832 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.003s	sys 0.016s
I20260709 12:28:53.797435 15897 raft_consensus.cc:3060] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:28:53.794116 15736 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:53.783383 15849 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:28:53.796892 15888 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:53.791599 15817 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:28:53.796139 15865 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.814934 15898 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:53.880654 15817 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:53.881126 15817 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:53.858881 15499 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:53.847757 15880 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:53.882797 15499 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:53.863730 15900 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:53.884117 15499 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.864594 15896 consensus_queue.cc:1059] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 17, Time since last communication: 0.021s
I20260709 12:28:53.885368 15896 consensus_queue.cc:1243] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:28:53.885417 15499 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:52.831800  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:53.866364 15883 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.837853 15748 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.888423 15883 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:53.859768  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78208.5us
I20260709 12:28:53.890004 15809 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:28:53.820029 15878 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 17, Time since last communication: 0.072s
I20260709 12:28:53.890049  6274 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:53.893538 15486 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:53.894251 15486 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:53.860133 15807 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:53.895233 15872 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:53.870293 15736 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:53.895723 15872 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:53.895429 15486 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.896967 15872 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.898187 15872 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:53.842440 15794 raft_consensus.cc:3055] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:28:53.836901 15655 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:53.901324 15807 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:28:53.826529 15797 raft_consensus.cc:3055] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:53.858180  7331 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:53.826128  7319 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:28:53.827414 15832 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:53.824971 15876 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:53.846386  6202 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 18 to 21, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 21 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:53.825627 15861 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:53.958053 15861 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.949267 15797 raft_consensus.cc:740] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:53.959625 15797 consensus_queue.cc:260] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.960498 15861 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:53.907145  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:53.960949 15797 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:53.911135 15748 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:53.912446  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:53.962292  7332 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:53.921056  6451 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:28:53.922560  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.884171 15888 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:53.953908  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.926676 15794 raft_consensus.cc:740] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:53.973415 15911 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:53.973533  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66495.9us
I20260709 12:28:53.973903 15832 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:53.977716 15865 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:53.983703  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:53.995162  6377 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:53.949018 15835 raft_consensus.cc:3055] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:53.882012 15817 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:53.968719  6385 raft_consensus.cc:2376] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 36. Re-sending same reply.
I20260709 12:28:53.992563  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:53.991801  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 312792us
W20260709 12:28:54.006130  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 243130us
I20260709 12:28:54.012380 15888 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:54.006114 15883 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.015805 15883 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.016573 15817 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:54.017802 15883 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:54.003332 15835 raft_consensus.cc:740] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:54.020857  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.021175 15897 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:28:53.995026 15900 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.022523 15748 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:54.023146 15748 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:54.025465  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:53.996270  7332 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 32.
W20260709 12:28:54.000885  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.026655  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:54.027460  6451 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 31.
I20260709 12:28:53.910056 15613 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 10, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.029361  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:54.027542  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:54.029296  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:54.036046  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:54.036283  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:54.037824  6307 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:28:53.955629 15499 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.051s
I20260709 12:28:54.041100  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:54.046900  6385 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:54.053782  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:54.054445  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:54.064165  6377 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:54.020051 15835 consensus_queue.cc:260] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.065403  7332 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEARNER]: Advancing to term 26
I20260709 12:28:54.080910  6310 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:54.033706 15802 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:54.022908 15900 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.035521 15865 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:54.084698  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 55507.7us
I20260709 12:28:54.087534  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:54.088635  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.090328  6300 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:54.039842 15882 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.045s
I20260709 12:28:54.123358  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:54.013929 15794 consensus_queue.cc:260] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.125406  6377 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 27: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:54.108660  7332 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:28:54.009964 15832 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.099535 15910 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:28:54.065737 15891 consensus_queue.cc:1175] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "561a4e21d8b344a79af44568ca80a3c5" responder_term: 21 status { last_received { term: 20 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:28:54.138264 15907 log.cc:927] Time spent T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:28:54.139833  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 147378us
I20260709 12:28:54.146135  6202 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 22 to 27. New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.100376 15835 raft_consensus.cc:3060] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:28:54.159583  6197 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 274) took 77472 ms (client timeout 15000 ms). Trace:
I20260709 12:28:54.163017  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:54.209762  6377 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:54.179021 15748 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:54.123322 15809 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:54.182624  6451 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:54.091537 15897 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:54.192206  7287 leader_election.cc:336] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.544s (SENT)
I20260709 12:28:54.193882 15910 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:54.115236 15883 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:54.094334 15802 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.219348  6301 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:54.120357 15736 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.232453  6202 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 24 to 31, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 31 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.193439  6383 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:54.100934 15865 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.209255 15549 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:54.255614  6377 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:54.234892 15794 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:54.257622  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:54.259260  7287 leader_election.cc:336] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.219s (SENT)
I20260709 12:28:54.263972  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.260003  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:54.260978  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.266407  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.256484  6202 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 16 to 24. New cstate: current_term: 24 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.262259  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:54.274236  6377 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:54.240625 15809 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:54.252926 15629 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.040s
I20260709 12:28:54.257987 15549 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.268676  6202 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 13 to 17, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 17 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.284372  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.285195  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:54.304028  6385 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 32.
I20260709 12:28:54.309468  6377 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
I20260709 12:28:54.330801  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:54.339645  6202 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 21 to 27. New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:28:54.341503  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57233.8us
I20260709 12:28:54.343492  6348 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.317032 15891 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:54.347694  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 88465.9us
W20260709 12:28:54.348423  7287 leader_election.cc:336] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.241s (SENT)
I20260709 12:28:54.356362  6307 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:28:54.286229 15865 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:54.297086 15900 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:54.296325 15802 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:54.297756 15549 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:54.332365 15895 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:28:54.368256  7287 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.647s (SENT)
W20260709 12:28:54.372435  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.414165 15895 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:54.423826  6202 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 23 to 26, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 26 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.345157 15862 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:54.385162  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:54.390506  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.288275 15910 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:54.429634  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.419821  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:54.423120  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.375896 15888 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:54.426642  6202 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 24 to 26, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 26 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.492946  6202 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 18 to 22, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 22 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.493585 15895 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:54.518407  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 175957us
I20260709 12:28:54.519448  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.292989 15809 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.533977  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:54.548628  6301 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:54.552488  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:54.515337 15727 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:54.468375 15549 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:28:54.383599 15914 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.153s
I20260709 12:28:54.575011  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:54.585599 15797 consensus_meta.cc:287] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.624s	user 0.005s	sys 0.008s
I20260709 12:28:54.587688  6348 leader_election.cc:400] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
I20260709 12:28:54.588271  6348 leader_election.cc:403] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:54.552141 15878 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:54.594954  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:54.595978 15921 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:28:54.598407  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:54.600725 15920 log.cc:927] Time spent T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
W20260709 12:28:54.601485 15817 consensus_meta.cc:287] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.000s
W20260709 12:28:54.602638 15917 log.cc:927] Time spent T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:28:54.605335  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:54.602962  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:54.602348 15775 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:28:54.601506  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:54.610810  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 242577us
I20260709 12:28:54.578138 15913 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:54.576058 15518 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.560395  6202 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 15 to 21. New cstate: current_term: 21 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.586948 15861 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:54.588203 15736 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:54.635387  6310 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:54.631227  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:54.623752 15913 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.646347  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:54.615010 15322 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:28:54.624783 15861 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:54.649411  6202 catalog_manager.cc:5697] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.656080  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.660423 15491 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:54.660971 15491 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:54.649121 15913 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:54.662941 15655 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:54.663358 15655 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:54.663728 15655 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:54.670821 15861 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:54.661875 15491 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:54.209462  6197 rpcz_store.cc:269] 0709 12:27:36.686884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.687205 (+   321us) service_pool.cc:224] Handling call
0709 12:28:54.159459 (+77472254us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:51.336670 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:51.337319 (+   649us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:51.337370 (+    51us) write_op.cc:435] Acquired schema lock
0709 12:28:51.337387 (+    17us) tablet.cc:662] Decoding operations
0709 12:28:51.337799 (+   412us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:51.337900 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:51.337924 (+    24us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:28:51.338436 (+   512us) tablet.cc:701] Row locks acquired
0709 12:28:51.348299 (+  9863us) write_op.cc:260] PREPARE: finished
0709 12:28:51.348499 (+   200us) write_op.cc:270] Start()
0709 12:28:51.348658 (+   159us) write_op.cc:276] Timestamp: P: 1783600131348428 usec, L: 0
0709 12:28:51.348698 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 12:28:51.349429 (+   731us) log.cc:844] Serialized 901 byte log entry
0709 12:28:52.079169 (+729740us) op_driver.cc:464] REPLICATION: finished
0709 12:28:52.151921 (+ 72752us) write_op.cc:301] APPLY: starting
0709 12:28:52.157984 (+  6063us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:52.204309 (+ 46325us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:52.204445 (+   136us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:52.205679 (+  1234us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:52.205993 (+   314us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:28:52.206063 (+    70us) write_op.cc:312] APPLY: finished
0709 12:28:52.206444 (+   381us) log.cc:844] Serialized 33 byte log entry
0709 12:28:52.243402 (+ 36958us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:52.246059 (+  2657us) write_op.cc:454] Released schema lock
0709 12:28:52.253397 (+  7338us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:52.571849 (+318452us) spinlock_profiling.cc:243] Waited 7.03 ms on lock 0x7b1800410b60. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa6cddef0 00007f6fa6cde489 00007f6fa6ce071a
0709 12:28:52.571973 (+   124us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:52.572010 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":520149,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":50272,"apply.run_cpu_time_us":10102,"apply.run_wall_time_us":219680,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":475088,"prepare.run_cpu_time_us":16833,"prepare.run_wall_time_us":197956,"raft.queue_time_us":182039,"raft.run_cpu_time_us":28989,"raft.run_wall_time_us":507424,"replication_time_us":730285,"spinlock_wait_cycles":16175872,"thread_start_us":649826,"threads_started":4,"wal-append.queue_time_us":174860,"wal-append.run_cpu_time_us":66602,"wal-append.run_wall_time_us":1704728}]]}
I20260709 12:28:54.670377  6202 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 15 to 24, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 24 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.631619 15518 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.643368 15849 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:28:54.624725 15817 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.025s
I20260709 12:28:54.677139 15807 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:54.687788  6202 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 15 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 22 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.684928 15849 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:54.707962 15807 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:54.753654 15807 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:54.748165  6202 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:54.764642  6202 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 18 to 20, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.778105  6202 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 19 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 22 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:54.798619  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 172800us
I20260709 12:28:54.811466  6202 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 19 to 20, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 20 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:54.817382 15518 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:54.844595  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 188600us
W20260709 12:28:54.830902  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.845439  6274 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:54.845862  7287 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.987s (SENT)
I20260709 12:28:54.846498  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:54.857913 15794 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:54.760671 15849 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.823400 15886 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:28:54.853015 15322 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:28:54.888664 15836 raft_consensus.cc:1064] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:28:54.892570 15836 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 11, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.900171 15673 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:54.900731 15673 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:54.901628 15673 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:54.906459  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 318956us
I20260709 12:28:54.907231  6348 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:54.907976 15935 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.125s	user 0.002s	sys 0.000s
I20260709 12:28:54.908425  6348 leader_election.cc:400] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:54.908737  6348 leader_election.cc:403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:54.909214 15891 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
W20260709 12:28:54.909665 15835 consensus_meta.cc:287] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.000s
I20260709 12:28:54.910035 15891 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:54.912813 15942 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:54.917462 15900 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:54.917397 15861 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.918227 15900 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:54.854771 15937 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:54.919663 15872 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.920182 15872 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:54.920657 15900 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:54.921911 15932 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 12:28:54.924983  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.925203  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.926025 15911 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:28:54.926667 15888 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:54.927909  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.928957 15673 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:54.922233 15872 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:54.931807 15891 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:54.933353  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:54.933161  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:54.935370  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 277068us
W20260709 12:28:54.935720  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:54.939209  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70721.4us
I20260709 12:28:54.943786  6348 leader_election.cc:400] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25.
I20260709 12:28:54.944324  6348 leader_election.cc:403] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:54.961227  6348 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:54.910225 15835 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.965243  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:54.966252  6310 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:54.966580  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:54.978515  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:54.984341  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:54.985620  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.906075 15913 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:54.993109  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47012.1us
I20260709 12:28:54.934582 15883 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:55.000085  6351 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:28:54.947544 15872 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:28:55.002391  6424 leader_election.cc:336] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.842s (SENT)
I20260709 12:28:55.005486  6424 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:54.950416 15891 raft_consensus.cc:3060] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:55.009281 15832 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:55.009723 15832 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.010042 15832 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:55.010644 15888 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:55.015478  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.951122 15937 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:54.987825 15878 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:54.991017 15897 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:28:55.028733  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 200669us
W20260709 12:28:55.028983  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 44787us
W20260709 12:28:55.031836  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.036604  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.046316  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.046994  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.047585  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.047978  6383 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:55.050376  6376 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 31. Current term is 32. Ops: [30.20-31.21]
I20260709 12:28:55.051774  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.007688 15849 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:28:55.056272  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.058254  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.059813  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.060833  6383 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:28:55.065634  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 118641us
W20260709 12:28:55.070873  7287 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.249s (SENT)
I20260709 12:28:55.072654 15727 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:55.073601 15727 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:55.074002 15727 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
W20260709 12:28:55.076304  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 128963us
I20260709 12:28:55.078667  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.079329  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:55.080226  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.082563  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.083662  6378 raft_consensus.cc:2410] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:28:55.084609  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:55.084725  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 90974.1us
I20260709 12:28:55.019510 15947 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:55.093433  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.103314  6382 raft_consensus.cc:2368] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 27. Current term is 28.
I20260709 12:28:55.093441  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:55.098659  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:54.991859 15835 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.130942  6305 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:55.130947  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:55.306756  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 198622us
I20260709 12:28:55.076789 15549 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.138849  7316 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:28:55.331389  6378 raft_consensus.cc:1275] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 12:28:55.331907  6378 pending_rounds.cc:85] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 16
I20260709 12:28:55.332183  6378 pending_rounds.cc:107] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:28:55.332471  6378 raft_consensus.cc:2889] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:55.336367  7329 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:28:55.135311 15499 consensus_queue.cc:1059] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.176s
I20260709 12:28:55.351748  6381 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:55.357102  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.324725 15613 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.361567  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:55.353334 15939 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.198s	user 0.000s	sys 0.005s
W20260709 12:28:55.363353 15807 consensus_meta.cc:287] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.003s
I20260709 12:28:55.353066 15886 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.345665 15322 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:28:55.141569 15727 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.462270  6381 raft_consensus.cc:2468] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
W20260709 12:28:55.479343  6350 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.383387 15910 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:55.489612  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.340274 15549 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.446692 15911 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.497550  6379 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:55.334007 15937 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:55.509106  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 148635us
I20260709 12:28:55.455802 15872 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.472337 15897 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.383188 15613 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:55.517509  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.492055 15876 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:55.530758  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 425954us
I20260709 12:28:55.530090  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.522034 15499 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.527541  6456 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:55.454559 15807 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.509235 15836 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:55.517243 15727 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:55.492141 15322 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.495990 15910 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.522969 15897 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.561712 15897 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.468443 15886 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.524576 15913 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.554598  6458 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 20. Current term is 21. Ops: []
I20260709 12:28:55.554716  6379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
W20260709 12:28:55.604411 15967 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:28:55.607362 15613 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.558378 15499 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:55.559000 15959 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
W20260709 12:28:55.534757 15958 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
I20260709 12:28:55.510339 15911 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.570974  6453 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28. Ops: []
W20260709 12:28:55.556013 15956 log.cc:927] Time spent T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 12:28:55.535490 15876 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:55.589092  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.597535  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.598407  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.602890  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.528843 15872 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.616353  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:55.632512  6459 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:55.636560  6454 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:55.639624  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.550525 15836 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.640376  6453 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:55.642549  6458 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:28:55.649950  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.624845 15655 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.961s	user 0.018s	sys 0.000s
W20260709 12:28:55.652343  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:55.655431  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.666443 15817 consensus_queue.cc:1048] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:55.669189  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:55.669535 15794 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.659565 15973 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.685405  6459 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 32.
I20260709 12:28:55.659729 15655 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:55.706797 15965 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:28:55.685089 15971 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.690600 15835 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.717603  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:55.712896  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 153857us
I20260709 12:28:55.718461  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.719154  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.723917  6381 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:28:55.718303  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.730211  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.732659  6381 raft_consensus.cc:2410] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:28:55.733390  6309 raft_consensus.cc:1240] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:55.733448  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.734992  6378 raft_consensus.cc:2410] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:28:55.734905  6385 raft_consensus.cc:1217] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Deduplicated request from leader. Original: 19.11->[21.12-25.14]   Dedup: 21.12->[23.13-25.14]
I20260709 12:28:55.735805  6382 raft_consensus.cc:2393] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:55.736240  6385 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [21.12-25.14]
I20260709 12:28:55.737038  6383 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:28:55.717721 15957 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:28:55.693231 15794 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.737926  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:55.714825  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 68738.7us
W20260709 12:28:55.725541 15673 consensus_meta.cc:287] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.000s
I20260709 12:28:55.721671 15613 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.751536  6382 raft_consensus.cc:1275] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:55.706597 15973 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.755575 15973 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:55.753712 15968 log.cc:927] Time spent T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:28:55.749221  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:55.759358  6383 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:55.737478 15971 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.764578 15836 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.749063 15966 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:28:55.765161 15910 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.766156 15911 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.767197 15876 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.747677 15549 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:55.768659  6310 raft_consensus.cc:1240] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 31. Current term is 32. Ops: [31.23-31.23]
I20260709 12:28:55.748070 15807 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.777377 15974 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:55.777920 15974 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.778242 15974 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:55.787918 15549 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.788724 15549 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.791565 15549 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:55.796413 15549 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:28:55.798278 15891 consensus_meta.cc:287] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.003s
I20260709 12:28:55.799036 15891 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25.
W20260709 12:28:55.805307  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 358984us
I20260709 12:28:55.808969  6376 raft_consensus.cc:2468] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:55.809691 15995 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.810441 15963 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.810427 15995 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.811349 15963 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.813035 15995 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.813449  6307 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:55.813599 15963 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.813915 15729 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:55.815403 15888 consensus_meta.cc:287] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.795s	user 0.006s	sys 0.007s
I20260709 12:28:55.815336 15729 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.815923 15888 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:55.817476 15729 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:55.820806 15849 consensus_meta.cc:287] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.728s	user 0.009s	sys 0.000s
I20260709 12:28:55.822732 15886 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:55.823498  6379 raft_consensus.cc:606] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:55.822942  6380 raft_consensus.cc:2410] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:28:55.823944  6379 raft_consensus.cc:614] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:28:55.806854  6424 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 3.102s (SENT)
I20260709 12:28:55.825559  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:55.826068  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1677) took 1137 ms. Trace:
W20260709 12:28:55.827005  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 157887us
I20260709 12:28:55.826458  7332 rpcz_store.cc:276] 0709 12:28:54.689086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.689256 (+   170us) service_pool.cc:224] Handling call
0709 12:28:54.689724 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:54.693005 (+  3281us) raft_consensus.cc:1534] Early marking committed up to index 23
0709 12:28:54.940378 (+247373us) log.cc:844] Serialized 14 byte log entry
0709 12:28:55.090987 (+150609us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:55.098256 (+  7269us) log.cc:844] Serialized 26 byte log entry
0709 12:28:55.123168 (+ 24912us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:28:55.123231 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.123338 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:55.825875 (+702537us) raft_consensus.cc:1685] finished
0709 12:28:55.825893 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.826007 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14555,"prepare.queue_time_us":189253,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":15405,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":31604,"thread_start_us":273578,"threads_started":2,"wal-append.queue_time_us":380458}
I20260709 12:28:55.756971 15613 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:55.837385  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 163089us
I20260709 12:28:55.847167  6459 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:28:55.850149  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.855427  6300 raft_consensus.cc:1240] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:55.823398 15499 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:55.832804 15963 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.832762 15891 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.859426 15963 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.860334 15891 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.861629 15613 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.863200 15999 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.830446 15872 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.823913 15794 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:55.870772  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63961.8us
I20260709 12:28:55.829501 15655 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.863863 15999 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:55.882998 15999 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.883955 15861 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.854713  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:55.872505  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:55.872936  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:55.875478  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.833310 15807 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:55.878835 15971 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:55.879345 15832 consensus_meta.cc:287] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.869s	user 0.013s	sys 0.000s
I20260709 12:28:55.862264 15891 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.888969  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.887704  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:55.872267  6457 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:55.832978 15549 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:55.897017  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.897444  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.899315  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:55.900736 15807 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:55.895975 15872 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:55.902282  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:55.868376 15966 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.904856  6455 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:55.907595  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:55.908352  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.911957  6457 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 34.
I20260709 12:28:55.911967  6302 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:55.912487  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.909802  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:55.913968 15499 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:55.916280 15974 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.919251 15966 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:55.921576 15966 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:55.922338  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:55.906759 15832 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:55.925019  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.927320  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:55.929941  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:55.932206 15966 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:55.932642 15966 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.933010 15966 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:28:55.935824 15981 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.226s	user 0.000s	sys 0.003s
I20260709 12:28:55.930701  6452 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:28:55.939592  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:55.941896 15727 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:55.942322 15727 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:55.926476 15974 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.942786 15727 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:55.941054 15797 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:55.943572 15797 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.943930 15797 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:55.908265 15886 consensus_queue.cc:1048] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:28:55.923612 15613 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.024s
I20260709 12:28:55.908619 15891 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:55.942867  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:55.951556  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:55.951390  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 43872.1us
I20260709 12:28:55.961167  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:55.942902 15952 log.cc:927] Time spent T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.889s	user 0.003s	sys 0.004s
I20260709 12:28:55.969837  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:55.972764 15963 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:55.978919  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.980049  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:55.981014  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.982016  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:55.987304  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:55.992871  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:55.998329  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:55.999855  6450 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:56.001780  6452 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31. Ops: [30.20-30.20]
I20260709 12:28:56.003643 15886 consensus_queue.cc:1048] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:28:56.007897 15832 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.009617 15807 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.015188  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:56.016846  6452 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:56.028527 15985 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:56.029130 15985 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.007419  6424 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:56.031459 15985 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.031580 16001 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:56.032713 15775 raft_consensus.cc:3055] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:56.033331 15775 raft_consensus.cc:740] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:56.034461 15775 consensus_queue.cc:260] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.035868 15775 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:56.041945  6378 raft_consensus.cc:2468] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:28:56.038105 15797 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.051030 15727 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:56.052654 15954 log.cc:927] Time spent T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.929s	user 0.009s	sys 0.011s
I20260709 12:28:56.053612 15727 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.055284  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.056258  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:56.058084  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.058969  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.059584  6385 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31. Re-sending same reply.
I20260709 12:28:56.060009  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.060329  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.060509  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.060987  6378 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:56.061983  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.046200 15797 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.066416  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.077386  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:56.082043  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.067328  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.069905  6452 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:28:56.083482  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.084625  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:56.088400  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.089655  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.090631  6378 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:56.091679  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.092322  6378 raft_consensus.cc:2376] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31. Re-sending same reply.
I20260709 12:28:56.093294  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.097218  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:56.097827  6454 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:56.089056  6452 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:56.102550  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.107611  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.096921 15949 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:56.072187 15966 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.113029 16001 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:56.138062  7287 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 1.843s (SENT)
I20260709 12:28:56.146884  6454 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:56.149668  7285 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:56.152650  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 4106) took 1042 ms. Trace:
I20260709 12:28:56.122831 15862 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:56.125571 16003 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.150355 15996 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:56.155134 15794 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.158980 15998 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:56.115834 15865 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:28:56.132788 15727 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:56.133160 15949 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:56.132395 15797 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.118846 15974 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:28:56.137805 15629 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.181430  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3911) took 2204 ms. Trace:
I20260709 12:28:56.152269 15985 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.208233  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.178093  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.211107  6300 raft_consensus.cc:1240] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:56.178692  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 4109) took 1066 ms. Trace:
I20260709 12:28:56.225485  6308 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:56.218360  6377 rpcz_store.cc:276] 0709 12:28:53.977225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.326040 (+348815us) service_pool.cc:224] Handling call
0709 12:28:54.326529 (+   489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:54.327057 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:28:54.390627 (+ 63570us) spinlock_profiling.cc:243] Waited 6.5 ms on lock 0x7b48001f0158. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:54.398253 (+  7626us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b1800402e80. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa0134c0a 00007f6fa9874219 00007f6fa01bca40 00007f6fa01b74e4 00007f6fa01b7455 00007f6fa6c9f3c7 00007f6fa32b19a7 00007f6fa32c10bf 00007f6fa32c1038
0709 12:28:54.729103 (+330850us) log.cc:844] Serialized 14 byte log entry
0709 12:28:54.914045 (+184942us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:54.914637 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:28:54.914777 (+   140us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:54.914980 (+   203us) log.cc:844] Serialized 14 byte log entry
0709 12:28:54.915177 (+   197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:54.915248 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:56.051889 (+1136641us) spinlock_profiling.cc:243] Waited 5.92 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:56.181171 (+129282us) raft_consensus.cc:1685] finished
0709 12:28:56.181196 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.181342 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60574,"prepare.queue_time_us":73253,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":61,"raft.queue_time_us":59395,"raft.run_cpu_time_us":3017,"raft.run_wall_time_us":50430,"spinlock_wait_cycles":40184704,"thread_start_us":434425,"threads_started":2,"wal-append.queue_time_us":253797}
I20260709 12:28:56.212322  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.246989 15949 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.182826  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.248507  6385 raft_consensus.cc:2376] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26. Re-sending same reply.
I20260709 12:28:56.249488  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.250260  6385 raft_consensus.cc:2376] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28. Re-sending same reply.
I20260709 12:28:56.209550  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.252530  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.231304 15797 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.253525  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.254669  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.153039  6451 rpcz_store.cc:276] 0709 12:28:55.111414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.133838 (+ 22424us) service_pool.cc:224] Handling call
0709 12:28:55.976557 (+842719us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b5800003c98. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa0182a22 00007f6fa909b8d9 00007f6fa81f5ee0 00007f6fa82bf0ab 00007f6fa82bf131 00007f6fa8280fc0 00007f6fa826b929 00007f6fa826b3d2 00007f6fa17e2bd4
0709 12:28:55.976746 (+   189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.976950 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:28:55.977225 (+   275us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.152321 (+175096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.152370 (+    49us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:56.152410 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.152472 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.152574 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16343,"prepare.queue_time_us":5515,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":16499,"spinlock_wait_cycles":15412096,"thread_start_us":158305,"threads_started":1}
I20260709 12:28:56.255353 15797 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.255796  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.257107  6454 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:28:56.257381  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.181010  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.211647  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.258149  6377 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:56.229406  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.225852  6456 rpcz_store.cc:276] 0709 12:28:55.112501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.112634 (+   133us) service_pool.cc:224] Handling call
0709 12:28:55.112856 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.532489 (+419633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.178397 (+645908us) spinlock_profiling.cc:243] Waited 412 ms on lock 0x7b54003c6254. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:56.178607 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948281216}
I20260709 12:28:56.210085  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.260834  6303 raft_consensus.cc:1240] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:56.225476  6379 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:56.261835  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:56.151382 16001 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.174685 15322 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:56.236194  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.160367 15971 raft_consensus.cc:2804] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:56.178404 16003 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.231197 15794 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.161617 15977 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.078s
I20260709 12:28:56.165551 15862 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:56.243592  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:56.241791  6424 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.079s (SENT)
W20260709 12:28:56.325577  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 83823.4us
I20260709 12:28:56.328277 16003 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:56.172427 15966 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.247450  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:56.223340 15629 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.329219 15971 raft_consensus.cc:697] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:56.235126 15998 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.331204 15629 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:56.331358 15998 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:56.263339 16016 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.077s	user 0.000s	sys 0.002s
I20260709 12:28:56.332592 15971 consensus_queue.cc:237] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.263226  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.334968  6380 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:28:56.174526  6378 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:56.192178 15996 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.268694  6377 raft_consensus.cc:606] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:56.235849 15985 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:56.275794 16014 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.103s	user 0.001s	sys 0.001s
W20260709 12:28:56.344878  6376 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1397) took 1286 ms (client timeout 1544 ms). Trace:
I20260709 12:28:56.274278 15882 consensus_queue.cc:1059] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 30.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:28:56.276767 15976 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:28:56.353919  6378 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:28:56.275080 15972 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:28:56.269904  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.184029  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:28:56.356173  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 206570us
W20260709 12:28:56.356537  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 172591us
I20260709 12:28:56.358691 15775 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:56.358759  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:56.359560 15775 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.341581  6377 raft_consensus.cc:614] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:56.364319  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.365315  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:56.343935  6380 raft_consensus.cc:3055] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:56.231451 15727 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.346235  6384 raft_consensus.cc:1275] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:28:56.368011  6380 raft_consensus.cc:740] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:56.345414  6376 rpcz_store.cc:269] 0709 12:28:55.051450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.051573 (+   123us) service_pool.cc:224] Handling call
0709 12:28:56.337222 (+1285649us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b5400037854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:56.337337 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1719219584}
I20260709 12:28:56.359931  6452 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:56.368865  6380 consensus_queue.cc:260] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.360021 15775 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:56.380671  6452 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 35.
W20260709 12:28:56.383272  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 113705us
I20260709 12:28:56.384306  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.385210  7332 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:28:56.388381  6380 raft_consensus.cc:3060] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:28:56.382980  6424 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.095s (SENT)
I20260709 12:28:56.390899  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.391366  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.392522  7332 raft_consensus.cc:2410] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:28:56.391038  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
W20260709 12:28:56.393817  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 236267us
I20260709 12:28:56.394498  6376 raft_consensus.cc:606] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:56.395001  6376 raft_consensus.cc:614] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:56.396312  6452 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:56.396382  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.398380  7329 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:28:56.410887  7331 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:56.411264  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.414808  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:56.419490  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.421252  6452 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:56.424175  6378 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:28:56.426069  7319 raft_consensus.cc:2393] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:28:56.427225  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.403306 15882 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:56.431349  6376 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:28:56.435536  6376 raft_consensus.cc:1275] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:28:56.436764  6376 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:28:56.443037  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:56.443167  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:56.444396  6376 raft_consensus.cc:606] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:28:56.445060  6376 raft_consensus.cc:614] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:28:56.449577  6376 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f53146508e564f8aae279a61e6696e80"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:28:56.451879 15499 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:56.451936  6376 raft_consensus.cc:606] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
W20260709 12:28:56.453886 15951 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 1.358s	user 0.003s	sys 0.006s
I20260709 12:28:56.454746  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:56.406889 15966 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.450150  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3596) took 1323 ms. Trace:
I20260709 12:28:56.466684  6301 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3594) took 1340 ms. Trace:
I20260709 12:28:56.469820  6309 rpcz_store.cc:276] 0709 12:28:55.127241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.129352 (+  2111us) service_pool.cc:224] Handling call
0709 12:28:55.129616 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.738186 (+608570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.449826 (+711640us) spinlock_profiling.cc:243] Waited 603 ms on lock 0x7b5400020dd4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:56.450069 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388041984}
I20260709 12:28:56.467716  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:56.470255  6301 rpcz_store.cc:276] 0709 12:28:55.126064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.128737 (+  2673us) service_pool.cc:224] Handling call
0709 12:28:55.129050 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:55.340621 (+211571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.340669 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:55.341242 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:28:56.013137 (+671895us) spinlock_profiling.cc:243] Waited 7.05 ms on lock 0x7b18002155c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:56.139712 (+126575us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:56.139741 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.152517 (+ 12776us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:56.274706 (+122189us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:56.466358 (+191652us) raft_consensus.cc:1685] finished
0709 12:28:56.466392 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.466599 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":205025,"raft.queue_time_us":232225,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":80000,"spinlock_wait_cycles":89855488,"thread_start_us":126426,"threads_started":1,"wal-append.queue_time_us":798241}
W20260709 12:28:56.559757  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.566087  6378 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:28:56.567061  6378 pending_rounds.cc:85] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 11
I20260709 12:28:56.569211  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:56.569897  7329 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:56.570024  6424 leader_election.cc:336] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Timed out: RequestConsensusVote RPC to 127.5.4.66:34641 timed out after 2.147s (SENT)
W20260709 12:28:56.569123 15988 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.141s	user 0.000s	sys 0.009s
I20260709 12:28:56.576371 15862 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.577343 15797 raft_consensus.cc:3055] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:56.577888 15797 raft_consensus.cc:740] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:56.581737  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:56.582881  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.588414  7329 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:28:56.578974 15797 consensus_queue.cc:260] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.452615 15499 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:56.600944 15955 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.173s
W20260709 12:28:56.601980  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 184965us
I20260709 12:28:56.567519  6378 pending_rounds.cc:107] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 20.12
I20260709 12:28:56.460667 15491 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.142s
I20260709 12:28:56.605206  6380 raft_consensus.cc:1275] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:56.638473  6378 raft_consensus.cc:2917] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Aborting config change with OpId 20.12: Aborted: Op aborted by new leader
I20260709 12:28:56.640224  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.640919  6381 raft_consensus.cc:2368] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28.
W20260709 12:28:56.620530  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 565289us
I20260709 12:28:56.617020  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1399) took 1564 ms. Trace:
I20260709 12:28:56.640769  7285 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:56.699105  6378 raft_consensus.cc:2889] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:56.689004  6384 rpcz_store.cc:276] 0709 12:28:55.052141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.096174 (+ 44033us) service_pool.cc:224] Handling call
0709 12:28:56.337545 (+1241371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.368264 (+ 30719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.616739 (+248475us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b5400037850. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:56.616938 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2854262656}
I20260709 12:28:56.701820  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.699628  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.619112 15549 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:56.703809  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.742094  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.743333  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.744302  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:56.745302  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.703917 15997 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.746656 15911 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.746785 15997 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.747447 15911 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:56.705129 16001 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:56.706707  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.749815  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.709570 15974 raft_consensus.cc:1064] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:28:56.750545  6381 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 21.
I20260709 12:28:56.751758  6381 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:28:56.752563 15974 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.711580 15727 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.749940 15911 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.750574  6384 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:28:56.756461 16013 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.423s	user 0.003s	sys 0.000s
I20260709 12:28:56.746224  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.703836 15990 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:56.756166 15549 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:56.710700 16025 log.cc:927] Time spent T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 12:28:56.759542  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.755085 15985 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.535243 15966 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:56.769908  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 129201us
I20260709 12:28:56.770955  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:56.698009 15878 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:56.695446 15910 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.698109 15897 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:56.775202  6378 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:56.779385  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:56.754760 15794 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:56.737303 15883 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.763013 15862 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.736256 15970 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.796468  7332 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:28:56.778111 15613 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.793907 15891 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:56.789837 15910 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.780413 15876 raft_consensus.cc:2804] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:28:56.798044 15883 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.869338 15989 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.870003 15910 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:28:56.870726 16027 log.cc:927] Time spent T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:28:56.871088 15883 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:56.783941 15897 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.883400  6351 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:56.905083  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:56.884582 15996 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:56.910118  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.855918 15853 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:56.794697 15911 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.071s
I20260709 12:28:56.800027 15862 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.872844 15876 raft_consensus.cc:697] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:56.803452 15970 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.812925 15613 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:56.960086  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.953173  6274 leader_election.cc:400] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28.
I20260709 12:28:56.961128  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.961395  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.856673 15891 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.952227  6307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3609) took 1120 ms. Trace:
I20260709 12:28:56.960827  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:56.961745  6274 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.923832 15878 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:56.979339  6274 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:56.982286  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 286954us
I20260709 12:28:56.983417  6348 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:56.809921 15499 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.929840 15883 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:56.989295  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.989589  7319 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:56.927260 15966 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:56.931006 15911 consensus_queue.cc:1243] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:56.869974 15989 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:56.997401  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:56.967154  6307 rpcz_store.cc:276] 0709 12:28:55.831511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.831617 (+   106us) service_pool.cc:224] Handling call
0709 12:28:55.831941 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.938923 (+106982us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:56.062510 (+123587us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.136735 (+ 74225us) log.cc:844] Serialized 14 byte log entry
0709 12:28:56.166083 (+ 29348us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.166209 (+   126us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:56.166317 (+   108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.237275 (+ 70958us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b18004ecbe0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:56.262779 (+ 25504us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:56.789656 (+526877us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b18004ecbe0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419
0709 12:28:56.952011 (+162355us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001b21d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:56.952052 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.952163 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26725,"prepare.queue_time_us":191733,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":2137,"raft.queue_time_us":32415,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"spinlock_wait_cycles":457019392,"thread_start_us":181871,"threads_started":2,"wal-append.queue_time_us":123038}
I20260709 12:28:56.999404  6274 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.977150 15832 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.002537  7316 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:56.996134  7331 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 34.
I20260709 12:28:57.008239  6274 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.964852 15835 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.037s
I20260709 12:28:56.999531 15883 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:57.010182 15832 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.008960 15910 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.009s
I20260709 12:28:57.015684  6274 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.016372 15729 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:57.016613 16011 log.cc:927] Time spent T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.822s	user 0.006s	sys 0.007s
I20260709 12:28:57.031919 15613 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.056984  6274 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.035565 15499 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:56.930666 15853 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.062291  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:56.995129 15491 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:57.001770 15895 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:57.036844 15655 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:56.994923 15913 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:57.033845 16026 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:28:57.012224 15549 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:56.999280 15888 consensus_queue.cc:1059] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:28:57.040234 15883 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:57.065956  7332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:57.055330 15978 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:28:57.066885  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:57.064193  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.066184  6348 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.093217 15888 raft_consensus.cc:3055] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:28:57.067400 15996 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:57.096688  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.068914 15963 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:57.103189  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.100075  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.105408  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.062829 16032 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:57.094785 15832 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.112877  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.112006  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.117184  6309 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:57.097846 15888 raft_consensus.cc:740] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:57.102762 15913 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:28:57.129601 15853 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.115427 15491 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:28:57.055967 15876 consensus_queue.cc:237] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.103323 16033 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:57.144462  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.111692 15849 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:57.147871 15964 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:57.102478 15655 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:28:57.155501  7332 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31. Ops: [30.20-30.20]
I20260709 12:28:57.145174 15978 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:57.145732 15913 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.146291 15613 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:57.161650 15897 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.116832 16032 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.172340  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.181483 15835 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:57.182581 15832 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:57.183183 15832 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:57.184334 15832 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.188710  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:57.190006  6425 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:57.190788 15891 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.191603 15549 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:57.156702 15849 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:57.192924 15849 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:57.193063  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3612) took 1375 ms. Trace:
I20260709 12:28:57.193573  6379 rpcz_store.cc:276] 0709 12:28:55.817251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.265162 (+447911us) service_pool.cc:224] Handling call
0709 12:28:56.336382 (+ 71220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:56.758951 (+422569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.758995 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:56.765381 (+  6386us) log.cc:844] Serialized 26 byte log entry
0709 12:28:57.054280 (+288899us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:57.093858 (+ 39578us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.094097 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.094177 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.192789 (+ 98612us) raft_consensus.cc:1685] finished
0709 12:28:57.192813 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.192994 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":465448,"prepare.queue_time_us":39768,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":429771,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":10056,"thread_start_us":251005,"threads_started":1,"wal-append.queue_time_us":330735}
I20260709 12:28:57.196117  6379 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 12:28:57.196762 15775 consensus_meta.cc:287] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.786s	user 0.009s	sys 0.001s
I20260709 12:28:57.197256 15775 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.198642 15862 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:57.200359  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3645) took 1162 ms. Trace:
I20260709 12:28:57.201246 16032 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.170063 15913 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:57.161335 15655 consensus_queue.cc:1243] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:57.202443 15832 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:57.158371 15964 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.203955 15964 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:57.165114 15888 consensus_queue.cc:260] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.206861 15888 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:28:57.209434  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.210107  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.212432 15986 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.213017 15986 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.215479 15986 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:57.203576  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.217403  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.218713  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.221700  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.222414  6452 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:57.222836  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.223403  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.224157  6457 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
W20260709 12:28:57.224635 16019 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.442s	user 0.005s	sys 0.013s
I20260709 12:28:57.230238 15989 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:57.232669  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51236.1us
W20260709 12:28:57.233237  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.198663  6379 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:28:57.200942  6376 rpcz_store.cc:276] 0709 12:28:56.037714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.468826 (+431112us) service_pool.cc:224] Handling call
0709 12:28:56.469696 (+   870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:56.765628 (+295932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.765666 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:56.766105 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 12:28:57.052814 (+286709us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:57.094459 (+ 41645us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.094713 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.094795 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.200140 (+105345us) raft_consensus.cc:1685] finished
0709 12:28:57.200164 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.200293 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":359570,"prepare.queue_time_us":41477,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":303086,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":11168,"thread_start_us":245673,"threads_started":1,"wal-append.queue_time_us":341238}
I20260709 12:28:57.234041  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.253628  7285 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:57.258226  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 68285.2us
I20260709 12:28:57.265980  6379 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: []
W20260709 12:28:57.224176 16030 log.cc:927] Time spent T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:28:57.272771  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.269753  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.272143  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.272403  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:57.272938  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.273655  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.255347  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:57.275174  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.278090  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.283043  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.294848  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.295929  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:57.301440  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46166.3us
I20260709 12:28:57.282001  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:57.302201  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 159350us
I20260709 12:28:57.265971 15897 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:57.305977  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:57.309887  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 209911us
I20260709 12:28:57.256832 15948 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.269797 16029 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:28:57.314306  6454 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:57.281103 15886 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:57.310972  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.318747  6379 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:57.319836  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:57.320188  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.321385  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.322834  6348 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:57.324702  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.322055  6379 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:57.326115  6379 raft_consensus.cc:1217] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Deduplicated request from leader. Original: 19.11->[21.12-26.15]   Dedup: 21.12->[23.13-26.15]
I20260709 12:28:57.326635  6379 pending_rounds.cc:85] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 12
W20260709 12:28:57.328644  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.329244  6348 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.319505 15832 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:57.312311 15549 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:57.333267 15832 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:57.333606 15775 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:57.342797 15979 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.343415 15979 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.326392 15878 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:28:57.352898  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.313151 15989 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:57.357385 15989 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.325707 15947 consensus_queue.cc:1059] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:28:57.359468 15947 raft_consensus.cc:3055] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:57.359582 15989 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.359854 15947 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:57.309803 15897 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:57.360499 15897 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:57.323902 15872 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:57.362703 15491 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.363229 15491 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.365150 15491 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.365588 15491 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.312368 15913 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.331658 15891 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.368974 15913 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.368777 15891 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.317906 15817 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:57.369679 15817 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.370028 15817 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:57.370784 15891 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.312534 15883 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.375420  6376 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:57.376287 15883 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.328292 15835 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:57.328439 16032 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.379122 15849 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.379715 16032 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.381174 15849 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.381567 16032 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.319031 15948 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.383494 15948 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.314853 16029 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.384127 16029 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:57.384506 16043 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:57.384891  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55723.6us
I20260709 12:28:57.385088 16043 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.387413 16044 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:57.388049 16044 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:57.388931 16044 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.389304 15947 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.389667  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.390991  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:57.392236  6379 pending_rounds.cc:107] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:28:57.394296  6379 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:57.394763  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:28:57.396656  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83005.8us
I20260709 12:28:57.397970  6385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3618) took 1579 ms. Trace:
I20260709 12:28:57.366010 15491 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.398721  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.405597  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.406142  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.406880  6452 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:57.407209  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3646) took 1368 ms. Trace:
I20260709 12:28:57.408735  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.410962  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.413345  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.415570  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.417173  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.333205 15549 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:57.419689  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.424393  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.407716  6383 rpcz_store.cc:276] 0709 12:28:56.039169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.587891 (+548722us) service_pool.cc:224] Handling call
0709 12:28:56.588351 (+   460us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:56.769905 (+181554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.769935 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:56.776486 (+  6551us) log.cc:844] Serialized 52 byte log entry
0709 12:28:57.101497 (+325011us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:57.113071 (+ 11574us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.113420 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.313057 (+199637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.314785 (+  1728us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.406937 (+ 92152us) raft_consensus.cc:1685] finished
0709 12:28:57.406965 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.407144 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":181973,"prepare.queue_time_us":53957,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":16283,"raft.queue_time_us":201047,"raft.run_cpu_time_us":498,"raft.run_wall_time_us":34618,"spinlock_wait_cycles":1792,"thread_start_us":498253,"threads_started":2,"wal-append.queue_time_us":513019}
I20260709 12:28:57.419781  6452 raft_consensus.cc:2393] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:28:57.399238 15836 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:57.346217 15979 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.398345  6385 rpcz_store.cc:276] 0709 12:28:55.818704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.270437 (+451733us) service_pool.cc:224] Handling call
0709 12:28:56.270847 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:56.765455 (+494608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.765497 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:56.765981 (+   484us) log.cc:844] Serialized 26 byte log entry
0709 12:28:57.040950 (+274969us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:57.276564 (+235614us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.277461 (+   897us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.277555 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.397740 (+120185us) raft_consensus.cc:1685] finished
0709 12:28:57.397765 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.397911 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":463252,"prepare.queue_time_us":52610,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":467173,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":27289,"thread_start_us":366469,"threads_started":2,"wal-append.queue_time_us":374759}
I20260709 12:28:57.390524 15947 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:57.426980  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:57.429875  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 121547us
I20260709 12:28:57.398933 15775 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:57.424274 15849 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.429055 15499 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:57.435642 15835 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:57.435796 15775 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:57.427392 15966 consensus_queue.cc:1048] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 12:28:57.449673  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.458819  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.460183 16043 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.445502 15518 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:28:57.445719 15892 consensus_queue.cc:1048] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:28:57.445782 15972 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:28:57.462832 15883 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.463291 16001 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:28:57.463205 15964 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:28:57.463927 15549 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.435508 15986 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.018s
I20260709 12:28:57.470444  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:57.472257  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.472841  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.543002  6376 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
W20260709 12:28:57.542277  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 211001us
I20260709 12:28:57.544092  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.473100  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.475742  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.478742  6308 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:28:57.476687  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:57.545185  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.446169 15499 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:57.492520  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.535112  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:28:57.522279  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.474671 15911 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:57.485772 15775 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:57.491487  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:57.447335 15849 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.479308  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:57.546726  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.553169  6383 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:57.554296  6301 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:57.548451  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.486070 16032 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:57.549535  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 57154us
I20260709 12:28:57.585807  6348 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:57.550050  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96263.1us
I20260709 12:28:57.588760  6348 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.547384  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.555230  6382 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:28:57.597760  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.598472  6382 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 25.
W20260709 12:28:57.551543  6192 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 63235.7us
I20260709 12:28:57.599288  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.540992 16001 consensus_queue.cc:1243] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 20.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:28:57.609143  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:57.600793  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 53512.6us
I20260709 12:28:57.610934  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:57.611203  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.611742  6301 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:57.612983  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.619210  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.549120 15499 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:57.630004  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 193878us
I20260709 12:28:57.490773 15817 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.485543 15835 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.529812 15897 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.540313 15883 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.557101 15979 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:57.556414 16044 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:28:57.640614  6383 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:57.617877 16052 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:57.609237 15888 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:57.636304  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.621459 15985 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:57.693642  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.636226 16037 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:57.552462 15911 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:28:57.696595  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 109317us
I20260709 12:28:57.629040 15891 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:57.703810  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67608.5us
I20260709 12:28:57.638100  7288 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.716888  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:57.711386  6376 raft_consensus.cc:3060] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:28:57.641395 15876 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:57.711416  6383 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:57.729593  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.689838 15979 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:57.731501  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.733451 15849 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.733958 16052 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:57.623900 16032 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.698966 16044 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.682924 15947 consensus_queue.cc:1048] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:28:57.713907 16051 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:57.739933  6385 raft_consensus.cc:2468] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
W20260709 12:28:57.749292  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 126853us
I20260709 12:28:57.755612  6376 raft_consensus.cc:2468] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 35.
I20260709 12:28:57.755527  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.734952 15966 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:57.764392  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.765002  6383 raft_consensus.cc:3060] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:57.737136 15491 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.713318 16029 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.768416  6382 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:28:57.770419 15849 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.771232 15964 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.743018 16052 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.750057 15979 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.756968 16044 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:57.785708  6383 raft_consensus.cc:2410] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:28:57.757866 16051 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.793804 16051 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:57.806950  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.808696  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:57.809369  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 67395.2us
I20260709 12:28:57.820508  6383 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:28:57.822093  6301 raft_consensus.cc:2468] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:57.821229  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.788074 16052 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:57.843341 15817 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.782131 15849 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.856665  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:57.865417  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:57.867470  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.868618  6383 raft_consensus.cc:2376] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25. Re-sending same reply.
I20260709 12:28:57.869107  6424 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:57.828470 15979 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:57.869892  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:28:57.842386 15886 consensus_meta.cc:287] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.000s
I20260709 12:28:57.878986  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.879889  6383 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:57.881036  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.881781  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:28:57.843462 15911 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.854791 16029 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.858879 15892 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:57.851046 15835 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.850273 15849 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.851114 16032 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.871237 15518 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:28:57.903364  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.883375 15886 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:57.909566  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.909943  6455 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:28:57.915731  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:57.910415  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:57.916671  6453 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:57.902117 16058 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:57.918274 16058 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:57.918679 16029 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:28:57.919235 16058 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.923468 16051 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.926591 16051 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:57.927050  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:57.926859 15886 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:57.882918  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.931003  6451 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:57.934429  6383 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:28:57.940623 15971 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.940963 15775 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:57.941274 15971 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:57.941532  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:57.943051  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:57.943825  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:57.948789  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:57.907706 15883 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:57.905972 16032 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.952718  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.909219 15997 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:57.958590  6383 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:57.931548 15849 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:57.934334 15886 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:28:57.963219  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 325184us
I20260709 12:28:57.955686 15876 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:57.964460 15886 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:57.988363  6383 raft_consensus.cc:2468] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:28:57.990770  7287 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:57.991556  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:57.992611  6383 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:57.973958 16065 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:57.960402 16032 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.001292 15876 raft_consensus.cc:2804] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:57.964146 15849 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.011929 15775 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.013249 15849 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.000671 16065 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:58.016090 16065 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:58.016242 15971 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:58.009428  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 239623us
I20260709 12:28:58.024775  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.008471  6383 raft_consensus.cc:2468] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:58.009212  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:58.011116 15876 raft_consensus.cc:697] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:58.035298  7287 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:58.040503  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.016455 16059 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.526s	user 0.003s	sys 0.000s
I20260709 12:28:58.036424  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.048498  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.065119  6453 raft_consensus.cc:2468] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:28:58.056084  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.075089 15876 consensus_queue.cc:237] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.078631 15849 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.079205 15849 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.081353 15849 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.086349  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3625) took 2266 ms. Trace:
I20260709 12:28:58.047245 15971 raft_consensus.cc:2804] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:58.101300 15971 raft_consensus.cc:697] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:58.102351 15971 consensus_queue.cc:237] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.113541  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:58.114192  6376 raft_consensus.cc:2376] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 35. Re-sending same reply.
I20260709 12:28:58.077991 15499 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.051461  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:58.094887  6384 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3929) took 1465 ms. Trace:
W20260709 12:28:58.201321  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 112756us
I20260709 12:28:58.201511  6384 rpcz_store.cc:276] 0709 12:28:56.630471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.755836 (+125365us) service_pool.cc:224] Handling call
0709 12:28:56.756081 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.872057 (+115976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.872111 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.872174 (+    63us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:28:56.872220 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.094631 (+1222411us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540010a4d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:58.094681 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.094808 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265859456}
I20260709 12:28:58.203487  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.095677 16032 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.086949  6377 rpcz_store.cc:276] 0709 12:28:55.820138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.361120 (+540982us) service_pool.cc:224] Handling call
0709 12:28:56.361595 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:56.765534 (+403939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.765562 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:56.779800 (+ 14238us) log.cc:844] Serialized 52 byte log entry
0709 12:28:57.257595 (+477795us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:57.297050 (+ 39455us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.297407 (+   357us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.297567 (+   160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.297637 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:57.354116 (+ 56479us) spinlock_profiling.cc:243] Waited 8.89 ms on lock 0x7b180003e980. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:58.003440 (+649324us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180003e980. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f
0709 12:28:58.004683 (+  1243us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b48001f0158. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa81d0aa1 00007f6fa81c7b37 00007f6fa81c79ca 00007f6fa82bf26f 00007f6fa82bef37 00007f6fa82d355c 00007f6fa82d3508 00007f6fa82d3488 00007f6fa82d3443 00007f6fa82d26ec 00007f6fa71fca96
0709 12:28:58.083965 (+ 79282us) spinlock_profiling.cc:243] Waited 4.07 ms on lock 0x7b180003e980. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:58.086094 (+  2129us) raft_consensus.cc:1685] finished
0709 12:28:58.086125 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.086282 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":416693,"prepare.queue_time_us":105915,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":418067,"raft.run_cpu_time_us":4012,"raft.run_wall_time_us":718709,"spinlock_wait_cycles":46848128,"thread_start_us":454733,"threads_started":1,"wal-append.queue_time_us":528445}
I20260709 12:28:58.100361 16044 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.116825  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:58.077919 16052 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.210988  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.117167  6455 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 29. Current term is 30. Ops: []
W20260709 12:28:58.198721  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 82338.1us
I20260709 12:28:58.213754  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.214918  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.216903  6383 raft_consensus.cc:2393] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:58.216822  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.219460  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:58.224303  7285 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:58.229003  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:58.229441  6452 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:58.231978  6454 raft_consensus.cc:1240] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:58.234586  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.234745 15897 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:58.235795  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.248593  6425 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:28:58.243686  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.248265  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3936) took 1551 ms. Trace:
I20260709 12:28:58.251420  6378 rpcz_store.cc:276] 0709 12:28:56.697045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.858115 (+161070us) service_pool.cc:224] Handling call
0709 12:28:56.877504 (+ 19389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.913424 (+ 35920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.913470 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.913529 (+    59us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:56.913575 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.248040 (+1334465us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b5400038754. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:58.248082 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.248195 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82016000}
I20260709 12:28:58.234797 16018 raft_consensus.cc:2804] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:58.227101 15886 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.284307  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.241351 15775 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:58.292017  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.293148  7318 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:58.298099  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:58.298883  6378 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:58.266827 15673 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:58.263204 16032 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:58.306638  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.283504 16071 raft_consensus.cc:2804] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:28:58.308142 15549 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:58.312439 15549 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.316408 15549 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.248682 15892 consensus_queue.cc:1059] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:28:58.283252 15897 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:58.253700 15979 consensus_queue.cc:1059] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:28:58.334954  6378 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:28:58.335495 15910 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:58.335577 15499 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.342989 15991 consensus_queue.cc:1059] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:58.345284 16071 raft_consensus.cc:697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:58.346976 15886 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.349851  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:28:58.369146  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.369263  6378 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:58.296623 16067 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:58.352795  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 46244.6us
I20260709 12:28:58.356393 15549 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:58.365073 15979 raft_consensus.cc:3055] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:58.340221 16051 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:28:58.384809  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:58.391220 16071 consensus_queue.cc:237] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.394919 16044 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.396595 16065 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.354727 15897 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.401247 15897 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:58.386257 15979 raft_consensus.cc:740] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:58.402997 15979 consensus_queue.cc:260] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.404212 15979 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:58.413582 16052 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.360420 15910 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.415330  6457 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 4134) took 1163 ms. Trace:
I20260709 12:28:58.415894 15997 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:58.416786 15910 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.415794  6457 rpcz_store.cc:276] 0709 12:28:57.252367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.252505 (+   138us) service_pool.cc:224] Handling call
0709 12:28:57.252763 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:57.269997 (+ 17234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.270039 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:57.280543 (+ 10504us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.117383 (+836840us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b18004a44c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e470d1 00007f6fa3262a0f
0709 12:28:58.254044 (+136661us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:28:58.254090 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.254194 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.414965 (+160771us) raft_consensus.cc:1685] finished
0709 12:28:58.415002 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.415258 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4918,"raft.queue_time_us":17576,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15259,"spinlock_wait_cycles":4743040,"thread_start_us":53162,"threads_started":1,"wal-append.queue_time_us":1073577}
I20260709 12:28:58.417550 15910 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:58.419461  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.421137  6383 raft_consensus.cc:3060] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:58.340924 16032 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.427071  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:58.432003  6376 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:58.434628  7286 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.436504  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 128874us
I20260709 12:28:58.444185  6378 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:28:58.457423  6376 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:28:58.478519  6385 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 21.
I20260709 12:28:58.481082  6383 raft_consensus.cc:1275] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:56.793449  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:28:58.491518  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 196476us
I20260709 12:28:58.497283  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3654) took 1007 ms. Trace:
I20260709 12:28:58.497820  6378 rpcz_store.cc:276] 0709 12:28:57.489899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.486265 (+996366us) service_pool.cc:224] Handling call
0709 12:28:58.496785 (+ 10520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:58.496963 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.496987 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:58.497030 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:58.497062 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.497116 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.497219 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:58.437698 15910 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.507982  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:58.508649  6378 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:58.509749 16018 raft_consensus.cc:697] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:58.541795  6378 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:28:58.544538  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.546586  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.547638  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.549808  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.550611  6307 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:58.551640  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:58.552824  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.553229  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.555035  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.556452  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.557921  6305 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:28:58.558244  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.558787  6305 pending_rounds.cc:85] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Aborting all ops after (but not including) 14
I20260709 12:28:58.559042  6307 raft_consensus.cc:2393] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:58.559299  6305 pending_rounds.cc:107] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:28:58.559738  6305 raft_consensus.cc:2889] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:58.561772  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.562007  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.561904  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.563292  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.564586  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:58.561276  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.566726  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:58.576562  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 99019us
I20260709 12:28:58.585250  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.588153  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.598129  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.586284 15849 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:58.601824  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.599073  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.599397  6305 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 24: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:58.580186 16033 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:58.558930 15655 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:28:58.604143  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.608232  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.610306 16003 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:58.611078 16003 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.603606 16032 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.614719  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.614889  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.615267  7331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43766 (request call id 1387) took 1304 ms. Trace:
I20260709 12:28:58.610347  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.614146 16044 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:58.613562 15729 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:28:58.616070  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.621752 15673 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:58.626986  6348 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.629206 15955 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.626983 15673 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:58.629729 15955 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.631417 15886 consensus_queue.cc:1059] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:58.632233 15886 consensus_queue.cc:1243] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 29.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:28:58.632694 15886 raft_consensus.cc:3055] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:58.633069 15886 raft_consensus.cc:740] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:58.633571  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:58.635737 16083 log.cc:927] Time spent T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:28:58.636072  6455 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 28. Current term is 29. Ops: []
W20260709 12:28:58.637575 16081 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:28:58.639266  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.639086 16018 consensus_queue.cc:237] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.639380  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:46610 (request call id 1413) took 1162 ms. Trace:
I20260709 12:28:58.640419  6382 rpcz_store.cc:276] 0709 12:28:57.476744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.608862 (+132118us) service_pool.cc:224] Handling call
0709 12:28:58.639220 (+1030358us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b540023e1d4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:58.639322 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364381440}
I20260709 12:28:58.641286  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1414) took 1159 ms. Trace:
I20260709 12:28:58.643537  6457 raft_consensus.cc:1217] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Deduplicated request from leader. Original: 26.18->[27.19-32.21]   Dedup: 27.19->[30.20-32.21]
I20260709 12:28:58.644138  6457 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 32. Current term is 33. Ops: [27.19-32.21]
I20260709 12:28:58.616143  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:28:58.644760 16057 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.907s	user 0.000s	sys 0.002s
I20260709 12:28:58.645598  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.646203 16051 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:58.646885  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.647603 16089 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:58.648073 15955 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:58.646420  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:58.649698  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.617425  6378 raft_consensus.cc:1240] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:28:58.623495  6309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57668 (request call id 3629) took 1317 ms. Trace:
I20260709 12:28:58.650866  6309 rpcz_store.cc:276] 0709 12:28:57.306313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.307863 (+  1550us) service_pool.cc:224] Handling call
0709 12:28:57.308191 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.687556 (+379365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.687602 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:57.687643 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:57.687672 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.623198 (+935526us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54000aa854. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:58.623257 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.623427 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":872009984}
I20260709 12:28:58.651888  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.652487  6452 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:58.625720  7332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:43752 (request call id 1689) took 1444 ms. Trace:
W20260709 12:28:58.653296  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65528.8us
I20260709 12:28:58.654078  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.641680  6377 rpcz_store.cc:276] 0709 12:28:57.482234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.219895 (+737661us) service_pool.cc:224] Handling call
0709 12:28:58.220306 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.220818 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.220862 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.223656 (+  2794us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.445845 (+222189us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:58.537515 (+ 91670us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.537861 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.537967 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.641076 (+103109us) raft_consensus.cc:1685] finished
0709 12:28:58.641102 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.641227 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9822,"prepare.queue_time_us":140590,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":3033,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":24875,"thread_start_us":221559,"threads_started":2,"wal-append.queue_time_us":259725}
I20260709 12:28:58.655629  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.654913  6452 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:28:58.653810  7332 rpcz_store.cc:276] 0709 12:28:57.181449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.186706 (+  5257us) service_pool.cc:224] Handling call
0709 12:28:57.187030 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.187237 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 24
0709 12:28:57.187508 (+   271us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.625310 (+1437802us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b1800445240. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa2e4f4bc 00007f6fa019d48b 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b7455 00007f6fa329f060 00007f6fa32b1716 00007f6fa32c2287
0709 12:28:58.625401 (+    91us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:58.625449 (+    48us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:28:58.625492 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.625554 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.625664 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23529,"prepare.queue_time_us":158,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":27249,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":86,"spinlock_wait_cycles":9548032,"thread_start_us":215906,"threads_started":2,"wal-append.queue_time_us":239370,"wal-append.run_cpu_time_us":1346,"wal-append.run_wall_time_us":1046660}
I20260709 12:28:58.659189  7316 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:58.659565  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:58.658808  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:58.662760  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 46769.7us
I20260709 12:28:58.664034  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.654206  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:28:58.666814  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 89265.4us
I20260709 12:28:58.648607 15948 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.008s
I20260709 12:28:58.615989  7331 rpcz_store.cc:276] 0709 12:28:57.310503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.310647 (+   144us) service_pool.cc:224] Handling call
0709 12:28:57.314356 (+  3709us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.314608 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:28:57.319451 (+  4843us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.373238 (+1053787us) spinlock_profiling.cc:243] Waited 4.19 ms on lock 0x7b18004afaa0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa019ce07 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa2e4120b 00007f6fa2e46d23 00007f6fa2e4723a 00007f6fa32b1a2a
0709 12:28:58.602739 (+229501us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.613899 (+ 11160us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:58.614019 (+   120us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:58.614271 (+   252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.614398 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.615162 (+   764us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16418,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":30058,"raft.queue_time_us":3102,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":4250,"spinlock_wait_cycles":16612864,"thread_start_us":228790,"threads_started":2,"wal-append.queue_time_us":1267325}
W20260709 12:28:58.656041 16078 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.162s	user 0.001s	sys 0.001s
I20260709 12:28:58.667909  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.670823  6348 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.688501  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.654681 15985 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:58.637630 15947 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.617549 15797 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:58.612867 15897 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:28:58.633961 15886 consensus_queue.cc:260] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.669631 16032 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:58.696736  6274 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.699632  7314 raft_consensus.cc:1240] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:28:58.666760 16033 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:58.700805 16097 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.019s
I20260709 12:28:58.719483 16096 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:58.719964 16096 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:58.720316 16096 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:58.720767 15886 raft_consensus.cc:3060] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:58.722370 16074 log.cc:927] Time spent T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.456s	user 0.001s	sys 0.000s
I20260709 12:28:58.723237  6274 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.723519 15978 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:58.690088 16095 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:58.723915 15978 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:58.724184 16095 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:58.725989  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3949) took 1402 ms. Trace:
I20260709 12:28:58.726533  6379 rpcz_store.cc:276] 0709 12:28:57.323508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.325516 (+  2008us) service_pool.cc:224] Handling call
0709 12:28:57.325906 (+   390us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:28:57.418355 (+ 92449us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:28:57.429311 (+ 10956us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.489084 (+ 59773us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:28:57.489851 (+   767us) log.cc:844] Serialized 78 byte log entry
0709 12:28:57.489996 (+   145us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:57.490241 (+   245us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.515195 (+ 24954us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.515532 (+   337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.515655 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.104932 (+589277us) spinlock_profiling.cc:243] Waited 4.35 ms on lock 0x7b18004c4e60. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:58.644173 (+539241us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:58.725654 (+ 81481us) raft_consensus.cc:1685] finished
0709 12:28:58.725688 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.725895 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24273,"prepare.queue_time_us":26874,"prepare.run_cpu_time_us":171,"prepare.run_wall_time_us":38626,"raft.queue_time_us":25971,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":9236,"spinlock_wait_cycles":174787072,"thread_start_us":35391,"threads_started":1,"wal-append.queue_time_us":246264}
I20260709 12:28:58.735702  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:58.736325  6379 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:28:58.724454 15978 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:58.739239  6379 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: [21.12-21.13]
I20260709 12:28:58.700414 15861 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:58.703373  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.730120 15948 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:58.744005  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 80055.9us
W20260709 12:28:58.713948 16049 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.941s	user 0.003s	sys 0.000s
I20260709 12:28:58.745692  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:58.753450  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.691555 15985 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.716226 16003 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.724759 15910 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.763192 15910 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:58.763370  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:58.726686 16044 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:58.764524 16044 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:58.710693 16033 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:58.765125 15985 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.759820  6274 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.724677 16095 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:58.695587 15947 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.767494  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.769842  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.781719 16058 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:58.782310 16058 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.784489 16058 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:58.785254 15947 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.745312 16084 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:58.791510 16087 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:28:58.792893 15970 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.802414  6456 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 4135) took 1549 ms. Trace:
I20260709 12:28:58.802944  6456 rpcz_store.cc:276] 0709 12:28:57.252732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.253970 (+  1238us) service_pool.cc:224] Handling call
0709 12:28:57.254353 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:57.271054 (+ 16701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.271088 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:57.271564 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:28:57.547177 (+275613us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:57.547225 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.547295 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.713329 (+1166034us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:28:58.802059 (+ 88730us) raft_consensus.cc:1685] finished
0709 12:28:58.802090 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.802341 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15707,"raft.queue_time_us":16036,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":15894,"spinlock_wait_cycles":83031424,"thread_start_us":275292,"threads_started":1,"wal-append.queue_time_us":472176}
W20260709 12:28:58.807344  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 47625us
I20260709 12:28:58.807684 15655 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:58.767012 15948 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:58.766944 16097 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:58.831259  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 160572us
I20260709 12:28:58.835124  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:58.837342 15861 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:58.837734 16106 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.838418 16106 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.834031 15948 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:58.843405 16097 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:58.845181  7329 raft_consensus.cc:2410] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 32 because replica has last-logged OpId of term: 31 index: 23, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 22.
I20260709 12:28:58.846665 16097 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:58.849499 16097 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.846029 15998 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.850610 15998 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.856365 16109 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:58.847338 15673 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.867951 16086 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:58.868376 16086 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:58.869238 16086 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.873849 16065 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:58.875569  6383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1419) took 1391 ms. Trace:
I20260709 12:28:58.875976  6383 rpcz_store.cc:276] 0709 12:28:57.483715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.227696 (+743981us) service_pool.cc:224] Handling call
0709 12:28:58.420990 (+193294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:58.483778 (+ 62788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.875327 (+391549us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54003e4ed0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:58.875501 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443838720}
I20260709 12:28:58.877655  6383 raft_consensus.cc:1275] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:28:58.825893 16084 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.890445 16084 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.883803 16106 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:58.876513 16097 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:58.899546  6376 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3951) took 1358 ms. Trace:
I20260709 12:28:58.899921  6376 rpcz_store.cc:276] 0709 12:28:57.541579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.481579 (+940000us) service_pool.cc:224] Handling call
0709 12:28:58.898581 (+417002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:58.898771 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.898797 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:58.898836 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:28:58.898867 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.898918 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.899408 (+   490us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400194bd0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a68ad 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:58.899495 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958349184}
I20260709 12:28:58.901583  6376 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 20. (term mismatch)
I20260709 12:28:58.901901  6376 pending_rounds.cc:85] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 19
I20260709 12:28:58.902108  6376 pending_rounds.cc:107] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 30.20
I20260709 12:28:58.886687 16095 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.905018 16095 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:58.870371 16086 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:58.909524  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.888206  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3930) took 2256 ms. Trace:
I20260709 12:28:58.888062 16096 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:58.916086  7329 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43752 (request call id 1679) took 2152 ms (client timeout 2048 ms). Trace:
I20260709 12:28:58.914786  6380 rpcz_store.cc:276] 0709 12:28:56.631668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.757425 (+125757us) service_pool.cc:224] Handling call
0709 12:28:56.757676 (+   251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.872481 (+114805us) raft_consensus.cc:1534] Early marking committed up to index 24
0709 12:28:57.144830 (+272349us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.324930 (+180100us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:57.324987 (+    57us) raft_consensus.cc:1645] Marking committed up to 24
0709 12:28:57.325050 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.887952 (+1562902us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54001f4ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:58.888000 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.888121 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":220994,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":90,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":35921,"spinlock_wait_cycles":263510272,"thread_start_us":451791,"threads_started":2,"wal-append.queue_time_us":284371,"wal-append.run_cpu_time_us":1311,"wal-append.run_wall_time_us":1027669}
I20260709 12:28:58.891607  6383 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 27. Current term is 28. Ops: []
W20260709 12:28:58.893512 16103 log.cc:927] Time spent T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:28:58.917423  6380 raft_consensus.cc:1240] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:28:58.892144  6384 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:58.916306 16096 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:58.918757 15775 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:28:58.919144 15775 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:58.919940 15775 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.921063 15775 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:28:58.921526  6384 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: [25.13-26.14]
I20260709 12:28:58.918349  6381 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 3940) took 2152 ms. Trace:
I20260709 12:28:58.930938  6381 rpcz_store.cc:276] 0709 12:28:56.766252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.172284 (+406032us) service_pool.cc:224] Handling call
0709 12:28:57.182068 (+  9784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.200179 (+ 18111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.200214 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:57.200253 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:28:57.200289 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.918170 (+1717881us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5400109ad4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:58.918209 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.918296 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41072384}
I20260709 12:28:58.896757  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:59632 (request call id 4144) took 1449 ms. Trace:
I20260709 12:28:58.932261  6451 rpcz_store.cc:276] 0709 12:28:57.447788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:57.447990 (+   202us) service_pool.cc:224] Handling call
0709 12:28:57.448364 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.945097 (+496733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.896534 (+951437us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b5400266954. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32a7e01 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2 00007f6fa15e248c 00007f6fa15e18c6
0709 12:28:58.896698 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102152064}
I20260709 12:28:58.915313  6379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3664) took 1033 ms. Trace:
I20260709 12:28:58.936097 15673 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:58.937427  6378 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3661) took 1056 ms. Trace:
I20260709 12:28:58.937804  6378 rpcz_store.cc:276] 0709 12:28:57.881232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.650860 (+769628us) service_pool.cc:224] Handling call
0709 12:28:58.651156 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.661836 (+ 10680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.661863 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.662316 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.747644 (+ 85328us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:58.887689 (+140045us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b180037c5a0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:28:58.935938 (+ 48249us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.937053 (+  1115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.937132 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.937169 (+    37us) raft_consensus.cc:1685] finished
0709 12:28:58.937249 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.937371 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14634,"prepare.queue_time_us":20566,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":18904,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":17646,"spinlock_wait_cycles":14073984,"thread_start_us":223109,"threads_started":2,"wal-append.queue_time_us":79884}
I20260709 12:28:58.935869  6379 rpcz_store.cc:276] 0709 12:28:57.881921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.740042 (+858121us) service_pool.cc:224] Handling call
0709 12:28:58.740460 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.740797 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.740823 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.745205 (+  4382us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.830691 (+ 85486us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:28:58.892463 (+ 61772us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.914957 (+ 22494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.915047 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:58.915087 (+    40us) raft_consensus.cc:1685] finished
0709 12:28:58.915100 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.915245 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8757,"prepare.queue_time_us":61930,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":22071,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":4227,"thread_start_us":61706,"threads_started":1,"wal-append.queue_time_us":98052}
I20260709 12:28:58.944276 15964 raft_consensus.cc:3055] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:58.944456  7318 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:28:58.944886 15964 raft_consensus.cc:740] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:58.945930  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:58.946832  7316 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 32.
I20260709 12:28:58.957270 16097 consensus_queue.cc:1059] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:58.945849 15964 consensus_queue.cc:260] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.958947 15964 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:58.959642 15729 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:58.960336 15729 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:58.957281 15991 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:28:58.969745  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:28:58.974004  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 140524us
I20260709 12:28:58.975665  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:58.976035  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:58.977926  6305 raft_consensus.cc:1240] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 21. Current term is 22. Ops: []
I20260709 12:28:58.981148 16089 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:58.981500  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:58.981832 16089 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:58.985824 16089 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:58.916539  7329 rpcz_store.cc:269] 0709 12:28:56.763410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.763620 (+   210us) service_pool.cc:224] Handling call
0709 12:28:58.915856 (+2152236us) spinlock_profiling.cc:243] Waited 2.06 s on lock 0x7b5400566154. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3210571 00007f6fa32ab2fa 00007f6fa826cb17 00007f6fa17e76d4 00007f6fa17e7662 00007f6fa17e7591 00007f6fa17e750c 00007f6fa17e67c2 00007f6fa15e248c 00007f6fa15e18c6 00007f6fa15e126f
0709 12:28:58.915988 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4748511104}
I20260709 12:28:58.990321 16071 consensus_queue.cc:1059] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:58.991202 16071 consensus_queue.cc:1243] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 25.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:28:58.991780 15673 raft_consensus.cc:3055] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:28:58.992200 15673 raft_consensus.cc:740] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:58.995318 15673 consensus_queue.cc:260] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:58.996611 15673 raft_consensus.cc:3060] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:28:58.902328  6376 raft_consensus.cc:2889] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:58.998620 15655 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:58.999912 15655 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:28:59.006630  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:58.961483 15729 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.019819 15729 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:59.019986  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.020952 16109 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:59.021865  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 173675us
I20260709 12:28:59.023145 16109 consensus_queue.cc:1059] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:59.024014 16109 consensus_queue.cc:1243] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:59.024610 15888 raft_consensus.cc:3055] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:28:59.024945 15888 raft_consensus.cc:740] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:28:59.025741 15888 consensus_queue.cc:260] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.028043  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.028805  6303 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 32: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:28:59.029531 15888 raft_consensus.cc:3060] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:59.029297  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.030038  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.030212  6309 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:28:59.030910  6308 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:59.032337 15998 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:28:59.032990  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79550.3us
I20260709 12:28:59.034462  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.035001  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:59.035118  6379 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:59.036398  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:59.038049  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.051348 15986 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:59.053378  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:59.054222 15986 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:59.054617 15986 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:28:59.054963 15986 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:58.966980 15977 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:59.062789 15977 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.066664 15977 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:28:59.081429  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 58549.1us
I20260709 12:28:59.082466  6348 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.087392  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.098145  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:28:59.098959  7288 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.099947  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
W20260709 12:28:59.102088  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 398783us
I20260709 12:28:59.104076  6275 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:28:59.108068  6275 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.5.4.68:43625 timed out after 2.048s (SENT) [suppressed 15 similar messages]
W20260709 12:28:59.080338 16110 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
I20260709 12:28:59.113183  6348 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.115370  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.117834  6305 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:28:59.116964  6197 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 14 to 20, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 20 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.052546 15978 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.124828  6197 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.143667  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.144472  6309 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:28:59.144299  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.132283  6197 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 25. New cstate: current_term: 25 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:59.098131 15518 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.153942  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.148524  6197 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.102509 15977 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.102802 16101 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:58.949851 15948 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.123901 16109 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:28:59.110983 16106 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:28:59.155584  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.123225 15888 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:59.156487  6301 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 22. Current term is 23. Ops: []
W20260709 12:28:59.160457  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 148) took 29512 ms (client timeout 15000 ms). Trace:
I20260709 12:28:59.161216  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:59.161685  6197 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 25 to 28. New cstate: current_term: 28 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.166965  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:59.159643  6425 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:59.173074  6197 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 27. New cstate: current_term: 27 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:28:59.156638 16119 log.cc:927] Time spent T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:28:59.179409  6382 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51854 (request call id 3660) took 1298 ms. Trace:
I20260709 12:28:59.179527  6197 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:28:59.183904  6424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.182565  6382 rpcz_store.cc:276] 0709 12:28:57.880983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.641341 (+760358us) service_pool.cc:224] Handling call
0709 12:28:58.641637 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.710316 (+ 68679us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:28:58.738658 (+ 28342us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.861710 (+123052us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.862192 (+   482us) log.cc:844] Serialized 26 byte log entry
0709 12:28:58.862355 (+   163us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:58.977305 (+114950us) log.cc:844] Serialized 14 byte log entry
0709 12:28:58.977517 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.977588 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:59.179133 (+201545us) raft_consensus.cc:1685] finished
0709 12:28:59.179165 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.179353 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62154,"prepare.queue_time_us":44401,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":95,"raft.queue_time_us":55680,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":5582,"thread_start_us":247362,"threads_started":3,"wal-append.queue_time_us":209540}
I20260709 12:28:59.184139  7317 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33. Ops: [32.22-32.22]
I20260709 12:28:59.188206  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.183446  6197 catalog_manager.cc:5697] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 26, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 26 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.156028 16116 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:59.193111  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:59.192970  6197 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:59.190611  6456 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 22.
I20260709 12:28:59.175246 16099 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:28:59.170583  6202 rpcz_store.cc:269] 0709 12:28:29.648742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:40.444974 (+10796232us) service_pool.cc:224] Handling call
0709 12:28:59.160374 (+18715400us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:55.621533 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:55.727488 (+105955us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:55.732118 (+  4630us) write_op.cc:435] Acquired schema lock
0709 12:28:55.737873 (+  5755us) tablet.cc:662] Decoding operations
0709 12:28:55.761307 (+ 23434us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:55.761384 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:55.761412 (+    28us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:28:55.791012 (+ 29600us) tablet.cc:701] Row locks acquired
0709 12:28:55.791036 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:28:55.791273 (+   237us) write_op.cc:270] Start()
0709 12:28:55.791445 (+   172us) write_op.cc:276] Timestamp: P: 1783600135791203 usec, L: 0
0709 12:28:55.791470 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:28:55.792519 (+  1049us) log.cc:844] Serialized 6564 byte log entry
0709 12:28:56.172874 (+380355us) op_driver.cc:464] REPLICATION: finished
0709 12:28:56.231777 (+ 58903us) spinlock_profiling.cc:243] Waited 37.7 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa019d0ac 00007f6fa8ff79a9 00007f6fa01b969b 00007f6fa01b793f 00007f6fa01b9c02 00007f6fa6cddef0
0709 12:28:56.397751 (+165974us) write_op.cc:301] APPLY: starting
0709 12:28:56.397886 (+   135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:56.446011 (+ 48125us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:56.446037 (+    26us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:56.573269 (+127232us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:56.573724 (+   455us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 12:28:56.573792 (+    68us) write_op.cc:312] APPLY: finished
0709 12:28:56.574374 (+   582us) log.cc:844] Serialized 156 byte log entry
0709 12:28:56.632873 (+ 58499us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:56.737916 (+105043us) write_op.cc:454] Released schema lock
0709 12:28:56.765201 (+ 27285us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:56.797324 (+ 32123us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:56.799586 (+  2262us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":259442,"reported_tablets":31,"child_traces":[["op",{"apply.queue_time_us":224517,"apply.run_cpu_time_us":24338,"apply.run_wall_time_us":381519,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75221,"num_ops":17,"prepare.queue_time_us":551726,"prepare.run_cpu_time_us":32500,"prepare.run_wall_time_us":280138,"raft.queue_time_us":236864,"raft.run_cpu_time_us":23474,"raft.run_wall_time_us":237018,"replication_time_us":381177,"spinlock_wait_cycles":86775808,"thread_start_us":587044,"threads_started":4,"wal-append.queue_time_us":111162,"wal-append.run_cpu_time_us":80400,"wal-append.run_wall_time_us":1704152}]]}
I20260709 12:28:59.181511 15948 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:59.199306  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.202025  6453 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:59.203608  6455 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:59.181175 16065 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:59.213364 16065 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.213804 16065 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:59.189044 16089 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:59.202983  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:28:59.204658  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 96622.8us
I20260709 12:28:59.159186 15518 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.224362  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:59.189577 16112 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.195650  6451 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 12:28:59.186275 16062 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
W20260709 12:28:59.243672  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 130547us
I20260709 12:28:59.206072  6197 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 9, term changed from 16 to 19, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67), NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added, now has a pending config: VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65), VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66), VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67), VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 19 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:59.244663  6348 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.246073  6197 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 26 to 27, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.248193  6197 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 23, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 23 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:59.225337  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.230027 16095 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:28:59.235056  6451 pending_rounds.cc:85] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 16
I20260709 12:28:59.250039 16095 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.250185  6451 pending_rounds.cc:107] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:28:59.250761 16095 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:59.250801  6451 raft_consensus.cc:2889] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:59.225308  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.158460 15977 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.252523  7316 raft_consensus.cc:1240] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:59.253856 15977 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.250319  6197 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.255033  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.255702  6452 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:59.255540  6197 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 26, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 26 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.257017  7285 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:59.257138  6197 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 26, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 26 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:59.258993  6197 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 27, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 27 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:28:59.260335  6197 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 24 to 28, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:28:59.263144  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73199.4us
I20260709 12:28:59.265445  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.206400 15499 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:28:59.268651 15977 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:59.245002 16112 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.209054 16099 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.270792  6377 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46610 (request call id 1421) took 1324 ms. Trace:
I20260709 12:28:59.272396 15998 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:59.204459 16116 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.275671 16084 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:59.312497 15978 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.244288 16029 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:59.313303 16029 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.313727 16029 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:59.308208 15998 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.315500 15998 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:59.239771 16051 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.058s
I20260709 12:28:59.239509 16089 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.320469 16089 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.278597  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.262025  6197 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 23, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.279191 16101 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:59.321715  6348 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.321810 16101 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:59.322328 16101 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:59.322790 16101 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:59.322103  6197 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 14 to 19, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 19 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:28:59.322911  6348 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.323822 16116 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:59.324333  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.323825  6197 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 25, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.325057 15897 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:59.325812 15897 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:59.263577 16070 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:59.325914  6197 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 24 to 25, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.288295 15518 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:59.285328 16113 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:28:59.293933  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:59.295437  6382 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: [25.13-26.14]
I20260709 12:28:59.263088 15910 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:59.297653  6377 rpcz_store.cc:276] 0709 12:28:57.945922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.860243 (+914321us) service_pool.cc:224] Handling call
0709 12:28:58.860578 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:58.860920 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.860962 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:58.865089 (+  4127us) log.cc:844] Serialized 26 byte log entry
0709 12:28:59.158296 (+293207us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:28:59.190913 (+ 32617us) log.cc:844] Serialized 14 byte log entry
0709 12:28:59.263309 (+ 72396us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:59.263393 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:59.270555 (+  7162us) raft_consensus.cc:1685] finished
0709 12:28:59.270575 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:59.270727 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91633,"prepare.queue_time_us":25419,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":75900,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":1763,"thread_start_us":272493,"threads_started":1,"wal-append.queue_time_us":314579}
I20260709 12:28:59.218178 15986 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.326952 16070 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:59.356787  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.360214  6382 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:28:59.328825  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:59.329553  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.296612 15979 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:59.330260  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.292917  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:59.331504  6197 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 14, term changed from 20 to 24, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to <none>, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 24 committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:28:59.372365  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.372277  6197 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 22, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 22 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.331995  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.335273  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.336261  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.376183  6197 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 26, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 26 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.378521  6197 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 21. New cstate: current_term: 21 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.310799 15964 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:59.342208  6450 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:28:59.325976 16112 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.343776 16099 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.346750 16050 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:59.347376 15817 consensus_queue.cc:1048] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:59.347743 15862 consensus_queue.cc:1048] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:28:59.348670  6459 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
W20260709 12:28:59.355790  6425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.390586  6457 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:28:59.390905  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.391639  6380 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:28:59.364324  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:59.366335  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.368252  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.369560  6378 raft_consensus.cc:1275] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:59.337188  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.384752  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 35 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.394520  6385 raft_consensus.cc:2393] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 35: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:59.399165  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.385027 15964 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:28:59.411777 15977 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:59.412592 15977 consensus_queue.cc:1243] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer 4e0e521cb08c420f981a29576eee90a6 log is divergent from this leader: its last log entry 29.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:28:59.412812  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.414011  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.414633  6377 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 32: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:28:59.414779  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:59.417457  6381 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:59.417703  6451 raft_consensus.cc:1217] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Deduplicated request from leader. Original: 26.18->[27.19-33.21]   Dedup: 27.19->[30.20-33.21]
I20260709 12:28:59.418176  6451 pending_rounds.cc:85] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 19
I20260709 12:28:59.418391  6451 pending_rounds.cc:107] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted NO_OP operation due to leader change: 29.20
I20260709 12:28:59.418634  6451 raft_consensus.cc:2889] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:59.419031  6383 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:59.419729  6385 raft_consensus.cc:1275] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:28:59.356367 15910 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:59.420156  6385 pending_rounds.cc:85] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 12
I20260709 12:28:59.429749  6385 pending_rounds.cc:107] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 23.13
I20260709 12:28:59.430119  6385 raft_consensus.cc:2889] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:59.434422  6380 raft_consensus.cc:1275] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:28:59.437141  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.439205  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.433357  6348 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.413807 15986 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:59.441516  6384 raft_consensus.cc:1275] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 21. (term mismatch)
I20260709 12:28:59.443638  6384 pending_rounds.cc:85] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 20
I20260709 12:28:59.444370  6384 pending_rounds.cc:107] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 32.21
I20260709 12:28:59.444829  6384 raft_consensus.cc:2889] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:59.443738  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:28:59.468833  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.473592  6452 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:28:59.435461 15910 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:59.491318  6348 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.387507 15729 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 12:28:59.498939  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.443090 15977 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:28:59.505923  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 214931us
W20260709 12:28:59.507640  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 205416us
I20260709 12:28:59.513463  6303 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:59.509476  6424 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.486828 15888 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:59.515364  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 290111us
I20260709 12:28:59.519671  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.502117 16102 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:59.467090 16050 consensus_queue.cc:1048] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.045s
I20260709 12:28:59.505165 16109 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:28:59.532478  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:28:59.503675 16086 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:59.492295 15849 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:28:59.541357  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.541285  6272 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.515913 15817 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:59.550817 16089 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:59.551733 15974 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:59.551564 15549 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:28:59.552337 15974 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:59.553830  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:28:59.553524 15974 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.467079 16070 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.563824  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.558097  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.554867  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.536765 16095 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.538391 15888 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:59.525249 16062 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:59.524444 15729 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.578234  6456 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
W20260709 12:28:59.583932  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 85093.3us
I20260709 12:28:59.534618 16061 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:59.603577  6301 raft_consensus.cc:1240] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23. Ops: []
I20260709 12:28:59.543349 16086 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.606693 16086 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.608034  6302 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
W20260709 12:28:59.611104 16124 log.cc:927] Time spent T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:28:59.602919 15729 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.615492 15729 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.587294  6459 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:59.618026  6459 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:59.593255  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.522096 16045 consensus_queue.cc:1059] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.058s
I20260709 12:28:59.619062  6303 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:28:59.601338  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.621008  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.624895  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:28:59.624630  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.628260  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:28:59.633217  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71166.7us
I20260709 12:28:59.640914  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.642050  6301 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:28:59.542739 15998 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.651472  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:59.652372  6304 raft_consensus.cc:2393] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 32: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:59.659578  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 114814us
I20260709 12:28:59.667090  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:28:59.627887 15888 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:59.584971 16105 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:28:59.673540  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.627586 16062 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:59.634464 16109 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.634243 16061 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:59.668041 15794 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:28:59.663637 16134 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:59.675148 15797 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:59.674646 15729 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.695384  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.680074 15888 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:28:59.713881  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85717.1us
I20260709 12:28:59.714915  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:28:59.682076 16109 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:59.714074  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 81439.2us
I20260709 12:28:59.681108 15910 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.679605 16071 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
I20260709 12:28:59.732694  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:28:59.733310  6380 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:28:59.735242  6382 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35. Ops: []
I20260709 12:28:59.715963  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.712154 16113 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:28:59.721225 16061 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.723511 15998 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.701534 15729 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:59.729269 16065 consensus_meta.cc:287] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.004s
W20260709 12:28:59.785952  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 61839.4us
I20260709 12:28:59.730882 15862 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:28:59.788559 16084 consensus_queue.cc:1048] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:28:59.795470  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:28:59.800487  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 84584.5us
I20260709 12:28:59.802273 15910 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.803584 15964 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.804998 15964 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:59.730151 16095 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:28:59.806313  6380 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:28:59.806403  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:59.813365  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.813791  6302 raft_consensus.cc:2393] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:28:59.818681  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 144719us
I20260709 12:28:59.699913 16134 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.807332 15729 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.812021 16113 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:59.834546 16106 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:28:59.718063 16062 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:28:59.850737  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49105us
I20260709 12:28:59.799893 16065 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.799373 16029 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.791571 15491 consensus_queue.cc:1059] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 34.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.041s
I20260709 12:28:59.835240  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:28:59.825805 15862 consensus_queue.cc:1048] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.033s
W20260709 12:28:59.878690  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 43694.8us
I20260709 12:28:59.879623  6272 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:28:59.903522  7316 raft_consensus.cc:2368] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33.
I20260709 12:28:59.906255  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:28:59.906997  6308 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:59.875877 16101 consensus_meta.cc:287] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.005s
I20260709 12:28:59.906064  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:28:59.912508  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:28:59.914987  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:59.833591 15888 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.891018 16106 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:59.932288  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:28:59.873467 16142 consensus_queue.cc:1059] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:28:59.916473 15729 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:59.930022 15998 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:59.944835 16029 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.947149 16109 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.940835 16139 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:59.955406 15794 raft_consensus.cc:3055] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:59.955911 15794 raft_consensus.cc:740] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:59.956209 16065 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:28:59.956877 15794 consensus_queue.cc:260] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:28:59.959877 15797 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:59.921096 16101 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.962371 15817 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:59.962893  6376 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:28:59.963056 15817 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:59.965201 16101 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:28:59.966123 15998 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:28:59.966804 15998 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:59.968247  7314 raft_consensus.cc:1240] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35. Ops: []
I20260709 12:28:59.935302 16106 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.970872 16106 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:59.982249 15849 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:28:59.983027 15849 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:28:59.984612 15322 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:28:59.985244 15322 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:28:59.993693  6308 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:28:59.998138  7288 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:28:59.959767 16134 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:00.008000  6457 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:59.988689 15849 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.007726 16109 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.010221 15862 raft_consensus.cc:2804] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:00.019577 16109 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.021983 16109 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.026778 15862 raft_consensus.cc:697] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:28:59.986342 15322 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.028107 15322 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:00.028008 15862 consensus_queue.cc:237] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:00.039819  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 366341us
I20260709 12:29:00.045419 16136 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:00.045993 16136 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:00.046401 16136 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:00.050356 16070 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:00.053057 16071 consensus_queue.cc:1243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:29:00.055863 15794 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:00.062419  6459 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.063390 16106 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.064210  6459 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:00.065079 16116 raft_consensus.cc:3055] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:29:00.065483 16116 raft_consensus.cc:740] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:00.065730 16106 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.066279 16116 consensus_queue.cc:260] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.077557 16116 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:29:00.078946 15817 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.081482 15817 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.089629 16113 raft_consensus.cc:3055] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:29:00.090173 16113 raft_consensus.cc:740] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:00.091135 16113 consensus_queue.cc:260] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.092511 16113 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:29:00.095353 16099 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:00.095772 16099 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:00.096616 16099 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.112319 16099 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:00.115540 16136 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.117343 16136 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:00.189981  6272 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.194211  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:00.204771  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:00.206207  6308 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:29:00.225698  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.227833  6422 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:00.230904  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 189272us
I20260709 12:29:00.235337  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.236418  6424 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:00.238159  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.239797  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:29:00.240608  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:00.244390  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:29:00.244783  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 419439us
I20260709 12:29:00.245064  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.246001  6451 raft_consensus.cc:2368] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:29:00.248298  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.251341  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:00.255533  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:00.260838  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.262914  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.285332  6376 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:00.285614  6305 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:29:00.288913  6452 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:00.268077  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:29:00.296900  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 54581.9us
I20260709 12:29:00.300529  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:00.069365 15998 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.296926  6453 raft_consensus.cc:3060] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:29:00.306476  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:00.265614  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.320529  6376 raft_consensus.cc:2468] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
W20260709 12:29:00.322497  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73983.5us
I20260709 12:29:00.323800  6308 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:29:00.332993 15998 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:00.340950 15897 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:00.342583 15897 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.346896  6302 raft_consensus.cc:2368] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:29:00.347615  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:00.354364 15964 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.355333 15964 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.358281 16129 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.359344  6459 raft_consensus.cc:3060] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:00.359141 16129 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.370210  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:00.373638 16162 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.375146 16060 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:00.375818 15849 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.376394 16060 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:00.376976 16060 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.376736 15849 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.377424  7288 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:00.380185 16139 raft_consensus.cc:2804] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:29:00.380280 16162 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.382082  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:00.374033 15974 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.019s
I20260709 12:29:00.377434 16060 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:00.397508 16134 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.391484 15897 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.380980 16139 raft_consensus.cc:697] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:00.402384 16134 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.403831 15849 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.403724 16139 consensus_queue.cc:237] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 31.23, Last appended by leader: 23, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:00.407896  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 109911us
I20260709 12:29:00.395834 16162 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.424419  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:00.431483  7287 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:00.431267  6424 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:00.433490  7318 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:00.433424 16162 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.427240 16134 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.443334 15499 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:00.444203 15849 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:00.443411  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:00.445801  6458 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:00.445655 15849 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.450932 16147 consensus_queue.cc:1048] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:29:00.452100 15849 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.441960  6275 leader_election.cc:400] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33.
I20260709 12:29:00.452870  6275 leader_election.cc:403] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
W20260709 12:29:00.454383  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 59400.6us
I20260709 12:29:00.447624 16162 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.449036  7314 raft_consensus.cc:1240] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33. Ops: [32.22-32.22]
W20260709 12:29:00.455842  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 180721us
I20260709 12:29:00.458078 15729 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 32. Current term is 33.
I20260709 12:29:00.467409  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.468400  6455 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:29:00.468918 16165 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.469800 16165 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.467410  6376 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:29:00.471431  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:00.472198  6384 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:00.472800 16165 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.466450  6453 raft_consensus.cc:2468] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:29:00.480211 16163 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.481230 15964 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:00.481421 16163 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.488628 16162 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.473451 16129 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.493763  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:00.494473  6453 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:00.495329  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.496498  6455 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 27: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:29:00.500000  7287 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:00.460158  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:00.501829 16086 raft_consensus.cc:2804] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:29:00.503029 16086 raft_consensus.cc:697] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:00.504494 16086 consensus_queue.cc:237] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.505049  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.505860  6376 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 30.
I20260709 12:29:00.506582  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.510860  6377 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:29:00.508080  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.527390  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.527803  6380 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:29:00.531194  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.513113 16163 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.515858  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:00.516752 15518 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.518802 16147 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:29:00.536906  6302 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:29:00.520299  6453 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
W20260709 12:29:00.519974  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 79037.9us
I20260709 12:29:00.540719  7318 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:29:00.549244  7318 pending_rounds.cc:85] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 16
I20260709 12:29:00.556355  6459 raft_consensus.cc:2468] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:29:00.559731 15518 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.533501 15964 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.563031 15964 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.565693 15964 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.567214  7314 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:29:00.568295  6451 pending_rounds.cc:85] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Aborting all ops after (but not including) 16
I20260709 12:29:00.568693  6451 pending_rounds.cc:107] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 26.17
I20260709 12:29:00.569096  6451 raft_consensus.cc:2917] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Aborting config change with OpId 26.17: Aborted: Op aborted by new leader
I20260709 12:29:00.569231  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:00.569669  6451 raft_consensus.cc:2889] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:00.572252  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.555346  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.583240  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.549597  7318 pending_rounds.cc:107] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 26.17
I20260709 12:29:00.584517  7318 raft_consensus.cc:2917] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Aborting config change with OpId 26.17: Aborted: Op aborted by new leader
I20260709 12:29:00.584929  7318 raft_consensus.cc:2889] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:00.594142  6458 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:29:00.589792  6303 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:29:00.595949  6384 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:29:00.604880  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:00.605247  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:00.605944  6377 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:00.606729 16163 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.609264  6459 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 29: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:29:00.614418  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.615310  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.590809 15964 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:00.621827  7316 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEADER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 31. Current term is 32. Ops: []
I20260709 12:29:00.607407 15817 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.612623 16129 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:29:00.617274 15964 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.622400  6455 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:00.607626 16163 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.629004  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.633491  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.634286  6303 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:29:00.636710  6450 raft_consensus.cc:1240] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 21. Current term is 22. Ops: []
I20260709 12:29:00.636418 15817 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.640251 15518 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:00.640515  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:29:00.642243  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 43909.9us
I20260709 12:29:00.634060 15964 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:00.662835  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:00.668576 15817 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.672577  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.671452  6378 raft_consensus.cc:1240] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 31. Current term is 32. Ops: [31.20-31.20]
I20260709 12:29:00.681313  6377 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:29:00.692109  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.692987  6305 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:29:00.694461  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.696167 15655 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.696432  6458 raft_consensus.cc:1275] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:29:00.696803 15655 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.697554 16113 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.698825  6308 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:00.699815 16139 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:29:00.700392 16139 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:00.701506 16139 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.703537  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.704418  6456 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:00.707008 16106 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:00.708067  6303 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:29:00.698161 16113 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.711252 16113 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.711311 15655 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.712364  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.724709 16163 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.727934 16050 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:00.672359 16099 consensus_meta.cc:287] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Time spent flushing consensus metadata: real 0.501s	user 0.011s	sys 0.012s
I20260709 12:29:00.729095 16050 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.733286 16050 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:00.734694  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:00.670365 16060 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.735649  6376 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 in current term 32: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:00.736522  6454 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 25: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:29:00.737447 16060 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:00.745464  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.752497  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.746373  6302 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:29:00.751991  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.755421  6380 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
I20260709 12:29:00.757062  7285 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:00.757344  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:29:00.757491  7285 leader_election.cc:403] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:00.670652 16140 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.091s
I20260709 12:29:00.768318  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.769292  6303 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:29:00.770264 15817 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:29:00.776067  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.777029  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:00.779094  6272 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:00.780174  6272 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:00.782225 16096 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:00.777055  6380 raft_consensus.cc:2368] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:00.792788  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:00.782660 16096 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.813333  6453 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:00.814559 16129 raft_consensus.cc:1064] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
W20260709 12:29:00.815122 16129 raft_consensus.cc:1068] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Unable to promote non-voter 561a4e21d8b344a79af44568ca80a3c5: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:29:00.817610 15964 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.791989 16099 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:00.824107 15817 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:00.789876 15998 consensus_queue.cc:1048] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.037s
I20260709 12:29:00.801783 16096 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:00.827564  6274 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:29:00.844154  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 78664.9us
I20260709 12:29:00.841984 16099 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:00.846550 16099 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.856729  6348 leader_election.cc:400] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:29:00.857117  6348 leader_election.cc:403] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:29:00.859894 16162 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:29:00.861847  6348 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:00.863399  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:00.850613 16113 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:29:00.856223  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:00.868547 16163 raft_consensus.cc:2764] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election decision vote started in defunct term 26: won
I20260709 12:29:00.868395  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:00.870803 16163 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:00.872084  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80752.3us
I20260709 12:29:00.873277  6379 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:29:00.874282  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:00.850989 16147 consensus_queue.cc:1059] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.023s
I20260709 12:29:00.866622 16113 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.907873  6351 leader_election.cc:400] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27.
I20260709 12:29:00.909238  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:00.910255 15888 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:00.910403  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:00.916360  6348 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:00.947171  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58961.8us
I20260709 12:29:00.915678 15888 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.917730  6351 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:00.948437 15888 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:00.912400 16113 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:00.911237 16163 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:00.915438 15964 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:00.946677 16165 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:29:00.915947 16147 consensus_queue.cc:1243] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 32.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:29:00.945261 15849 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:29:00.967530 16109 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:00.968281 16109 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.968833 16109 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:00.972003 15974 raft_consensus.cc:3055] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:29:00.972437 15974 raft_consensus.cc:740] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:00.973254 15974 consensus_queue.cc:260] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:00.974912  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:00.975502  7316 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:29:00.955401 16134 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:00.980912 16134 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:00.995638 15974 raft_consensus.cc:3060] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:00.963754 16165 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.999285 16165 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:00.967911 15849 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:01.003942 16134 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:01.004910  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 214616us
I20260709 12:29:00.961817 16163 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:01.005527 16163 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:01.005971  6272 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:01.010839 16106 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:01.047322  6453 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 31 index: 23. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:29:01.050985 15998 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:29:01.065507  7316 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
W20260709 12:29:01.070714  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 183270us
I20260709 12:29:01.072181  6275 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:01.078788  6348 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.079612  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.080454 16099 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:01.088557 16140 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.088778  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.089212 16140 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.089937  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.090592  6303 raft_consensus.cc:3060] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:01.100600 16140 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:01.088442 16116 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:01.109191 16134 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:01.069558 15729 consensus_queue.cc:1059] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:29:01.111279  6348 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.116861 16096 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.118279  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.121225  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.124585  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:01.123513 16106 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:01.108492 16099 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:01.152063  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 234481us
I20260709 12:29:01.158357  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.158666  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:01.158898  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:01.127787 16134 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:01.161149  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.163043  6303 raft_consensus.cc:2468] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:29:01.137224 15849 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:01.130570 16116 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:01.167059  6300 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:29:01.167963  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.170926  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.172646  6456 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:29:01.180238  7314 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:29:01.184831 16178 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.005s
I20260709 12:29:01.189851  7288 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:01.191416  6385 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:29:01.214103  7287 leader_election.cc:400] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 22 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for earlier term 22. Current term is 23.
I20260709 12:29:01.171530 15849 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:01.219403  6377 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:29:01.218632  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:01.223815  6379 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:01.221379  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.185967 16178 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:01.237425  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.238193  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
W20260709 12:29:01.246877  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 88287.5us
I20260709 12:29:01.224957 15849 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:01.221415 15518 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:29:01.256083  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.233116 16178 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:01.235510 16134 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:29:01.289801  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 111850us
I20260709 12:29:01.299429 16050 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:01.300048 15655 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.300159 16050 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.300952 15655 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.302672  6305 raft_consensus.cc:2393] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 23: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
W20260709 12:29:01.303498 16175 log.cc:927] Time spent T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:29:01.247975 16106 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.300184  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.306981 16177 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.307535 16177 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.308080 16050 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.275538 16162 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:01.313683  6453 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:29:01.262852 15964 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:01.322525  6304 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
I20260709 12:29:01.332743  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.333284  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:01.334903  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:01.335245  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.336396  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.344842  6305 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 32.
I20260709 12:29:01.337101  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.338785  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.284992 16178 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:01.310189 16177 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:01.349934  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 95871.1us
I20260709 12:29:01.286715 16134 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:01.253268 15888 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.356016 16099 raft_consensus.cc:3055] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:29:01.358744 16096 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.361641 16165 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.362596 15964 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.313040 16116 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.387657  6379 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 29.
I20260709 12:29:01.395004  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:01.430043  6376 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 30. Current term is 31. Ops: []
I20260709 12:29:01.389554 16099 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
W20260709 12:29:01.438778  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:01.454093  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 102464us
W20260709 12:29:01.454748 16176 log.cc:927] Time spent T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:29:01.455644  6424 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:01.456523 15518 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:01.457237 16162 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:01.457374 15518 raft_consensus.cc:2804] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:01.458118 15518 raft_consensus.cc:697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:01.459422 15518 consensus_queue.cc:237] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.462600 16179 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.462709 16134 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.463506 16179 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.464118 16134 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:01.467211 15655 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:01.465512 16109 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.468930 15817 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.469520 15817 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.470870 15888 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.472010 15817 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:01.472769 16165 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.461861  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:29:01.452638  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66292.7us
I20260709 12:29:01.486145  6308 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 24. Current term is 25. Ops: [24.13-24.13]
I20260709 12:29:01.388751 16113 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.393649 16163 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.465775 16179 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:01.462633 16188 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.540959  6309 raft_consensus.cc:1240] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 30. Current term is 31. Ops: []
I20260709 12:29:01.504161 16109 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.541591  6377 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:29:01.542984  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 207844us
I20260709 12:29:01.546988 16189 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:01.551352  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:01.551614 16189 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.549037  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:01.552074  7329 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:29:01.555858  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.556631  6304 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 35.
I20260709 12:29:01.556864 16109 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:29:01.557406 16109 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.557762 16109 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:29:01.541469 15849 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.562196  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.563274  6383 raft_consensus.cc:1240] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:01.563963 15849 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.564459 15964 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.565706 16099 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.567428 16099 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:29:01.570564 16169 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:29:01.572407  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.539170 16188 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.576434 16188 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.579147 16163 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.578866 16061 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:01.588279  6377 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:29:01.586520  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.590356  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.591964  6348 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.593735  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:29:01.595083  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:01.596949  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 208091us
I20260709 12:29:01.602335  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.597936  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.604506  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:01.566079 15886 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:29:01.609961  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.610656  6302 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 26: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:29:01.608316  6274 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.611580  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.613454  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:01.614018  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 429934us
I20260709 12:29:01.586632 16113 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.552217 16189 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:01.574134 16178 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.624596  6383 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:01.617909 16177 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:01.621343  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:01.636811  6349 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:01.610837 15849 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:01.615345 16163 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:01.646131  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.647253  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.653303  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.646021  7329 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 35.
I20260709 12:29:01.650841  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:01.651190  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 139328us
I20260709 12:29:01.619136 16165 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:29:01.658862 16179 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:29:01.660356  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.667444 16185 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:01.623162  6379 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:01.668473 16178 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.669016 15849 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.670276  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.671110  6459 raft_consensus.cc:2393] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 29: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:29:01.671165 15849 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:01.659353 16165 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.672187  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.673790  6308 raft_consensus.cc:1275] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 33 index: 24. (index mismatch)
I20260709 12:29:01.676873 16203 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:01.678603  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.680677  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.681490  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.681594  6379 raft_consensus.cc:606] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:01.681645  6304 raft_consensus.cc:2410] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 34 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 21.
I20260709 12:29:01.679224  6305 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:29:01.682283  6379 raft_consensus.cc:614] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:01.648018 16177 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:01.689908  6304 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:29:01.678519 16139 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:01.640558 15518 consensus_queue.cc:1059] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:29:01.709844  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.672199 16165 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:29:01.735553  6384 raft_consensus.cc:1275] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:29:01.727068  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66414.1us
I20260709 12:29:01.697438 16177 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:01.712129  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61307.5us
W20260709 12:29:01.742847  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76537.4us
I20260709 12:29:01.743474  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:29:01.712422  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 91805.3us
I20260709 12:29:01.744145  6383 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:29:01.691123 16199 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:01.747970  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:01.747390  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.711648 16139 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.738201 15655 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.755170  6300 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 33.
I20260709 12:29:01.692155 16061 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.058s
I20260709 12:29:01.779305  7288 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:01.760006  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:01.772584  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.787202 15655 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:01.796963 16211 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:01.773850 16139 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:01.808686  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.836089  6384 raft_consensus.cc:2410] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 561a4e21d8b344a79af44568ca80a3c5 for term 34 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 21.
I20260709 12:29:01.798425 15655 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:01.848779  6300 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:29:01.850466  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:01.851430  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:01.819628 15997 consensus_queue.cc:1059] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 12:29:01.799542  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:01.807653 16140 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:01.859606 16185 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:01.798075 16211 consensus_queue.cc:1048] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.054s
I20260709 12:29:01.818789 16200 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:01.883033  6310 raft_consensus.cc:2368] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
I20260709 12:29:01.887677  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.888368  6384 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:29:01.894140  6456 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:01.870229 16129 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:01.872787 16140 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.886730 16185 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.904428  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.848244 16109 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.837145 16213 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:29:01.906800  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:01.892063 16163 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:01.908599  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:01.917941  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:01.911618  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 132441us
I20260709 12:29:01.914516  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.905090 16140 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.914845  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:01.901400 16129 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:01.911908  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:01.921974  6376 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Rejecting Update request from peer 561a4e21d8b344a79af44568ca80a3c5 for earlier term 32. Current term is 33. Ops: []
I20260709 12:29:01.922376  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:01.906585 16185 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:01.926415  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:01.916971 15655 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.927109 16191 raft_consensus.cc:3055] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:29:01.927615 16209 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:01.925052 16129 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.931581  6274 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
W20260709 12:29:01.997416  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65907.3us
I20260709 12:29:01.932052  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:01.935787 16189 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:01.938560 16180 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.457s	user 0.005s	sys 0.000s
I20260709 12:29:01.938946  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:01.939124  6351 leader_election.cc:400] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
I20260709 12:29:02.001443  6351 leader_election.cc:403] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:29:02.001730  6377 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:29:01.938723 16163 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.939601 15886 raft_consensus.cc:3055] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:02.010905 15886 raft_consensus.cc:740] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:02.011942 15886 consensus_queue.cc:260] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.013266 15886 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:01.923432 15849 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:01.920631 16203 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:29:01.914721 16134 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:02.014809  6381 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 22.
I20260709 12:29:01.974614 16109 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:01.975097 16211 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:29:01.978006 16209 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.969753 16191 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:01.970245 15729 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:01.980247  6425 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:29:02.052757  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 130519us
W20260709 12:29:02.053272  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73095us
I20260709 12:29:01.993422 16140 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:01.995088 16177 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.002429  6305 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
W20260709 12:29:02.008595  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 88516.5us
I20260709 12:29:02.001032  6382 raft_consensus.cc:606] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:02.012570 16163 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.051116  6458 raft_consensus.cc:2368] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
I20260709 12:29:02.007520 16096 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:02.051908 16113 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:01.971349 15655 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.017215 15998 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:02.052204 16134 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:02.015937 16179 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
I20260709 12:29:02.016765 16203 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:02.084198  6382 raft_consensus.cc:614] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:02.076385 16213 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:02.075291 16147 consensus_queue.cc:1059] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:29:02.079734  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 160866us
I20260709 12:29:02.088929  6274 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:02.051911 16061 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:29:02.088838 16096 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:02.148916  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61742.6us
I20260709 12:29:02.100044  6451 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38272 (request call id 1788) took 1217 ms. Trace:
I20260709 12:29:02.099330 16203 raft_consensus.cc:3060] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:29:02.133450 16096 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:02.100430 16165 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.151695 16213 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:02.165876 16165 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 36 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.093729 16113 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.169165  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:02.170773  7318 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 33: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:02.172891  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.177278 16113 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.178014 16140 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:02.178062  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:02.180384  6305 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:29:02.182628 16216 log.cc:927] Time spent T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:29:02.176877 16191 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.185015 16134 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.185679 16191 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:02.190821 16209 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.154418  6451 rpcz_store.cc:276] 0709 12:29:00.877847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:00.878037 (+   190us) service_pool.cc:224] Handling call
0709 12:29:00.883620 (+  5583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:00.913663 (+ 30043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:00.913703 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:00.952118 (+ 38415us) log.cc:844] Serialized 26 byte log entry
0709 12:29:01.248782 (+296664us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:29:01.248817 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:01.248897 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:01.807219 (+558322us) spinlock_profiling.cc:243] Waited 7.77 ms on lock 0x7b24001ea808. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00409c4 00007f6fa003eacb 00007f6fa01f9dc9 00007f6fa01f9a55 00007f6fa01f909d 00007f6fa2e2b10e 00007f6fa2e4288c 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221
0709 12:29:02.090314 (+283095us) raft_consensus.cc:1685] finished
0709 12:29:02.090343 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:02.095300 (+  4957us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6939,"raft.queue_time_us":59559,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":10952,"spinlock_wait_cycles":17876352,"thread_start_us":265745,"threads_started":1,"wal-append.queue_time_us":332614}
I20260709 12:29:02.167078 16214 consensus_queue.cc:1048] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.041s
I20260709 12:29:02.196713  6305 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 33.
I20260709 12:29:02.103902 15729 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:02.216193  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:02.206694 15499 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:29:02.217823 16222 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:29:02.218083 15499 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.219838 16185 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.222163 16185 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.222795  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.226301  6457 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:29:02.224786  6453 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:02.231230  6451 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:29:02.231308 16096 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.233512 16096 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.218261  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:02.241245  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:02.241665 16199 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:29:02.242067  7329 raft_consensus.cc:2368] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
W20260709 12:29:02.243026 16224 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:29:02.244829 15655 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.246289 16177 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.247136 16230 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:02.247746 16230 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.249996 16230 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.166463 16213 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.251951 16225 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:29:02.253188 16225 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:29:02.253604 16225 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:02.254791 16225 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.258567  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:02.256475 16225 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:29:02.259307  6385 raft_consensus.cc:2393] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:29:02.260111  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:02.260746  6382 raft_consensus.cc:3060] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:29:02.279740 16139 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.282109  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:02.282603 16139 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.282912  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.283437  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:02.284019  6457 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:02.284029 16199 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:29:02.286957  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.287606  6450 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
W20260709 12:29:02.294595  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 192407us
I20260709 12:29:02.300614  6456 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:02.301331  6456 raft_consensus.cc:606] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:02.301716  6457 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 27.
I20260709 12:29:02.301753  6456 raft_consensus.cc:614] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:02.308035 16060 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:29:02.309545  6451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:42278
I20260709 12:29:02.310150  6451 raft_consensus.cc:606] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Received request to transfer leadership
I20260709 12:29:02.310501  6451 raft_consensus.cc:614] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:02.312191  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 30 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:02.312080 15964 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:02.313716  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:02.314360  7318 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:02.315248 15729 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:02.315900  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.316289 15729 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.316941  7331 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:29:02.325008  6382 raft_consensus.cc:2468] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
I20260709 12:29:02.326553  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:02.330826  6272 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:02.334825  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:02.342000  6455 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:02.342736  6455 raft_consensus.cc:606] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:02.343060  6455 raft_consensus.cc:614] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:02.230363 16129 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.359166  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.361646  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:02.362906  7318 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:29:02.363338  6275 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:02.373229  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.338776 16201 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:02.335659 16199 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:02.378351  6303 raft_consensus.cc:2368] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:29:02.379645  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:02.389568  6304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.401980  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:02.378474 16201 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:02.410741  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:02.412251  6301 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:29:02.389297 16199 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:02.381927 15655 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:29:02.410398 16201 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:02.435061  6382 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:02.471148  6382 raft_consensus.cc:606] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:02.471597  6382 raft_consensus.cc:614] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:02.473294  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:29:02.436770 16186 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:29:02.443375 16199 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.478561 16129 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.418387 16096 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:02.492080  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
W20260709 12:29:02.493111  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 91645.9us
I20260709 12:29:02.503394  7314 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:02.416108 16213 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.074s
I20260709 12:29:02.514096 16187 raft_consensus.cc:1064] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:02.524401 16203 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.525803 16204 raft_consensus.cc:3055] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:02.528810  6451 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:29:02.530457  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:02.531934  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:02.509111 16189 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:02.532883 16204 raft_consensus.cc:740] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:02.553459 16204 consensus_queue.cc:260] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.554662 16204 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:02.540622 16225 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:02.540120 16060 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:02.569337  6425 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:02.571067 16189 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:29:02.569208 15974 consensus_queue.cc:1059] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:02.574640  6308 raft_consensus.cc:1275] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:29:02.576467 16165 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:02.577068 16165 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.580838 16165 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.591163  6381 raft_consensus.cc:1240] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:02.592116  6385 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:29:02.599934  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:02.593716 16177 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:02.608521 16177 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.617504  6383 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:29:02.612869  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:02.637506  6378 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:02.618857 16189 consensus_queue.cc:1048] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:29:02.644093  7331 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
W20260709 12:29:02.653077 16235 log.cc:927] Time spent T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
I20260709 12:29:02.657138  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:02.658640  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:02.659801  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:02.659910 16061 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:29:02.660382  6380 raft_consensus.cc:2393] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:02.660619  6382 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 23.
I20260709 12:29:02.661520  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:29:02.662750  6254 auto_rebalancer.cc:893] Removing replace marker failed: Replica 4e0e521cb08c420f981a29576eee90a6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } } pending_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:02.663205  6254 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 12:29:02.663508  6254 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:29:02.668574  6351 leader_election.cc:400] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:29:02.668992  6351 leader_election.cc:403] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:02.669698  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:02.680215  7287 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:02.679345  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:02.689042  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:02.690465  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.578325 16187 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 16, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.691931  6351 leader_election.cc:400] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:29:02.696682  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:02.697736  6303 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:29:02.700814  6348 leader_election.cc:400] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 27. Current term is 28.
I20260709 12:29:02.687484 16162 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625) with higher term. Message: Invalid argument: T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:29:02.703888 16065 raft_consensus.cc:3055] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:02.704332 16065 raft_consensus.cc:740] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:02.705304 15499 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:02.693781 16134 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:02.707684 16134 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:02.708657 16134 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.702468  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:02.712044  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 36 candidate_status { last_received { term: 34 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:02.712706  7331 raft_consensus.cc:3060] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:29:02.718457  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.719746 16212 raft_consensus.cc:2804] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:02.720364 16212 raft_consensus.cc:697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:02.721338 16212 consensus_queue.cc:237] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.722915 16179 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:02.723906 15729 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:02.724871 16162 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.719638  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:02.727768  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:02.694769 16165 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:29:02.751061  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:02.761510  6303 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 36.
I20260709 12:29:02.714416 15886 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.778523  6274 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:02.765576  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.714701 15974 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:02.744285 16162 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.765794 16223 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:29:02.733218 16106 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:29:02.730402 16179 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:02.775418 16109 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:02.792634  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:02.756775 16163 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:02.776095 16209 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:29:02.804682  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:02.732941 16113 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 12:29:02.782140 15849 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:02.836413  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:02.787187 15886 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.821920  6351 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:02.800170 16185 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:29:02.815688 16220 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:29:02.818053 16203 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.897671  6303 raft_consensus.cc:2468] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 33.
I20260709 12:29:02.899338  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:02.839474 16209 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:02.900223  7318 raft_consensus.cc:2393] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 33: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:02.886610 15655 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:02.901389 15655 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.903797 15655 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:02.914956 16203 raft_consensus.cc:2804] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:29:02.915552 16203 raft_consensus.cc:697] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:02.916468 16203 consensus_queue.cc:237] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 34.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.920642  7331 raft_consensus.cc:2468] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 36.
I20260709 12:29:02.923487 16179 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.822146 16163 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:02.960031 16201 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:02.962877  6309 raft_consensus.cc:2368] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:29:02.966631 16162 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.969041 16209 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.970242 16163 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:02.662206  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:02.996320  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.984050  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:02.987486  6451 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:02.989147  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:02.989624  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:02.988654  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.000732  6303 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:02.990388  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:03.002671  6457 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:03.002717  6351 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:03.009282  6351 leader_election.cc:400] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 election: Vote denied by peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395) with higher term. Message: Invalid argument: T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24.
I20260709 12:29:03.011557  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.038156  6309 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: []
W20260709 12:29:02.979771 16246 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.415s	user 0.003s	sys 0.000s
I20260709 12:29:03.041190  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:03.010313  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:02.962445 16201 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:03.054047  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 71794.3us
I20260709 12:29:02.890755 15849 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:02.970458 16209 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:02.986333 16177 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:03.061262  7331 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:03.039448 16256 raft_consensus.cc:2804] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:29:02.990269 15886 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.012616 16162 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:29:02.988916 16163 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.062120 16256 raft_consensus.cc:697] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:03.053130 16109 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:03.071125  6457 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:29:03.066191 16165 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:03.087381 16165 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:03.077600 16211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.012s
I20260709 12:29:03.092058  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:03.108596 16256 consensus_queue.cc:237] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:03.119822 16106 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Committing config change with OpId 21.13: config changed from index 9 to 13, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:03.067407 16163 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.125620  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:03.126754 16187 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:03.072924 16109 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.140945  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:03.141148  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 66502.5us
I20260709 12:29:03.163535  6458 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:29:03.098263 16211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.009s	sys 0.012s
I20260709 12:29:03.150885  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.147864  6275 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:03.125698 16163 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.141161 16109 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:03.147135  6457 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 9 to 13, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:03.166854 15964 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:03.192783  6348 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:03.242755 16163 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:03.167743 16134 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 12:29:03.243528 16163 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.246953  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:03.243891 16134 consensus_queue.cc:1243] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:29:03.249255  6377 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:29:03.254341  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 106609us
W20260709 12:29:03.264065  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 62521.7us
W20260709 12:29:03.260417  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 67768.8us
I20260709 12:29:03.270237  6348 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:03.271037  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.272156  6308 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:29:03.285089  6458 raft_consensus.cc:1240] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 26. Current term is 27. Ops: []
I20260709 12:29:03.286152  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.284188  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:03.253856 15964 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:03.274456 16163 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.310508 16163 consensus_queue.cc:1059] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:03.287623  6455 raft_consensus.cc:2393] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 22: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:29:03.287791 16116 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.016s
I20260709 12:29:03.247910 16065 consensus_queue.cc:260] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.315927 16065 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:03.264257 16199 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:03.272922 16248 log.cc:927] Time spent T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:29:03.320488 16187 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:03.304078  6458 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:29:03.309185 16177 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.325131 16177 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:03.287592 16252 log.cc:927] Time spent T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:29:03.328006 16177 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.333134 16109 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:03.335278  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 51206.2us
I20260709 12:29:03.349030  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.350004  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.315110  6383 raft_consensus.cc:3055] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:29:03.351285  6308 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:29:03.308401 15849 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:03.351251  6383 raft_consensus.cc:740] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:03.353188  6459 raft_consensus.cc:2410] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 32 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:29:03.353977  6383 consensus_queue.cc:260] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.363245  6383 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:03.369486  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.371788  6376 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
W20260709 12:29:03.363835 16265 log.cc:927] Time spent T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:29:03.370364  6310 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:29:03.353128 16177 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:29:03.376111  6458 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:29:03.377866  6381 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:29:03.379866 16264 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:29:03.326043 15964 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.392200 15655 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:03.392753  6310 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:29:03.409792  6305 raft_consensus.cc:1240] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:29:03.410394  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.428994  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:29:03.425513  6383 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:29:03.369627 16186 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:03.436748 16186 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.378620 16177 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:03.438985 16177 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:29:03.439339 16186 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.440649 15655 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.423611 16256 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:03.444593 16185 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:03.447561 15655 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.449505 16274 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:03.451051 16129 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:29:03.430349  6383 pending_rounds.cc:85] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 16
I20260709 12:29:03.431938 16109 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:03.433012 16197 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:29:03.455307  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.456164  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.457544  6383 pending_rounds.cc:107] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 26.17
I20260709 12:29:03.458434  6383 raft_consensus.cc:2889] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:03.479516  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:29:03.481146 16179 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:29:03.485400  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:03.487005  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:03.487484  7317 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:29:03.487818  6378 raft_consensus.cc:2393] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 24: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:03.480015 16256 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:29:03.496945  6459 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 24.
I20260709 12:29:03.502214  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:03.503237  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:03.522763 16109 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.001s
I20260709 12:29:03.528158 16109 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.528760 16109 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.529682  7329 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:29:03.531736  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:03.546689  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:03.526779 16203 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:29:03.533208 16147 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.535673 16060 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.551774 15729 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:03.532388 16187 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
I20260709 12:29:03.538602 16162 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.042s
I20260709 12:29:03.573738 16147 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.576041 16060 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.667569 16267 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:03.668130 16267 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:29:03.668308  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:03.668654 16147 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.583657 15974 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:29:03.668475 16267 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:03.724016 16060 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.707875 16109 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.588903 16096 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 14 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:03.715934 16147 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.735458 16147 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:03.743196  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 161440us
I20260709 12:29:03.738196 16147 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.763309  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.764956  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.765724  6307 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:29:03.767923  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.771596  6310 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:29:03.766647 16259 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:03.786527  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:03.767195 16177 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.797075  6425 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:03.799149 16060 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:29:03.802032 16060 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.802398 16060 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:03.803277 16267 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.805338 16267 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.812816  6459 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:03.797706 16256 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.818367  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:03.819172  6308 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:03.842018  6308 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:29:03.831543 16243 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:29:03.851843 16243 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.852305 16243 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:03.847622 16177 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.855861  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:03.863600  6310 raft_consensus.cc:1240] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 23. Current term is 24. Ops: []
I20260709 12:29:03.864130  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.864897  6307 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:29:03.831945 16256 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.873533 16256 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.885720 16234 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:03.886369 16234 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:03.888572 16234 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:03.890573 16060 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.891235  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:03.900395  6351 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:03.900326 16179 consensus_queue.cc:1059] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:03.901455 16179 consensus_queue.cc:1243] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 4268023ecea149cd83cba77db53c3100 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:29:03.901633 16109 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:03.902005 16259 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:03.902084 16109 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.902621 16259 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:03.902779 16109 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:03.903559 16259 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.904482 16177 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:29:03.904834 16259 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:03.900900  6307 raft_consensus.cc:1275] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 34 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:29:03.904886 16177 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.901439  6308 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:29:03.908253 16177 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:03.917119  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:03.925959  6376 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 14 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:03.940021  6302 raft_consensus.cc:1275] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:29:03.941284  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:03.941556 16060 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.943441  6377 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:03.945851  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:03.947464  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.949021  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:03.951203  6305 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 22.
I20260709 12:29:03.958837  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.959614  6459 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:29:03.960251  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:03.950258  6455 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:29:03.961043  6383 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 22.
I20260709 12:29:03.960625  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:03.969475  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:03.971333  6451 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:29:03.975042  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:03.966301 15849 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:03.984417 16203 consensus_queue.cc:1048] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:29:03.990739 16162 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:03.991405 16162 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:03.991840 16162 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:03.935240 16243 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:03.995743 16243 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:03.996838 15729 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.997391 15729 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.000190 15729 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.001600 16230 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:04.005986  6458 raft_consensus.cc:2368] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
W20260709 12:29:04.018671 16275 log.cc:927] Time spent T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.276s	user 0.000s	sys 0.004s
I20260709 12:29:04.006075 16225 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:04.022080 16225 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.025041 16225 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.035346  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:04.041601  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.050061  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.044456 16267 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:29:04.057056  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:29:04.061018  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 85055.9us
I20260709 12:29:04.065219  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:04.065862  6381 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:29:04.067248  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 66838us
I20260709 12:29:04.064584  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:04.079950  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:29:04.082077  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78903.9us
W20260709 12:29:04.083807  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83684us
I20260709 12:29:04.086221  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.092422  6305 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:29:04.084582 16267 consensus_queue.cc:1243] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:29:04.090404 16280 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.356s	user 0.004s	sys 0.000s
I20260709 12:29:04.075927 16256 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:29:04.079087 16264 consensus_queue.cc:1048] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.022s
I20260709 12:29:04.209560  6381 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
W20260709 12:29:04.210178 16288 log.cc:927] Time spent T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.271s	user 0.002s	sys 0.000s
W20260709 12:29:04.219867 16286 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
W20260709 12:29:04.221273 16289 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:29:04.197186 16099 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.224174 16099 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.221834 16162 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:04.197752 16280 log.cc:927] Time spent T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:29:04.214052 16134 raft_consensus.cc:2764] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader election decision vote started in defunct term 22: won
I20260709 12:29:04.118713 16203 consensus_queue.cc:1059] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.099s
I20260709 12:29:04.243449  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.209460 15655 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.252180  6379 raft_consensus.cc:2393] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 25: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:04.240356  6424 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:04.216203 16188 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.265229  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:04.260985  6381 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:29:04.266093  6381 pending_rounds.cc:85] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 14
I20260709 12:29:04.266407  6381 pending_rounds.cc:107] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:29:04.266745  6381 raft_consensus.cc:2889] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:04.271159  6459 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:04.247109 16291 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.240928 16292 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.283640 16264 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.003s
I20260709 12:29:04.276547 16189 raft_consensus.cc:2804] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:29:04.300149 16189 raft_consensus.cc:697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:04.301401 16189 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.328393  6308 raft_consensus.cc:1275] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:04.333680  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:04.335093  6272 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:29:04.344990  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 238439us
I20260709 12:29:04.283917 16147 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:04.296857 16099 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.263979 15655 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.344408 15729 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:29:04.278201 16188 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.344708 16294 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.320032 16177 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.296365 16291 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.320346 16292 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.372116 15655 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.381834 16162 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.362612 16147 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:04.386834  6378 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 32: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:04.365633 15729 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.385576  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:04.393990  6457 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:04.388551  7317 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 9 to 13, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:04.391824 16188 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.360265 16099 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:29:04.398416 16147 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:04.392417 16284 log.cc:927] Time spent T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.672s	user 0.009s	sys 0.007s
I20260709 12:29:04.401007 16264 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:04.401626 16264 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.401991  6380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51842 (request call id 4036) took 1198 ms. Trace:
I20260709 12:29:04.383246 16232 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.402379  6380 rpcz_store.cc:276] 0709 12:29:03.203120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:03.203826 (+   706us) service_pool.cc:224] Handling call
0709 12:29:03.204355 (+   529us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:29:03.245234 (+ 40879us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:03.245271 (+    37us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:29:03.455415 (+210144us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b1800439ea0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01882a6 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a
0709 12:29:03.455810 (+   395us) spinlock_profiling.cc:243] Waited 14.1 ms on lock 0x7b2c0005c2d8. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa1ba0a11 00007f6fa1bb0372 00007f6fa32e2a01 00007f6fa32a743a 00007f6fa32a683e 00007f6fa826b419 00007f6fa17e2bd4 00007f6fa17e2b62 00007f6fa17e2a91 00007f6fa17e2a0c 00007f6fa17e1cc2
0709 12:29:03.532809 (+ 76999us) log.cc:844] Serialized 592 byte log entry
0709 12:29:03.742920 (+210111us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:29:03.765787 (+ 22867us) log.cc:844] Serialized 14 byte log entry
0709 12:29:03.765992 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:03.766057 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:03.852075 (+ 86018us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b1800439ea0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:29:04.375250 (+523175us) spinlock_profiling.cc:243] Waited 53.2 ms on lock 0x7b4400022b4c. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa00e0c85 00007f6fa00e16a2 00007f6fa2e531db 00007f6fa2e4eaaf 00007f6fa2e4286e 00007f6fa2e41bb6 00007f6fa2e417d8 00007f6fa2e4b221 00007f6fa2e4b1d9 00007f6fa2e4b169
0709 12:29:04.401740 (+ 26490us) raft_consensus.cc:1685] finished
0709 12:29:04.401762 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:04.401900 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":23747,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":14182,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":8647,"spinlock_wait_cycles":167211392,"thread_start_us":177768,"threads_started":1,"wal-append.queue_time_us":172061}
I20260709 12:29:04.408062 16232 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.408910 16177 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.394946 15729 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:04.414523  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 31 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:04.415318  6457 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:04.417517 16291 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.369415 16294 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.395679 16298 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:04.418557  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:04.419691  6451 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 32: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
W20260709 12:29:04.387492 16281 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.618s	user 0.008s	sys 0.007s
I20260709 12:29:04.418455 16298 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.428432 16264 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.429860 16298 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.419730 16294 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:04.423349  6348 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:04.423789  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.433097  6457 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
W20260709 12:29:04.433203  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 92143.3us
I20260709 12:29:04.434226  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.434406  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:04.424306  7329 raft_consensus.cc:2410] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 34 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 21.
W20260709 12:29:04.426177 16109 consensus_meta.cc:287] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.523s	user 0.010s	sys 0.002s
I20260709 12:29:04.426294 16292 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.435056  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:04.448021  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 96016.2us
I20260709 12:29:04.453191  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.455225  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.443025 16291 raft_consensus.cc:2804] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:04.433897 16262 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.444599 16264 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:04.470562 16291 raft_consensus.cc:697] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:04.481729 16291 consensus_queue.cc:237] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.448951 16188 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:29:04.444693 16109 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.474952 16264 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.476208 16262 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.501562 15974 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.526250 15974 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.528702 15974 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.530665 16262 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.510993 16264 raft_consensus.cc:3060] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:29:04.536049  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68280.3us
I20260709 12:29:04.546888  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.548239 16213 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.548058 16264 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.548933 16213 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.549832  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:04.550441 16264 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.548259 16109 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.550437  6451 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:04.553404 16213 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.560411  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:04.563619  6377 raft_consensus.cc:1240] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:29:04.565306  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.568060  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.573302 16109 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.573971 16109 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.581084 16109 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.583688  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.584451  6454 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:29:04.586069  6348 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:29:04.594651  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 95182.2us
I20260709 12:29:04.561998 15729 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:04.606578  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 68697.2us
I20260709 12:29:04.607220  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:04.608302  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:04.610050  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 260994us
I20260709 12:29:04.611289  7331 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:29:04.611995  6457 raft_consensus.cc:2468] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:29:04.615759  6348 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:04.616164  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:04.617014  6451 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:04.628487  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:04.619071  6348 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:04.613180  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:04.603223 16177 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:04.643927  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.645423  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.646557  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.634297 16109 raft_consensus.cc:2804] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:04.614809 16231 consensus_queue.cc:1059] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:29:04.659135 16109 raft_consensus.cc:697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:04.656409  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:04.660142 16109 consensus_queue.cc:237] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.661149  6301 raft_consensus.cc:3060] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:29:04.663432 15729 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.658443 16267 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:04.668337 16267 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.641469  6425 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:04.635087 16140 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:04.671306 16231 consensus_queue.cc:1243] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:29:04.655601  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:04.672147  6303 raft_consensus.cc:3060] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:04.657501 16315 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.675273 16315 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.677397 16315 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.677930 16315 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:29:04.636677 16264 raft_consensus.cc:2804] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:04.679750  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:04.680202 16264 raft_consensus.cc:697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:04.681481 16264 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.642084 16213 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:29:04.693674  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:29:04.694350  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 52663us
I20260709 12:29:04.697043  6301 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:29:04.702744  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.678292 16315 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.726112  6348 leader_election.cc:400] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 22 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 22. Current term is 23.
I20260709 12:29:04.639015 16177 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.735472  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:04.737653  6303 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 32.
I20260709 12:29:04.729081 16315 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:04.753055  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:04.753773  7331 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 30: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:04.755971  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.734640 16177 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:04.759104  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.762018  7318 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 22. (term mismatch)
I20260709 12:29:04.763129  7318 pending_rounds.cc:85] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 21
I20260709 12:29:04.763396  7318 pending_rounds.cc:107] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted NO_OP operation due to leader change: 32.22
I20260709 12:29:04.763681  7318 raft_consensus.cc:2889] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:04.764526  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.760792  7285 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:04.765667  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:04.771163  6450 raft_consensus.cc:2368] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:29:04.771727  6452 raft_consensus.cc:2368] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:29:04.760814  6455 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 14 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:04.742527 16189 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:04.780292 16225 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:04.803277 16267 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.810182  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:29:04.822368  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 48682us
W20260709 12:29:04.822796  6275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 62999.4us
I20260709 12:29:04.824985  6377 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:29:04.826318  6377 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:04.827239  6377 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.783221 16293 consensus_queue.cc:1048] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:29:04.828539  6377 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:04.842720 16134 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:29:04.847385 16232 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:04.855746 16314 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.856459 16314 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.861326  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:04.865979 15997 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:04.866742  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:04.870873  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:04.878091  6377 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 18. (term mismatch)
I20260709 12:29:04.881028  6377 pending_rounds.cc:85] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 17
I20260709 12:29:04.881995  6377 pending_rounds.cc:107] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 29.18
I20260709 12:29:04.882294  6377 raft_consensus.cc:2889] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:04.883677  6275 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:04.884922 16113 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:04.886332 16065 raft_consensus.cc:3055] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:04.886785 16065 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:04.887691 16065 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:04.888947 16065 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:04.881078 15997 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:04.892726 16298 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:04.893288 16298 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:04.894750 16256 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 30 current term: 30 is higher.
I20260709 12:29:04.895354  6453 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:29:04.896176 16298 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:04.902835 16256 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:04.890003 16177 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.932109  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:04.932461  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.932869  6301 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:29:04.917485 15974 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:29:04.935748  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:04.923235 16201 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.940526  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:04.940876  6272 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:04.926014 15729 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.946548 16201 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.953748 16315 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.976375 16315 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.955307 16177 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.957858  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:04.986917  6457 raft_consensus.cc:2368] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
W20260709 12:29:04.987017  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46240.5us
I20260709 12:29:04.965214 16302 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:04.962751 15964 raft_consensus.cc:493] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.972028 15729 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:04.974433 16201 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.953830  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:04.992384  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:04.995930  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:04.996903  6348 leader_election.cc:400] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:29:05.000623  6377 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:05.000739  6348 leader_election.cc:403] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Cancelling election due to peer responding with higher term
I20260709 12:29:04.999987  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:05.001611  6309 raft_consensus.cc:3060] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:29:05.015280  6272 leader_election.cc:400] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:29:05.015692  6272 leader_election.cc:403] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:29:05.017336  6310 raft_consensus.cc:1240] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:29:05.019567  6272 leader_election.cc:400] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:29:05.019958  6272 leader_election.cc:403] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:05.020488  6425 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:05.022066  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:05.002995 16302 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:05.027828  6377 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:29:05.039841  6310 raft_consensus.cc:2393] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 32: Already voted for candidate 4e0e521cb08c420f981a29576eee90a6 in this term.
I20260709 12:29:04.997566 15655 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:05.040879  6424 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:05.021636 16129 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:05.048439  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.022611 16201 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:05.058974  6310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.039570 16134 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election lost for term 30. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 30. Current term is 31.
I20260709 12:29:05.062273  6302 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:29:05.042999 16302 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:04.999833 15964 raft_consensus.cc:515] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:05.042737 15499 raft_consensus.cc:2804] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:29:05.085125  6351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:29:05.093731 16324 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:29:05.095010  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:05.096077  7317 raft_consensus.cc:2393] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 30: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:05.067775 16204 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:29:05.119484  6302 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:05.121997  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:05.121910  6309 raft_consensus.cc:2468] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:05.114720 15499 raft_consensus.cc:697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
W20260709 12:29:05.129508  6423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:05.124217 16185 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:29:05.119771 16188 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.134267  6309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:05.156651  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:05.158020  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:05.158404  6310 raft_consensus.cc:1275] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:29:05.162195 16109 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:05.127894 16204 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:05.166199  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:05.179251 15729 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:05.160874 16177 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:29:05.152016 16314 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:05.190289  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 148092us
I20260709 12:29:05.160944 16188 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:05.210919 15964 leader_election.cc:290] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:05.201911 16129 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) with higher term. Message: Invalid argument: T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 24. Current term is 25.
I20260709 12:29:05.202543 16204 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:05.223376  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 177370us
I20260709 12:29:05.228848  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:05.233536  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:05.218253 15499 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:05.248842  6309 raft_consensus.cc:2393] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 28: Already voted for candidate 4268023ecea149cd83cba77db53c3100 in this term.
I20260709 12:29:05.252475  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:05.256203  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.258733  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.266706  6304 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:05.268059 16188 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:05.277549  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 52789.3us
I20260709 12:29:05.280437  6300 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:29:05.285986  6302 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:05.293569  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:05.313388  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48363.3us
I20260709 12:29:05.315249 16301 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:05.332552  6425 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:05.334646  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:05.338780  6458 raft_consensus.cc:2410] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:29:05.345261 16065 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
W20260709 12:29:05.350829  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 310022us
I20260709 12:29:05.353533  6377 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:29:05.343838  6425 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:05.412074  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 66360.6us
I20260709 12:29:05.343575 16340 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:05.420943 16343 log.cc:927] Time spent T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:29:05.350752 16060 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.070s
I20260709 12:29:05.435267 16282 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
W20260709 12:29:05.459563  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 115819us
I20260709 12:29:05.461278 16065 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:05.470986 16347 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:05.477898 16347 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:05.480180 16347 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:05.476277 16339 log.cc:927] Time spent T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.331s	user 0.003s	sys 0.013s
W20260709 12:29:05.478354  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 300750us
I20260709 12:29:05.502948  6457 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:29:05.512496 16177 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:05.513252  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:05.503250  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.503954  6455 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:29:05.518592 16177 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:29:05.520973  6384 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Committing config change with OpId 21.13: config changed from index 9 to 13, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:05.531253 16340 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:05.534617 16340 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:05.552973  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 790649us
I20260709 12:29:05.563122  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.563959  7329 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 27.
I20260709 12:29:05.565907  6274 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:05.570943  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.575101  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:05.575883  7318 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:05.583168 16340 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:05.586452  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.586783  7318 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 31.
I20260709 12:29:05.608726  6305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:05.589993 16188 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:05.572288 16226 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:05.615072  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.620364  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.650580  6384 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:29:05.631446 16134 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.615613 16226 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:05.660913  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.663460  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ddf475f266c47adb1a50aa986a0c44c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.664058  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:05.665243 15974 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:29:05.667618 16225 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:29:05.668103 16225 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:05.668473 16225 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:29:05.681133  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:05.692579  6305 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:29:05.714010 16264 consensus_queue.cc:1048] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:05.714635 16340 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:05.694478  7318 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 14 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:05.676800 16134 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:05.699414  7288 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:05.743299 16225 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:05.746165 16225 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:05.754614  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 88421.5us
I20260709 12:29:05.730315 16303 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:05.727336  7288 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:05.796825  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:05.803351  6307 raft_consensus.cc:1275] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:29:05.818637  6308 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:29:05.854133 16341 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:05.884249  6378 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:29:05.879333 16292 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:05.887784 16341 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:05.913657 16341 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:05.897233 16340 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.034s
W20260709 12:29:06.011139  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 83867.7us
W20260709 12:29:06.042999  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 92916us
I20260709 12:29:06.051023  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:06.052430 16364 log.cc:927] Time spent T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:29:06.057565 16134 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:06.075412 16342 consensus_queue.cc:1048] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:29:06.077423 16226 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:06.080950  7318 raft_consensus.cc:1240] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 33. Current term is 34. Ops: []
I20260709 12:29:06.087077  6308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:06.091912  6198 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 19 to 21, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 21 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:06.093713  6198 catalog_manager.cc:5697] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:05.002765  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:06.096262  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:06.097046  6453 raft_consensus.cc:2410] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:29:06.101610 16342 consensus_queue.cc:1059] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:29:06.102649 16360 raft_consensus.cc:3055] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:29:06.103051 16360 raft_consensus.cc:740] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:06.103857 16360 consensus_queue.cc:260] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:06.104697  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:06.106366 16256 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:29:06.111526  6198 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 31 to 34, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 34 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.112934  6348 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:06.115139 16256 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:06.119976  6200 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 300) took 99463 ms (client timeout 15000 ms). Trace:
W20260709 12:29:06.121009  6197 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 270) took 21692 ms (client timeout 15000 ms). Trace:
W20260709 12:29:06.121485  6197 rpcz_store.cc:269] 0709 12:28:44.428517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.691247 (+10262730us) service_pool.cc:224] Handling call
0709 12:29:06.120816 (+11429569us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:59.801349 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:59.801952 (+   603us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:59.801998 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:28:59.802018 (+    20us) tablet.cc:662] Decoding operations
0709 12:28:59.803960 (+  1942us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:59.804034 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:59.804061 (+    27us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:28:59.900420 (+ 96359us) tablet.cc:701] Row locks acquired
0709 12:28:59.900453 (+    33us) write_op.cc:260] PREPARE: finished
0709 12:28:59.900692 (+   239us) write_op.cc:270] Start()
0709 12:28:59.900885 (+   193us) write_op.cc:276] Timestamp: P: 1783600139900617 usec, L: 0
0709 12:28:59.900922 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:28:59.978903 (+ 77981us) log.cc:844] Serialized 9889 byte log entry
0709 12:29:00.421796 (+442893us) op_driver.cc:464] REPLICATION: finished
0709 12:29:00.540125 (+118329us) write_op.cc:301] APPLY: starting
0709 12:29:00.540304 (+   179us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:00.779601 (+239297us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:00.779758 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:00.836712 (+ 56954us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:00.867166 (+ 30454us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=0,mrs_lookups=0
0709 12:29:00.880175 (+ 13009us) write_op.cc:312] APPLY: finished
0709 12:29:00.899162 (+ 18987us) log.cc:844] Serialized 220 byte log entry
0709 12:29:00.899394 (+   232us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:00.899904 (+   510us) write_op.cc:454] Released schema lock
0709 12:29:00.900002 (+    98us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:00.900591 (+   589us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:00.900628 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":130247,"reported_tablets":50,"child_traces":[["op",{"apply.queue_time_us":117893,"apply.run_cpu_time_us":44476,"apply.run_wall_time_us":360459,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":209580,"prepare.run_cpu_time_us":11074,"prepare.run_wall_time_us":215067,"raft.queue_time_us":324591,"raft.run_cpu_time_us":9207,"raft.run_wall_time_us":110975,"replication_time_us":520453,"thread_start_us":480926,"threads_started":4,"wal-append.queue_time_us":90966,"wal-append.run_cpu_time_us":20358,"wal-append.run_wall_time_us":1837146}]]}
W20260709 12:29:06.136443  6200 rpcz_store.cc:269] 0709 12:27:26.673829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.683169 (+  9340us) service_pool.cc:224] Handling call
0709 12:29:06.119767 (+99436598us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2070886,"reported_tablets":50}
I20260709 12:29:06.137708  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:06.138785  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 72914.5us
I20260709 12:29:06.147091  6198 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 24 to 22, leader changed from <none> to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 22 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:06.154783 16360 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Advancing to term 34
W20260709 12:29:06.158638  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 49015us
I20260709 12:29:06.158638 16186 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:06.161973 16186 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:06.172873  6198 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.174872 16186 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:06.188628  6198 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 23 to 28. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:29:06.199393  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 65538us
W20260709 12:29:06.217988  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 490683us
I20260709 12:29:06.196915  6198 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 15 to 22, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 22 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:06.233134  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 188949us
I20260709 12:29:06.235589  6198 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 18 to 22, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 22 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:06.241257  7287 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:06.243156  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.249068 16366 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:06.250499 16340 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:06.258314  6198 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 17 to 21. New cstate: current_term: 21 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.259994  6275 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.261163  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.263111  6198 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 27 to 28, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 28 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.265811 16313 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:06.269817  6275 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.270529 16188 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:06.271653 16188 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:06.272989  6275 leader_election.cc:304] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.274226 16106 raft_consensus.cc:2749] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:06.265864  6198 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 19 to 25, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 25 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:06.277623  6275 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.247459 16186 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:06.306360  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:06.312033  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:06.313705  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:06.292526 16341 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:06.295425 16226 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:06.326268  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:06.345073  7373 heartbeater.cc:646] Failed to heartbeat to 127.5.4.126:35813 (9 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.5.4.126:35813 timed out after 15.000s (SENT)
I20260709 12:29:06.347318  7331 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:29:06.352449  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:06.299657 16186 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:06.356833  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:06.358685  6275 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:29:06.365497  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 50102.1us
I20260709 12:29:06.367686  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:06.368470  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:06.369153  6453 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:29:06.371906  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:06.360270 16106 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:29:06.374022  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 58933.1us
I20260709 12:29:06.380836  6379 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:06.381247  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.382251 16313 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:06.382669 16313 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:06.383062 16313 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:06.357148 16186 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:06.476509  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 102709us
I20260709 12:29:06.392009  7288 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:06.480996  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 92942.1us
W20260709 12:29:06.484836  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 92918.1us
I20260709 12:29:06.488166  6453 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:06.488818  6453 raft_consensus.cc:606] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:06.489174  6453 raft_consensus.cc:614] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:06.391571 16220 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:06.503572 16373 log.cc:927] Time spent T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:29:06.493673 16367 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:29:06.510006 16314 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:06.352524 16226 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:06.510548 16314 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:06.557894 16314 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:06.559108  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:06.548189 16372 log.cc:927] Time spent T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:29:06.577181  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:06.577826  6377 raft_consensus.cc:606] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:06.578171  6377 raft_consensus.cc:614] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:29:06.589321  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 110928us
W20260709 12:29:06.589689  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 97684.8us
I20260709 12:29:06.624429 16226 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:06.626353  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:06.631899 16313 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:06.634171 16313 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:06.687768  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:06.691572  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:29:06.692970  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 45028.5us
I20260709 12:29:06.688478  6425 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.696291  6377 raft_consensus.cc:2393] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 26: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:06.696081  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:06.706099 16370 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:06.707242  6458 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:06.707854  6458 raft_consensus.cc:606] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:06.708217  6458 raft_consensus.cc:614] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:06.712296  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:06.712733 16147 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.015s
W20260709 12:29:06.756214  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46711.9us
I20260709 12:29:06.766254  6457 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 26.
I20260709 12:29:06.793272  6457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:06.794018  6457 raft_consensus.cc:606] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:06.808139  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:06.811040 16383 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:29:06.830608  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:06.833181 16317 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:06.834893  6272 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:06.846776 16304 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:06.847375 16304 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:06.848277 16304 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:06.904723 15499 raft_consensus.cc:993] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:06.906845 15974 raft_consensus.cc:1081] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:06.913975  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:06.914628  6384 raft_consensus.cc:606] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:06.914991  6384 raft_consensus.cc:614] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:06.942286  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:29:06.944350  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:06.945231  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:06.919708  6377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:29:06.951185  6377 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:06.951632  6377 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:06.963642  6377 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:06.971549  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:06.972298  6303 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:06.979910  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:06.980636  6453 raft_consensus.cc:3055] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:06.980994  6453 raft_consensus.cc:740] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:06.971921  6377 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:06.981673  6453 consensus_queue.cc:260] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:06.999606  6453 raft_consensus.cc:3060] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:06.993877  6303 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:29:07.005066  6351 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:07.006008 16342 raft_consensus.cc:2804] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:07.006565 16342 raft_consensus.cc:697] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:07.007521 16342 consensus_queue.cc:237] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:07.028110  6198 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 303) took 25115 ms (client timeout 15000 ms). Trace:
W20260709 12:29:07.035908  6206 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 147) took 55948 ms (client timeout 15000 ms). Trace:
W20260709 12:29:07.036360  6206 rpcz_store.cc:269] 0709 12:28:11.133891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:13.971708 (+2837817us) service_pool.cc:224] Handling call
0709 12:29:07.035675 (+53063967us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25698,"reported_tablets":31}
W20260709 12:29:07.028635  6198 rpcz_store.cc:269] 0709 12:28:41.912338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:50.622074 (+8709736us) service_pool.cc:224] Handling call
0709 12:28:51.256443 (+634369us) spinlock_profiling.cc:243] Waited 9.85 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0183640 00007f6fa905a846 00007f6fa916490e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:29:07.027988 (+15771545us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:06.478321 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:06.537022 (+ 58701us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:06.546313 (+  9291us) write_op.cc:435] Acquired schema lock
0709 12:29:06.549680 (+  3367us) tablet.cc:662] Decoding operations
0709 12:29:06.586990 (+ 37310us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:06.587127 (+   137us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:06.587164 (+    37us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:29:06.612934 (+ 25770us) tablet.cc:701] Row locks acquired
0709 12:29:06.623654 (+ 10720us) write_op.cc:260] PREPARE: finished
0709 12:29:06.623828 (+   174us) write_op.cc:270] Start()
0709 12:29:06.623993 (+   165us) write_op.cc:276] Timestamp: P: 1783600146623761 usec, L: 0
0709 12:29:06.624019 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:29:06.624779 (+   760us) log.cc:844] Serialized 4344 byte log entry
0709 12:29:06.767854 (+143075us) op_driver.cc:464] REPLICATION: finished
0709 12:29:06.787650 (+ 19796us) write_op.cc:301] APPLY: starting
0709 12:29:06.787752 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:06.801009 (+ 13257us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:06.801160 (+   151us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:06.807314 (+  6154us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:06.807696 (+   382us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:29:06.807743 (+    47us) write_op.cc:312] APPLY: finished
0709 12:29:06.808099 (+   356us) log.cc:844] Serialized 105 byte log entry
0709 12:29:06.808299 (+   200us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:06.808475 (+   176us) write_op.cc:454] Released schema lock
0709 12:29:06.808545 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:06.809060 (+   515us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:06.809091 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":92476,"reported_tablets":50,"spinlock_wait_cycles":22644096,"child_traces":[["op",{"apply.queue_time_us":2614,"apply.run_cpu_time_us":14615,"apply.run_wall_time_us":21940,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50129,"num_ops":11,"prepare.queue_time_us":146096,"prepare.run_cpu_time_us":15570,"prepare.run_wall_time_us":217028,"raft.queue_time_us":60673,"raft.run_cpu_time_us":2688,"raft.run_wall_time_us":2687,"replication_time_us":143672,"spinlock_wait_cycles":1408,"thread_start_us":271569,"threads_started":4,"wal-append.queue_time_us":79502}]]}
I20260709 12:29:07.059516  6453 raft_consensus.cc:2468] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 29.
I20260709 12:29:07.391897  6200 catalog_manager.cc:5697] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 15 to 25, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.393877  6200 catalog_manager.cc:5697] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.395710  6200 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:07.400851  6200 catalog_manager.cc:5697] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 25 to 27, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 27 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.402855  6200 catalog_manager.cc:5697] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 20 to 25, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 25 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.404313  6200 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 29, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 29 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:07.405948  6200 catalog_manager.cc:5697] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 30, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 30 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:07.425280  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 85168us
I20260709 12:29:07.428421  6379 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:29:07.432021 16346 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:07.468924 16387 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:29:07.484364 16387 consensus_queue.cc:1243] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Peer 0bdd5f9c03b84f7289e750ebaab2fda6 log is divergent from this leader: its last log entry 24.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:29:07.508801  6200 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 30, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 30 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.465662 16346 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:07.511276  6200 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 18 to 28. New cstate: current_term: 28 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.513336  6200 catalog_manager.cc:5697] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.564574  6453 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:29:07.566102  6200 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 27, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.568150  6200 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 26 to 28, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 28 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:07.578677  6199 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 272) took 139525 ms (client timeout 15000 ms). Trace:
W20260709 12:29:07.579371  6199 rpcz_store.cc:269] 0709 12:26:48.053496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.053841 (+   345us) service_pool.cc:224] Handling call
0709 12:26:48.984572 (+930731us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa01830b2 00007f6fa90fc759 00007f6fa9094a76 00007f6fa9065d58 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc
0709 12:26:49.268100 (+283528us) spinlock_profiling.cc:243] Waited 72.1 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182e9c 00007f6fa01830f6 00007f6fa90fc759 00007f6fa9094a76 00007f6fa9065d58 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc
0709 12:26:49.275016 (+  6916us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:29:07.578537 (+138303521us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35940,"reported_tablets":49,"spinlock_wait_cycles":311623424}
I20260709 12:29:07.583459 16346 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:07.595082  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100" is_pre_election: true
I20260709 12:29:07.595950  6307 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 33.
I20260709 12:29:07.596803  6200 catalog_manager.cc:5697] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 27 to 31, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 31 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.599225  6200 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 25, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.604435  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:07.601332  6200 catalog_manager.cc:5697] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 33, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 33 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.617343  6200 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 26 to 27, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.622308  6200 catalog_manager.cc:5697] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.637426  6200 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 25, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 25 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:07.646113  6200 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:07.653687  6379 pending_rounds.cc:85] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting all ops after (but not including) 12
W20260709 12:29:07.655414  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 219155us
I20260709 12:29:07.657085  6379 pending_rounds.cc:107] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:29:07.657416  6379 raft_consensus.cc:2889] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:07.651201  6200 catalog_manager.cc:5697] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 26 to 29, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 29 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:07.664515  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 193469us
W20260709 12:29:07.651304  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 193480us
I20260709 12:29:07.637598 16346 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:29:07.613256 16317 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:07.668957  6200 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 27. New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.677485  6453 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:29:07.677516 16129 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:29:07.675110 16317 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:07.685317 16317 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:07.670926 16346 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:07.700464 16346 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:29:07.702003 16291 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:07.707109  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:07.707852  7317 raft_consensus.cc:2393] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 in current term 34: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:07.674641  6200 catalog_manager.cc:5697] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 24 to 31, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 31 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.725502  6200 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 27 to 28, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 28 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.726908  6200 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 31, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 31 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:07.733907  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 61984.9us
I20260709 12:29:07.754623  6307 raft_consensus.cc:1275] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:29:07.757513 16401 consensus_queue.cc:1048] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:29:07.812435  6192 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0bdd5f9c03b84f7289e750ebaab2fda6 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:07.819849  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 151721us
W20260709 12:29:07.808369  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 140673us
I20260709 12:29:07.732827  6200 catalog_manager.cc:5697] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 24, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 24 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.833952  6200 catalog_manager.cc:5697] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.857329  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:07.807572 16346 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:07.899549 16346 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:07.901984  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:07.902688  7331 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 34.
I20260709 12:29:07.914436  6379 raft_consensus.cc:2368] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35.
I20260709 12:29:07.919654  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:07.920449  6301 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:29:07.854907  6200 catalog_manager.cc:5697] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 21 to 25, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.937150  6200 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:07.938915  6200 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 27, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:07.940387  6200 catalog_manager.cc:5697] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 35. New cstate: current_term: 35 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:07.954905  6274 leader_election.cc:400] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35.
I20260709 12:29:07.955466  6274 leader_election.cc:403] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:07.941625  6200 catalog_manager.cc:5697] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 23, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.957165  6200 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 28. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:07.963711  6301 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 35.
I20260709 12:29:07.972452  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 35 candidate_status { last_received { term: 33 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:07.973152  7314 raft_consensus.cc:3060] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:29:07.971277  6200 catalog_manager.cc:5697] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.975593  6200 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 26 to 28, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 28 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:07.974236  6351 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:07.977368  6200 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:07.972792 16317 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641) with higher term. Message: Invalid argument: T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4268023ecea149cd83cba77db53c3100 for earlier term 34. Current term is 35.
I20260709 12:29:07.990396 16342 raft_consensus.cc:2804] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:29:07.999305 16342 raft_consensus.cc:697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:08.000211 16342 consensus_queue.cc:237] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:08.039309  7314 raft_consensus.cc:2468] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 35.
I20260709 12:29:08.093724  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:08.369642  6301 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:46152
I20260709 12:29:08.370333  6301 raft_consensus.cc:606] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:08.370711  6301 raft_consensus.cc:614] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:08.374037  6384 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:08.375623  6384 raft_consensus.cc:606] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:08.375969  6384 raft_consensus.cc:614] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:08.380715  6377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
mode: GRACEFUL
new_leader_uuid: "561a4e21d8b344a79af44568ca80a3c5"
 from {username='slave'} at 127.0.0.1:58396
I20260709 12:29:08.381294  6377 raft_consensus.cc:606] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Received request to transfer leadership to TS 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:08.381644  6377 raft_consensus.cc:614] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:08.384979  6457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:08.385124  7314 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:29:08.386879 16400 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:29:08.386947  6457 raft_consensus.cc:606] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:08.394893  6457 raft_consensus.cc:614] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:08.396206  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:29:08.398612  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:08.399477  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:08.410182  6300 raft_consensus.cc:1275] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 35 index: 23. (index mismatch)
I20260709 12:29:08.411717 16400 consensus_queue.cc:1048] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:29:08.665653  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46162
I20260709 12:29:08.666276  6300 raft_consensus.cc:606] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Received request to transfer leadership
I20260709 12:29:08.670718  6200 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49040 (request call id 271) took 7687 ms. Trace:
I20260709 12:29:08.670239  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46162
I20260709 12:29:08.671680  6300 raft_consensus.cc:606] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Received request to transfer leadership
W20260709 12:29:08.673482  6193 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 1ee2be3ef27a4ec7bd5946b81b8ee8c3 already in progress [suppressed 1 similar messages]
I20260709 12:29:08.671376  6200 rpcz_store.cc:276] 0709 12:29:00.983220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.138731 (+5155511us) service_pool.cc:224] Handling call
0709 12:29:06.221793 (+ 83062us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0182b71 00007f6fa01830b2 00007f6fa90fc759 00007f6fa9094a76 00007f6fa9065d58 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc
0709 12:29:08.670612 (+2448819us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:08.106169 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:08.106762 (+   593us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:08.106797 (+    35us) write_op.cc:435] Acquired schema lock
0709 12:29:08.106815 (+    18us) tablet.cc:662] Decoding operations
0709 12:29:08.109695 (+  2880us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:08.109754 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:08.109780 (+    26us) tablet.cc:685] Acquiring locks for 35 operations
0709 12:29:08.112120 (+  2340us) tablet.cc:701] Row locks acquired
0709 12:29:08.112139 (+    19us) write_op.cc:260] PREPARE: finished
0709 12:29:08.112275 (+   136us) write_op.cc:270] Start()
0709 12:29:08.112392 (+   117us) write_op.cc:276] Timestamp: P: 1783600148112229 usec, L: 0
0709 12:29:08.112414 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:29:08.113131 (+   717us) log.cc:844] Serialized 13450 byte log entry
0709 12:29:08.283700 (+170569us) op_driver.cc:464] REPLICATION: finished
0709 12:29:08.337036 (+ 53336us) spinlock_profiling.cc:243] Waited 3.85 ms on lock 0x7b180049a2c0. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa9082ab1 00007f6fa908f281 00007f6fa01ba3f3 00007f6fa01bc7c1 00007f6fa01bc779 00007f6fa01bc709 00007f6fa01bc6d1 00007f6fa01bb9cd 0000000000388634 0000000000388479 00007f6fa019d83a
0709 12:29:08.337193 (+   157us) write_op.cc:301] APPLY: starting
0709 12:29:08.337418 (+   225us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:08.447981 (+110563us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:08.448004 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:08.586995 (+138991us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:08.587977 (+   982us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=0,mrs_lookups=0
0709 12:29:08.588023 (+    46us) write_op.cc:312] APPLY: finished
0709 12:29:08.588616 (+   593us) log.cc:844] Serialized 300 byte log entry
0709 12:29:08.588809 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:08.589122 (+   313us) write_op.cc:454] Released schema lock
0709 12:29:08.589207 (+    85us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:08.607175 (+ 17968us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:08.607214 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":154823,"reported_tablets":50,"spinlock_wait_cycles":75600768,"child_traces":[["op",{"apply.queue_time_us":47188,"apply.run_cpu_time_us":44224,"apply.run_wall_time_us":269949,"cfile_cache_hit":142,"cfile_cache_hit_bytes":150497,"num_ops":35,"prepare.queue_time_us":14612,"prepare.run_cpu_time_us":32960,"prepare.run_wall_time_us":41331,"raft.queue_time_us":109986,"raft.run_cpu_time_us":7959,"raft.run_wall_time_us":37600,"replication_time_us":171121,"spinlock_wait_cycles":8958080,"thread_start_us":181181,"threads_started":4,"wal-append.queue_time_us":56604}]]}
W20260709 12:29:08.718400  6201 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 142) took 168075 ms (client timeout 15000 ms). Trace:
W20260709 12:29:08.719067  6201 rpcz_store.cc:269] 0709 12:26:20.643281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.644259 (+   978us) service_pool.cc:224] Handling call
0709 12:26:20.972963 (+328704us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400123ed4. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa3211090 00007f6fa32aff3a 00007f6fa9065e9c 00007f6fa916404e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:29:08.718140 (+167745177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14380,"reported_tablets":29,"spinlock_wait_cycles":43631360}
I20260709 12:29:08.734404  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:46162
I20260709 12:29:08.735177  6300 raft_consensus.cc:606] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Received request to transfer leadership
W20260709 12:29:08.740871  6254 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 12:29:08.741317  6254 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 1ee2be3ef27a4ec7bd5946b81b8ee8c3 already in progress
W20260709 12:29:08.741632  6254 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 1ee2be3ef27a4ec7bd5946b81b8ee8c3 already in progress
I20260709 12:29:08.777288 16341 raft_consensus.cc:993] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:08.777992 16396 raft_consensus.cc:1081] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:08.781241  6379 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:29:08.781831  6379 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:08.782226  6379 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:08.796025  6379 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:08.799445  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:08.800117  6300 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:08.800441  6300 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:08.801252  6300 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:08.802742  6300 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:08.841490  6300 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:29:08.843676  6379 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:08.843577  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:08.844381  6453 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:08.845467  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:08.846124  7314 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:08.856870  7314 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:29:08.858726  6351 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:08.859699 16342 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:08.860234 16342 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:08.861276 16342 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:08.863126  6453 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
W20260709 12:29:08.879249  6203 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 299) took 120923 ms (client timeout 15000 ms). Trace:
W20260709 12:29:08.879817  6203 rpcz_store.cc:269] 0709 12:27:08.018210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.026759 (+  8549us) service_pool.cc:224] Handling call
0709 12:29:08.864106 (+120837347us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":50}
W20260709 12:29:08.893457  6204 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 146) took 82791 ms (client timeout 15000 ms). Trace:
W20260709 12:29:08.894073  6204 rpcz_store.cc:269] 0709 12:27:46.102904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.108093 (+  5189us) service_pool.cc:224] Handling call
0709 12:29:08.893203 (+82785110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49894,"reported_tablets":29}
I20260709 12:29:08.971823  6201 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 16 to 24, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 24 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:08.991299  6201 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 25 to 27, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:08.992938  6201 catalog_manager.cc:5697] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 27 to 28, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:08.994635  6201 catalog_manager.cc:5697] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 31 to 32, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 32 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:08.996474  6201 catalog_manager.cc:5697] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 25 to 30, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 30 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.014430  6201 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 28. New cstate: current_term: 28 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:09.016219  6201 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:09.035952  6205 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 273) took 120108 ms (client timeout 15000 ms). Trace:
I20260709 12:29:09.034173  6201 catalog_manager.cc:5697] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 25, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 25 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:09.044759  6201 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 9 to 13, term changed from 19 to 23, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65), VOTER 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66), VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67), VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 23 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.046829  6201 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 14 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.048418  6201 catalog_manager.cc:5697] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 29, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 29 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:09.050449  6201 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 26. New cstate: current_term: 26 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
W20260709 12:29:09.049383  6205 rpcz_store.cc:269] 0709 12:27:08.927106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.930635 (+  3529us) service_pool.cc:224] Handling call
0709 12:29:09.035750 (+120105115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27988,"reported_tablets":49}
I20260709 12:29:09.051764  6201 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.055090  6201 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 17 to 27, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:09.056414  6201 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 19 to 23, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.057940  6201 catalog_manager.cc:5697] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 33, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 33 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.064949  6201 catalog_manager.cc:5697] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.066646  6201 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 26 to 30. New cstate: current_term: 30 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.068082  6201 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 25, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 25 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:09.273536  6301 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:09.276074 16400 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:29:09.441205  6202 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 21 to 22, leader changed from 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 22 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.455181  6202 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 27 to 31. New cstate: current_term: 31 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.463163  7331 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:09.465691 16346 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:29:09.466265  6202 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 20 to 22, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 22 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.481328  6453 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:09.485296 16346 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:29:09.518224  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 62376.8us
I20260709 12:29:09.515905  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:29:09.568745  6202 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 149) took 18381 ms (client timeout 15000 ms). Trace:
I20260709 12:29:09.580997  6201 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 27 to 28, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:09.569159  6202 rpcz_store.cc:269] 0709 12:28:51.222900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:59.242660 (+8019760us) service_pool.cc:224] Handling call
0709 12:29:09.568671 (+10326011us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:09.487471 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:09.487965 (+   494us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:09.488012 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:29:09.488035 (+    23us) tablet.cc:662] Decoding operations
0709 12:29:09.488602 (+   567us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:09.488653 (+    51us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:09.488679 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:29:09.488989 (+   310us) tablet.cc:701] Row locks acquired
0709 12:29:09.489011 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:29:09.489143 (+   132us) write_op.cc:270] Start()
0709 12:29:09.489241 (+    98us) write_op.cc:276] Timestamp: P: 1783600149489105 usec, L: 0
0709 12:29:09.489260 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:29:09.489812 (+   552us) log.cc:844] Serialized 1275 byte log entry
0709 12:29:09.491905 (+  2093us) op_driver.cc:464] REPLICATION: finished
0709 12:29:09.492257 (+   352us) write_op.cc:301] APPLY: starting
0709 12:29:09.492355 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:09.494149 (+  1794us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:09.494251 (+   102us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:09.496038 (+  1787us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:09.496257 (+   219us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=0,mrs_lookups=0
0709 12:29:09.496310 (+    53us) write_op.cc:312] APPLY: finished
0709 12:29:09.496542 (+   232us) log.cc:844] Serialized 41 byte log entry
0709 12:29:09.497394 (+   852us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:09.497514 (+   120us) write_op.cc:454] Released schema lock
0709 12:29:09.497567 (+    53us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:09.498004 (+   437us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:09.498039 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":105,"apply.run_cpu_time_us":5026,"apply.run_wall_time_us":5687,"cfile_cache_hit":12,"cfile_cache_hit_bytes":12546,"num_ops":3,"prepare.queue_time_us":111,"prepare.run_cpu_time_us":2580,"prepare.run_wall_time_us":2581,"replication_time_us":2484}]]}
I20260709 12:29:09.642199  6198 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.648236  6198 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 26 to 36. New cstate: current_term: 36 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.650565  6198 catalog_manager.cc:5697] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 34 to 36, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 36 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.664348  6198 catalog_manager.cc:5697] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 22 to 23, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 23 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.671583  6198 catalog_manager.cc:5697] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:09.685132  6198 catalog_manager.cc:5697] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 20 to 33. New cstate: current_term: 33 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:09.687163  6198 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:09.691447  6197 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 277) took 16743 ms (client timeout 15000 ms). Trace:
W20260709 12:29:09.695725  6197 rpcz_store.cc:269] 0709 12:28:52.947793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:06.127394 (+13179601us) service_pool.cc:224] Handling call
0709 12:29:06.214513 (+ 87119us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5400002490. stack: 00007f6fa0188343 00007f6fa0188240 00007f6f9fb171d4 00007f6fa9855304 00007f6fa9855299 00007f6fa0042db1 00007f6fa0183640 00007f6fa905a846 00007f6fa916490e 00007f6fa76252a4 00007f6fa7625232 00007f6fa7625161 00007f6fa76250dc 00007f6fa7624392 00007f6fa15e248c 00007f6fa15e18c6
0709 12:29:09.691359 (+3476846us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20128,"reported_tablets":49,"spinlock_wait_cycles":43598080}
I20260709 12:29:09.771198  6198 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:50336 (request call id 150) took 2858 ms. Trace:
I20260709 12:29:09.771791  6198 rpcz_store.cc:276] 0709 12:29:06.912785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.046987 (+134202us) service_pool.cc:224] Handling call
0709 12:29:09.716512 (+2669525us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:09.716543 (+    31us) write_op.cc:454] Released schema lock
0709 12:29:09.771131 (+ 54588us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:09.696699 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:09.697213 (+   514us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:09.697260 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:29:09.697277 (+    17us) tablet.cc:662] Decoding operations
0709 12:29:09.698094 (+   817us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:09.698158 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:09.698188 (+    30us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:29:09.698851 (+   663us) tablet.cc:701] Row locks acquired
0709 12:29:09.698867 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:29:09.699033 (+   166us) write_op.cc:270] Start()
0709 12:29:09.699144 (+   111us) write_op.cc:276] Timestamp: P: 1783600149698967 usec, L: 0
0709 12:29:09.699161 (+    17us) op_driver.cc:348] REPLICATION: starting
0709 12:29:09.699977 (+   816us) log.cc:844] Serialized 2772 byte log entry
0709 12:29:09.702817 (+  2840us) op_driver.cc:464] REPLICATION: finished
0709 12:29:09.703271 (+   454us) write_op.cc:301] APPLY: starting
0709 12:29:09.703354 (+    83us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:09.706954 (+  3600us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:09.707046 (+    92us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:09.712837 (+  5791us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:09.713049 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:29:09.714825 (+  1776us) write_op.cc:312] APPLY: finished
0709 12:29:09.715099 (+   274us) log.cc:844] Serialized 73 byte log entry
0709 12:29:09.715884 (+   785us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:09.716008 (+   124us) write_op.cc:454] Released schema lock
0709 12:29:09.716060 (+    52us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":46296,"reported_tablets":33,"child_traces":[["op",{"apply.queue_time_us":248,"apply.run_cpu_time_us":8976,"apply.run_wall_time_us":13085,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29274,"num_ops":7,"prepare.queue_time_us":1067,"prepare.run_cpu_time_us":3575,"prepare.run_wall_time_us":13857,"replication_time_us":3536}]]}
I20260709 12:29:09.795431  6199 catalog_manager.cc:5697] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 31 to 35, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 35 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:09.806670  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:46152
I20260709 12:29:09.807495  6300 raft_consensus.cc:606] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:09.828667  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:46152
I20260709 12:29:09.829730  6300 raft_consensus.cc:606] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:09.872368  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:46152
I20260709 12:29:09.873104  6300 raft_consensus.cc:606] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
W20260709 12:29:09.935299  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 49783.5us
I20260709 12:29:09.943132  6300 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:46152
I20260709 12:29:09.946866  6300 raft_consensus.cc:606] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:09.969888  6457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:09.970674  6457 raft_consensus.cc:606] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Received request to transfer leadership to TS 4268023ecea149cd83cba77db53c3100
I20260709 12:29:09.972813  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:29:09.975072  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:09.975674  6199 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49024 (request call id 278) took 1839 ms. Trace:
I20260709 12:29:09.976012  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:09.976181  6199 rpcz_store.cc:276] 0709 12:29:08.136325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:08.139814 (+  3489us) service_pool.cc:224] Handling call
0709 12:29:09.975582 (+1835768us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:09.823482 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:09.824063 (+   581us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:09.824104 (+    41us) write_op.cc:435] Acquired schema lock
0709 12:29:09.824124 (+    20us) tablet.cc:662] Decoding operations
0709 12:29:09.824463 (+   339us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:09.824528 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:09.824562 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:09.824765 (+   203us) tablet.cc:701] Row locks acquired
0709 12:29:09.824782 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:29:09.824932 (+   150us) write_op.cc:270] Start()
0709 12:29:09.825045 (+   113us) write_op.cc:276] Timestamp: P: 1783600149824885 usec, L: 0
0709 12:29:09.825066 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:29:09.825688 (+   622us) log.cc:844] Serialized 527 byte log entry
0709 12:29:09.831113 (+  5425us) op_driver.cc:464] REPLICATION: finished
0709 12:29:09.834793 (+  3680us) write_op.cc:301] APPLY: starting
0709 12:29:09.834901 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:09.835884 (+   983us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:09.836001 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:09.836769 (+   768us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:09.836944 (+   175us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:29:09.836986 (+    42us) write_op.cc:312] APPLY: finished
0709 12:29:09.837390 (+   404us) log.cc:844] Serialized 25 byte log entry
0709 12:29:09.837632 (+   242us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:09.837728 (+    96us) write_op.cc:454] Released schema lock
0709 12:29:09.837774 (+    46us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:09.838138 (+   364us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:09.838172 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":19401,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":3428,"apply.run_cpu_time_us":3331,"apply.run_wall_time_us":3329,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":627,"prepare.run_cpu_time_us":2504,"prepare.run_wall_time_us":2505,"replication_time_us":5875}]]}
I20260709 12:29:10.016192  6384 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.020721 16454 raft_consensus.cc:993] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.021410 16314 raft_consensus.cc:1081] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.030405  6377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:29:10.031286  6377 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.031666  6377 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:10.033044  6206 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49010 (request call id 304) took 7813 ms. Trace:
I20260709 12:29:10.033336  6457 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:10.033528  6206 rpcz_store.cc:276] 0709 12:29:02.219214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:07.037287 (+4818073us) service_pool.cc:224] Handling call
0709 12:29:10.032981 (+2995694us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44918,"reported_tablets":50}
I20260709 12:29:10.035558 16455 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:10.043915  6377 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:10.051126  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:10.051857  6457 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:10.051363  7331 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:10.053205 16341 raft_consensus.cc:993] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.053857 16454 raft_consensus.cc:1081] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.053613  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:10.055192  6300 raft_consensus.cc:3055] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:10.055434  6300 raft_consensus.cc:740] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:10.056107  6300 consensus_queue.cc:260] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:10.056874  6300 raft_consensus.cc:3060] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:10.072687  6377 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:10.074790  6384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:29:10.075237  6384 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.075541  6384 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:10.080485  6300 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:10.081897  6351 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:10.082376  6384 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.082832 16400 raft_consensus.cc:2804] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:10.083459 16400 raft_consensus.cc:697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:10.084371 16400 consensus_queue.cc:237] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:10.085246  6300 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:10.085641  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:10.085908  6300 raft_consensus.cc:3055] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:29:10.086207  6453 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:10.086369  6300 raft_consensus.cc:740] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:10.087390 16346 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Committing config change with OpId 28.20: config changed from index 18 to 20, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:10.089151 16342 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:10.089462  6459 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 18 to 20, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:10.095606  7331 raft_consensus.cc:2955] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 18 to 20, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:10.098994  6300 consensus_queue.cc:260] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.101871  6453 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:29:10.102819  6300 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:10.104918  6384 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:10.106377  6457 raft_consensus.cc:2468] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:10.108841  6300 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:29:10.110003  6351 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:10.124490  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:10.125161  7314 raft_consensus.cc:3060] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:10.123967 16342 raft_consensus.cc:2804] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:10.147593 16342 raft_consensus.cc:697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:10.148563 16342 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.143620  6206 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 18 to 20, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:10.177102  6299 tablet_service.cc:1558] Processing DeleteTablet for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 with delete_type TABLET_DATA_TOMBSTONED (TS 4268023ecea149cd83cba77db53c3100 not found in new config with opid_index 20) from {username='slave'} at 127.0.0.1:46162
I20260709 12:29:10.175859  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:29:10.186050  7314 raft_consensus.cc:2468] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:29:10.187043 16065 raft_consensus.cc:993] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:29:10.188055 15886 raft_consensus.cc:1081] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Signalling peer 4268023ecea149cd83cba77db53c3100 to start an election
I20260709 12:29:10.198424  6304 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158"
dest_uuid: "4268023ecea149cd83cba77db53c3100"
 from {username='slave'} at 127.0.0.1:57668
I20260709 12:29:10.199031  6304 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.199378  6304 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:10.216856  6301 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:10.235446  6304 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:10.263724  6275 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a96cfc92d6c6455eae7d1204353604ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.265621  6457 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:29:10.267246 16396 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:10.282953  6304 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Requested vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:10.284335  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:10.285034  7314 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:10.290848  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:10.291497  6459 raft_consensus.cc:3055] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:10.291780  6459 raft_consensus.cc:740] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:10.293744  7314 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:29:10.295025  6275 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:10.292562  6459 consensus_queue.cc:260] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.297075  6384 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:29:10.300414  6459 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:10.301666 16396 raft_consensus.cc:993] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.302201 16396 raft_consensus.cc:1081] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.308418 16396 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:10.308578  6459 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4268023ecea149cd83cba77db53c3100 in term 28.
I20260709 12:29:10.309047 16396 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Becoming Leader. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:10.310200 16396 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.323541 16458 tablet_replica.cc:333] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:10.324249 16314 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:10.324791 16458 raft_consensus.cc:2243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:10.325519 16458 raft_consensus.cc:2272] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:10.327488  6384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:29:10.328111  6384 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.328368  6384 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.331588  6206 catalog_manager.cc:5697] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 25 to 26, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 26 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.336083 16454 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Committing config change with OpId 27.17: config changed from index -1 to 17, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:10.341249  6384 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:10.344681  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:10.345417  6457 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.346570  6459 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index -1 to 17, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:10.349385 16341 raft_consensus.cc:993] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.350037 16341 raft_consensus.cc:1081] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:10.350651  6384 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:10.353148  6301 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:10.353813  6301 raft_consensus.cc:3055] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:10.354194  6301 raft_consensus.cc:740] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:10.358927  6384 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51842
I20260709 12:29:10.355836  6301 consensus_queue.cc:260] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:10.365809  6301 raft_consensus.cc:3060] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:10.375394  6384 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:10.377166  6457 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:10.380980  6348 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:10.375959  6384 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:10.390228  6301 raft_consensus.cc:2468] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:10.393157 16342 raft_consensus.cc:2804] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:10.401221 16342 raft_consensus.cc:697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:10.454433 16342 consensus_queue.cc:237] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:10.552124 16458 ts_tablet_manager.cc:1916] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:29:10.553004  6457 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:10.564833  6459 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:29:10.589332  6425 consensus_peers.cc:597] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 45579ad5e67d4843a6f90bf59939a2d7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:10.600136 16458 ts_tablet_manager.cc:1929] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 28.19
I20260709 12:29:10.600886 16458 log.cc:1199] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4268023ecea149cd83cba77db53c3100: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/wals/1ee2be3ef27a4ec7bd5946b81b8ee8c3
I20260709 12:29:10.638366  6198 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 reported cstate change: term changed from 27 to 28, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 4268023ecea149cd83cba77db53c3100 (127.5.4.65). New cstate: current_term: 28 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.638336  6193 catalog_manager.cc:5028] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): tablet 1ee2be3ef27a4ec7bd5946b81b8ee8c3 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:29:10.673554 16457 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 12:29:10.641562  6206 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 23 to 24, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 24 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:10.714396  6199 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 17, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:29:10.716279  6193 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 78231.6us
I20260709 12:29:10.726227  6301 raft_consensus.cc:1275] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:29:10.736100  7314 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:10.748001 16346 consensus_queue.cc:1048] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:29:10.779628  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 129696us
I20260709 12:29:10.781927 16468 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:10.787158  6377 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:10.787173  6455 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:10.805891  6300 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:10.814898 16400 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:10.816408  6300 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:10.824204 16431 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:10.889536  6379 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Committing config change with OpId 27.17: config changed from index -1 to 17, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:29:10.910643  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 80499.6us
W20260709 12:29:10.885126  6384 consensus_meta.cc:287] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: Time spent flushing consensus metadata: real 0.500s	user 0.010s	sys 0.010s
I20260709 12:29:10.915131  6384 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.930708  6380 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 29. Current term is 30. Ops: []
I20260709 12:29:10.931646  6384 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:10.956125  6455 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:10.963346 16455 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:10.977828  7314 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:10.977907  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: true dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:10.978977  7331 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:10.981017  6307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:10.981840  6307 raft_consensus.cc:3055] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:29:10.982204  6307 raft_consensus.cc:740] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4268023ecea149cd83cba77db53c3100, State: Running, Role: LEADER
I20260709 12:29:10.983420  6307 consensus_queue.cc:260] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:10.984717  6307 raft_consensus.cc:3060] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:10.988278 16396 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:10.999421  7331 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:29:11.023188  6198 catalog_manager.cc:5697] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 24 to 25, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 25 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:11.034601  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 74976.9us
I20260709 12:29:11.067836 16400 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:11.069761  6351 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:11.070887 16400 raft_consensus.cc:2804] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:11.071656 16400 raft_consensus.cc:697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:11.072651 16476 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:11.073215  6459 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:29:11.074033 16400 consensus_queue.cc:237] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:11.075191 16455 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:11.078713  6307 raft_consensus.cc:2468] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 30.
I20260709 12:29:11.080809  7331 raft_consensus.cc:1240] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Rejecting Update request from peer 4268023ecea149cd83cba77db53c3100 for earlier term 29. Current term is 30. Ops: []
I20260709 12:29:11.184371  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:29:11.193864 16479 log.cc:927] Time spent T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:29:11.227560  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 97873.4us
W20260709 12:29:11.239517  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 89445.2us
I20260709 12:29:11.247722 16487 ts_tablet_manager.cc:933] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:29:11.198796 15886 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index -1 to 18, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.251507  6310 raft_consensus.cc:1275] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:29:11.224290  6380 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index -1 to 18, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.325364 16400 consensus_queue.cc:1048] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:11.326299  6310 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:29:11.336135  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94530.3us
I20260709 12:29:11.381462  6206 catalog_manager.cc:5697] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 29 to 30, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 30 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:11.441113  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 49134.1us
W20260709 12:29:11.453626  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 55324.9us
I20260709 12:29:11.458585 16501 ts_tablet_manager.cc:933] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:11.495754  6199 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 18, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 27 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:11.452304 16487 tablet_copy_client.cc:323] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
W20260709 12:29:11.521200  6275 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f4d316e983e24c05a1bda96f3c053d46. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:11.562860  6384 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:29:11.566761  6459 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:29:11.570802  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet a96cfc92d6c6455eae7d1204353604ec from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:29:11.571836  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet a96cfc92d6c6455eae7d1204353604ec from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-a96cfc92d6c6455eae7d1204353604ec
I20260709 12:29:11.573235 16501 tablet_copy_client.cc:323] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:29:11.580034  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 45579ad5e67d4843a6f90bf59939a2d7 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:29:11.580621  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 45579ad5e67d4843a6f90bf59939a2d7 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-45579ad5e67d4843a6f90bf59939a2d7
I20260709 12:29:11.581019  7318 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:29:11.587682  6310 raft_consensus.cc:1275] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:29:11.595080 16500 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:11.626716 16469 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:11.629530  6469 tablet_copy_source_session.cc:215] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:11.633870  6320 tablet_copy_source_session.cc:215] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:11.648419 16500 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:11.671949 16487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a96cfc92d6c6455eae7d1204353604ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:11.672144 16396 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.676193  6457 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.650815  6307 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index -1 to 18, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.658869 16469 consensus_queue.cc:1048] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:29:11.683131  6377 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Committing config change with OpId 26.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.697546 16501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 45579ad5e67d4843a6f90bf59939a2d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:11.730121  6378 consensus_queue.cc:237] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:11.748983  7317 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:11.750828 16469 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:11.751456  6307 raft_consensus.cc:1275] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:11.793005  6198 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 26 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:11.772435 16469 consensus_queue.cc:1048] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:29:11.809798 16342 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index 13 to 16, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:11.820749  7317 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 13 to 16, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:11.827195  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6ea0c2d7699e4559823a28a280a0e713 with cas_config_opid_index 13: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:29:11.830013  6307 raft_consensus.cc:2955] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 13 to 16, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:11.826316  6377 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:11.858843  6206 catalog_manager.cc:5697] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 13 to 16, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 24 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:11.863526 16501 tablet_copy_client.cc:806] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:11.878400  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet 6ea0c2d7699e4559823a28a280a0e713 with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:42278
I20260709 12:29:11.880754  6459 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:29:11.882825 16494 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:29:11.908557  6351 consensus_peers.cc:597] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 481d3efac77d41e6bba0961a140626dc. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:11.923897 16501 tablet_copy_client.cc:670] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:11.923338 16487 tablet_copy_client.cc:806] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:11.949919 16512 tablet_replica.cc:333] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:11.951736 16512 raft_consensus.cc:2243] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:11.953378 16512 raft_consensus.cc:2272] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:11.948011 16487 tablet_copy_client.cc:670] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:11.973335  6307 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:29:12.048213  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 69662.3us
I20260709 12:29:12.060482 16501 tablet_copy_client.cc:538] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:29:12.072722 16501 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:29:12.086254 16494 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:29:12.085434  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 107601us
I20260709 12:29:12.092716 16512 ts_tablet_manager.cc:1916] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:29:12.093494 16487 tablet_copy_client.cc:538] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:29:12.105029 16513 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.142s	user 0.000s	sys 0.005s
I20260709 12:29:12.131462 16518 ts_tablet_manager.cc:933] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395)
I20260709 12:29:12.138109 16487 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:29:12.143030 16512 ts_tablet_manager.cc:1929] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 24.15
I20260709 12:29:12.143604 16512 log.cc:1199] T 6ea0c2d7699e4559823a28a280a0e713 P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/6ea0c2d7699e4559823a28a280a0e713
I20260709 12:29:12.139703 16518 tablet_copy_client.cc:323] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.65:37395
I20260709 12:29:12.147426  6320 tablet_copy_service.cc:140] P 4268023ecea149cd83cba77db53c3100: Received BeginTabletCopySession request for tablet f4d316e983e24c05a1bda96f3c053d46 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:53970)
I20260709 12:29:12.148138  6320 tablet_copy_service.cc:161] P 4268023ecea149cd83cba77db53c3100: Beginning new tablet copy session on tablet f4d316e983e24c05a1bda96f3c053d46 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:53970: session id = 561a4e21d8b344a79af44568ca80a3c5-f4d316e983e24c05a1bda96f3c053d46
I20260709 12:29:12.157310 16342 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Committing config change with OpId 28.16: config changed from index -1 to 16, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:12.175511  6320 tablet_copy_source_session.cc:215] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:12.146051  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet 6ea0c2d7699e4559823a28a280a0e713 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:29:12.220430 16518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4d316e983e24c05a1bda96f3c053d46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:12.223054  6454 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Committing config change with OpId 28.16: config changed from index -1 to 16, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:12.284313  6308 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Committing config change with OpId 28.16: config changed from index -1 to 16, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:12.287480  6198 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 16, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.292769  6459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:12.293457  6459 raft_consensus.cc:606] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:12.315124  6459 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
mode: GRACEFUL
new_leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:55284
I20260709 12:29:12.315615  6459 raft_consensus.cc:606] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Received request to transfer leadership to TS 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:12.368902  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:12.373459 16522 ts_tablet_manager.cc:933] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:12.372274  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:12.379172  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:12.443765 16518 tablet_copy_client.cc:806] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:12.444463 16518 tablet_copy_client.cc:670] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:12.455233 16518 tablet_copy_client.cc:538] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:29:12.476871 16522 tablet_copy_client.cc:323] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.66:34641
I20260709 12:29:12.492036  6395 tablet_copy_service.cc:140] P 0bdd5f9c03b84f7289e750ebaab2fda6: Received BeginTabletCopySession request for tablet 481d3efac77d41e6bba0961a140626dc from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:46610)
I20260709 12:29:12.492681  6395 tablet_copy_service.cc:161] P 0bdd5f9c03b84f7289e750ebaab2fda6: Beginning new tablet copy session on tablet 481d3efac77d41e6bba0961a140626dc from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:46610: session id = 561a4e21d8b344a79af44568ca80a3c5-481d3efac77d41e6bba0961a140626dc
I20260709 12:29:12.506318  6395 tablet_copy_source_session.cc:215] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:12.512090 16522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 481d3efac77d41e6bba0961a140626dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:12.542316 16522 tablet_copy_client.cc:806] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:12.543195 16522 tablet_copy_client.cc:670] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:12.555804 16518 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:29:12.556219 16522 tablet_copy_client.cc:538] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:29:12.563861 16501 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=1 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:12.581868 16522 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:29:12.613029 16501 tablet_bootstrap.cc:492] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:12.616499 16501 ts_tablet_manager.cc:1403] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.544s	user 0.047s	sys 0.046s
I20260709 12:29:12.638576 16402 raft_consensus.cc:993] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:12.640589 16402 raft_consensus.cc:1081] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:12.643692  6378 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:29:12.644353  6378 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:12.644721  6378 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:12.670547  6378 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting forced leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.672556  6378 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:12.674980  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:12.675539  6303 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:12.675599  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:12.676128  6454 raft_consensus.cc:3055] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:29:12.676432  6454 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:12.660195 16501 raft_consensus.cc:359] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.677188  6454 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.680850  6454 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:12.686815  6454 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:29:12.684366 16501 raft_consensus.cc:740] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:12.711182 16501 consensus_queue.cc:260] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.711863  6348 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:12.713079 16342 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:12.717206 16342 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:12.718132 16342 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.724905 16501 ts_tablet_manager.cc:1434] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.108s	user 0.007s	sys 0.002s
I20260709 12:29:12.732558  6303 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:29:12.742345  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-45579ad5e67d4843a6f90bf59939a2d7 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:29:12.743044  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-45579ad5e67d4843a6f90bf59939a2d7 on tablet 45579ad5e67d4843a6f90bf59939a2d7 with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:29:12.831573  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 53296.6us
I20260709 12:29:12.833894  6198 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 27 to 28, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:12.835052 16487 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=1 applied=17 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:29:12.839429  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 61924.5us
I20260709 12:29:12.844120 16495 raft_consensus.cc:993] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: : Instructing follower 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:12.860754 16487 tablet_bootstrap.cc:492] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:12.861596 16487 ts_tablet_manager.cc:1403] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.724s	user 0.062s	sys 0.016s
I20260709 12:29:12.862589 15886 raft_consensus.cc:1081] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Signalling peer 0bdd5f9c03b84f7289e750ebaab2fda6 to start an election
I20260709 12:29:12.873649 16487 raft_consensus.cc:359] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.890611  6377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35"
dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
 from {username='slave'} at 127.0.0.1:51854
I20260709 12:29:12.891331  6377 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:12.891806  6377 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:12.895794 16487 raft_consensus.cc:740] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:12.907265 16487 consensus_queue.cc:260] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:12.910040 16487 ts_tablet_manager.cc:1434] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.001s
I20260709 12:29:12.918690  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-a96cfc92d6c6455eae7d1204353604ec received from {username='slave'} at 127.0.0.1:53970
I20260709 12:29:12.919143  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-a96cfc92d6c6455eae7d1204353604ec on tablet a96cfc92d6c6455eae7d1204353604ec with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:12.921486  6377 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:12.937536  6377 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:12.938001  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 86321.3us
I20260709 12:29:12.942878  6303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: true dest_uuid: "4268023ecea149cd83cba77db53c3100"
I20260709 12:29:12.943528  6303 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:12.964653  6303 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:29:12.977824  6351 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100; no voters: 
I20260709 12:29:12.983316 16528 raft_consensus.cc:2804] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:29:12.983925 16528 raft_consensus.cc:697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:12.984961 16528 consensus_queue.cc:237] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:12.991600  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: true dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:12.996423  6459 raft_consensus.cc:3055] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:29:12.996785  6459 raft_consensus.cc:740] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:13.007899  6459 consensus_queue.cc:260] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:13.016350  6459 raft_consensus.cc:3060] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:29:13.066789  6459 raft_consensus.cc:2468] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 31.
I20260709 12:29:13.112751  6198 catalog_manager.cc:5697] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 30 to 31, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 31 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:13.125262 16522 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:13.134771 16518 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=2 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:13.138906 16522 tablet_bootstrap.cc:492] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:13.147210 16522 ts_tablet_manager.cc:1403] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.566s	user 0.052s	sys 0.020s
I20260709 12:29:13.140496 16518 tablet_bootstrap.cc:492] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:13.151561 16518 ts_tablet_manager.cc:1403] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.596s	user 0.053s	sys 0.038s
I20260709 12:29:13.162259 16522 raft_consensus.cc:359] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:13.167166 16522 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:13.163049 16518 raft_consensus.cc:359] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:13.167846 16518 raft_consensus.cc:740] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:13.168031 16522 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:13.174907 16518 consensus_queue.cc:260] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:13.179039  7318 raft_consensus.cc:1217] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Deduplicated request from leader. Original: 27.16->[27.17-27.17]   Dedup: 27.17->[]
I20260709 12:29:13.179181 16518 ts_tablet_manager.cc:1434] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:29:13.183418 16522 ts_tablet_manager.cc:1434] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.003s
I20260709 12:29:13.189826  6320 tablet_copy_service.cc:342] P 4268023ecea149cd83cba77db53c3100: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f4d316e983e24c05a1bda96f3c053d46 received from {username='slave'} at 127.0.0.1:53970
I20260709 12:29:13.210604  6320 tablet_copy_service.cc:434] P 4268023ecea149cd83cba77db53c3100: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-f4d316e983e24c05a1bda96f3c053d46 on tablet f4d316e983e24c05a1bda96f3c053d46 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:13.212550  6459 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:13.214475 16528 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:29:13.214823  6395 tablet_copy_service.cc:342] P 0bdd5f9c03b84f7289e750ebaab2fda6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-481d3efac77d41e6bba0961a140626dc received from {username='slave'} at 127.0.0.1:46610
I20260709 12:29:13.215490  6395 tablet_copy_service.cc:434] P 0bdd5f9c03b84f7289e750ebaab2fda6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-481d3efac77d41e6bba0961a140626dc on tablet 481d3efac77d41e6bba0961a140626dc with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:13.322166  6307 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:13.324584  7318 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Advancing to term 28
I20260709 12:29:13.341228 16527 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:13.370759  7318 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:13.384388 16527 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:13.536427  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:13.558276 16494 raft_consensus.cc:1064] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
W20260709 12:29:13.599396  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 58667us
I20260709 12:29:13.608393  7317 raft_consensus.cc:1217] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEARNER]: Deduplicated request from leader. Original: 26.14->[26.15-26.15]   Dedup: 26.15->[]
I20260709 12:29:13.749737 16494 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 18, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:13.761758  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 166354us
I20260709 12:29:13.840087  6303 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:29:13.873519  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:13.883852  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:13.888469  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:13.897147  6459 raft_consensus.cc:1275] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:29:13.903677 16560 raft_consensus.cc:1064] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:13.921425 16560 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 15, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:13.923190  6459 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:29:13.931209  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 312977us
I20260709 12:29:13.953002 16553 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:13.954783 16455 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:29:13.986925  6273 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:13.990710 16527 consensus_queue.cc:1048] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:14.022195  6310 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:14.038460  7317 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:29:14.060178  7329 raft_consensus.cc:1217] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEARNER]: Deduplicated request from leader. Original: 28.15->[28.16-28.16]   Dedup: 28.16->[]
I20260709 12:29:14.112181 16514 raft_consensus.cc:1064] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:14.153566 16514 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 13, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:14.169627  7329 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:14.181978  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:14.188827 16497 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:29:14.200023  6454 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:14.179280 16561 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
W20260709 12:29:14.238668  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53341.1us
I20260709 12:29:14.242313 16560 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.012s
I20260709 12:29:14.254945 16455 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:29:14.301421  7331 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 18 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:14.303045  6274 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 62998.5us
I20260709 12:29:14.295297 16455 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Committing config change with OpId 28.20: config changed from index 18 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:14.311305  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 52489.8us
I20260709 12:29:14.332916  6378 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:29:14.366621  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
I20260709 12:29:14.372330 16560 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:29:14.373859  6351 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:14.373366  6454 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 18 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:14.376703  6198 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 18 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:14.388777  7331 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 LEARNER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:29:14.404734  6454 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:29:14.418541 16514 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:29:14.448930  6351 consensus_peers.cc:597] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:14.456511 16576 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:14.501997 16575 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 17 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:14.544301  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 91090.3us
I20260709 12:29:14.582010  6377 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Refusing update from remote peer 4268023ecea149cd83cba77db53c3100: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:29:14.598688  6452 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:14.645478  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.717873  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 150640us
W20260709 12:29:14.717648  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 50383us
I20260709 12:29:14.728593 16555 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:14.730055 16553 raft_consensus.cc:1064] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:14.740391  7331 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:14.748365  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:14.768103  6205 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 reported cstate change: config changed from index 17 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 27 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:14.770439 16553 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 14, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:14.785578  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.804740  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.791395  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.808521  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.805848  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.817582  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.807860  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.818228  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.819231  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.828938  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.851459 16586 log.cc:927] Time spent T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:29:14.898816  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.903172  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.907253  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.919955  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.938350  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.945417  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.947149  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.947804  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.948472  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.949156  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:14.948087  6379 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 17 to 18, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:14.949878  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:14.967382  7329 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:29:14.981554  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.983387  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:14.992789  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:14.997493  7314 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:15.000492  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.001896  6453 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:29:15.008239  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.014909  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.008239 16527 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
W20260709 12:29:15.024070  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.024991 16554 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Committing config change with OpId 26.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.040215 16528 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.007s
W20260709 12:29:15.039073  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.049935  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.061978  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:15.069303  6379 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:15.096596  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 192553us
W20260709 12:29:15.117753  6351 consensus_peers.cc:597] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:15.133533  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 69285.1us
I20260709 12:29:15.123304 16596 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.136366 16596 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.158013 16596 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.165349  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.185218  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.191494  6454 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Committing config change with OpId 26.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.194849  6272 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:15.196208 16596 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:29:15.257285 16597 log.cc:927] Time spent T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:29:15.281677 16561 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.292460  6379 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.286077  7314 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.304275  6454 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:29:15.310612  7329 raft_consensus.cc:1275] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:29:15.311407 16528 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:29:15.321193 16575 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.321903 16575 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.326644 16575 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.327816  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.332690 16527 consensus_queue.cc:1048] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:29:15.328465  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.343289 16562 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Committing config change with OpId 28.21: config changed from index 20 to 21, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.346333  6454 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Committing config change with OpId 28.21: config changed from index 20 to 21, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.382511  6275 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:15.383970 16575 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:15.419643  7329 raft_consensus.cc:2955] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Committing config change with OpId 28.21: config changed from index 20 to 21, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.435904 16548 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.437039 16514 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.437237 16548 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.437691 16514 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.439698 16548 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.442924 16514 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.445055 16548 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.445286 16555 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.445677 16548 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.446206 16555 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.447924 16548 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.448959 16555 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.412218  6199 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 26 leader_uuid: "4268023ecea149cd83cba77db53c3100" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.467567 16596 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.468271 16596 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.475389 16596 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.478730  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.480967  6454 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Committing config change with OpId 28.17: config changed from index 16 to 17, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:15.543188  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80090.4us
I20260709 12:29:15.548444 16514 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.549145 16514 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.549392  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.548465  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.552515 16514 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.552662  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.553299 16514 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.554042  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.554037 16514 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.556629 16514 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.557642 16548 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.563700 16548 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.566037 16548 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.558091 16575 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.566958 16575 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.568887 16575 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.562979 16576 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.575829  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.577346  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.578003  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.579396  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.580610  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.569649 16576 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.596251 16576 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.609973  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.618855  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.546130 16596 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.633124  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 45579ad5e67d4843a6f90bf59939a2d7 with cas_config_opid_index 20: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:29:15.638382  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.622648 16548 raft_consensus.cc:493] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.624063 16554 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.653875  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.660678  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:15.662931  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.665181 16515 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.665968 16514 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.665833 16515 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.626387 16596 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.673051  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.639698 16555 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.682873  5137 tablet_replica.cc:333] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:15.683954  5137 raft_consensus.cc:2243] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.685400 16596 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.661150 16548 raft_consensus.cc:515] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.688028  6206 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 16 to 17, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.689327 16548 leader_election.cc:290] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.690145 16515 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.655385 16554 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.690834  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.691103  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.692662 16554 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.698823  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.674628 16514 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.710265  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ea0c2d7699e4559823a28a280a0e713" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.738893 16612 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.739684 16612 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.745141  6272 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:15.750881 16514 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.690328  6206 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 20 to 21, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.751689 16514 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.752276 16514 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.754619 16555 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.757328 16555 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.742184 16612 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:15.763001  6193 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 136138us
I20260709 12:29:15.768000  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.770198  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.772748  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.776927  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:15.777094  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.779876  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.781903 16575 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:15.790397  5137 raft_consensus.cc:2272] T 2ddf475f266c47adb1a50aa986a0c44c P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.799571  5137 tablet_replica.cc:333] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:15.800267 16612 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.801055  5137 raft_consensus.cc:2243] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.800971 16612 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.801978  5137 raft_consensus.cc:2272] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:15.803265 16612 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.807178 16610 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.808072 16610 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.810635 16548 raft_consensus.cc:493] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.811178 16548 raft_consensus.cc:515] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.813159 16548 leader_election.cc:290] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.814471 16548 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.823309 16548 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.837880  6384 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.842922 16548 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.853856  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.863329  6453 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:29:15.867210  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 122184us
I20260709 12:29:15.869395 16562 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:15.882779 16515 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.883293 16515 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.884042 16562 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Committing config change with OpId 28.18: config changed from index 17 to 18, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.885267 16515 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.885854 16515 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.886623 16548 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.887104 16548 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.888687 16514 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.889015 16548 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.890528 16610 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:15.891353  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): ChangeConfig:REMOVE_PEER RPC failed for tablet a96cfc92d6c6455eae7d1204353604ec: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:15.893208  6274 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:15.888898  6453 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:15.894349  6274 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:15.895485  6274 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:15.897382 16554 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:15.897852  5137 tablet_replica.cc:333] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:15.899958 16576 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:15.900794 16576 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:15.901142  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55b778a085744d27a1adc35598438230" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 34 candidate_status { last_received { term: 33 index: 24 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.901490 16576 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:15.902385  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.897948 16554 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:15.903648 16614 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.904237  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.904469 16614 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:15.905104 16554 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.905856  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.903718  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.906633  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.908041  6274 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:15.909433  6274 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:15.910043  6274 raft_consensus.cc:2686] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4268023ecea149cd83cba77db53c3100: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:15.910640 16548 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:15.905155  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.914026 16596 raft_consensus.cc:493] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.914702 16596 raft_consensus.cc:515] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.914299 16555 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.917117 16596 leader_election.cc:290] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.916932 16555 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.912856 16576 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:15.918433 16576 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.919314 16555 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.919395  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.920724 16576 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.921362 16576 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:15.921931 16576 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.924252 16576 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:15.927712  6274 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:15.930296  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.930651  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.937470  6274 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:15.939949  6274 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:15.886288 16515 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:15.941550 16514 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:15.938144 16615 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:15.940493 16610 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:15.945425  6274 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:15.947569  7329 raft_consensus.cc:1275] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:29:15.949234  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.952739  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.957018  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.958150  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.958369  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.959646  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.964618  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:15.968518  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:15.972143  5137 raft_consensus.cc:2243] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:15.937484 16614 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:15.968700 16609 consensus_queue.cc:1048] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:29:15.991027  5137 pending_rounds.cc:70] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Trying to abort 1 pending ops.
I20260709 12:29:15.950992 16610 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:15.991569  5137 pending_rounds.cc:77] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: Aborting op as it isn't in flight: id { term: 28 index: 20 } timestamp: 7305626229202227200 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" old_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } } new_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } } }
I20260709 12:29:16.003309  5137 raft_consensus.cc:2917] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Aborting config change with OpId 28.20: Aborted: Op aborted
I20260709 12:29:16.003885 16515 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:16.007716  5137 raft_consensus.cc:2889] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:29:16.008299  5137 raft_consensus.cc:2272] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.013504 16615 raft_consensus.cc:670] T 45579ad5e67d4843a6f90bf59939a2d7 P 4268023ecea149cd83cba77db53c3100: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopped
I20260709 12:29:16.017005  5137 tablet_replica.cc:333] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.018626  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eede017c955d4746bbbd40d9384932e7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:16.027480  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.028098  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:16.032439  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dda9448d18794a44a75d0b570e998c3a" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.031956  6199 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 17 to 18, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:16.033445  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:16.030797  5137 raft_consensus.cc:2243] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.033540  6190 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 4e0e521cb08c420f981a29576eee90a6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:16.035000  5137 raft_consensus.cc:2272] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.035069  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:16.036258  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:16.036893  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.038735  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.039877  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b18717d2a8534f54b725363ce75d49ed" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.042515  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.045042  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.046809  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.049041  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.051075  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.052055  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.053593  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:16.053143  7318 raft_consensus.cc:2955] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Committing config change with OpId 28.18: config changed from index 17 to 18, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:16.038810  6272 leader_election.cc:304] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.056316  6274 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.027477  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "4268023ecea149cd83cba77db53c3100" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:16.078359  6272 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.082335 16515 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:29:16.089866 16614 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:16.081125 16555 raft_consensus.cc:2749] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:16.097813  6272 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.103586  5137 tablet_replica.cc:333] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.104636  5137 raft_consensus.cc:2243] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.105290  5137 raft_consensus.cc:2272] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.107798  6274 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.109136  6274 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.110894 16610 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:16.112540 16610 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:16.115411  6274 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.116650  6272 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.117185 16515 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:29:16.119357 16596 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:29:16.121168 16515 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:16.122430  6272 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.131403  6272 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.144506  5137 tablet_replica.cc:333] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
W20260709 12:29:16.145123  6272 raft_consensus.cc:2686] T 4cde8a89c1124a31810e39cb4a6c2518 P 4268023ecea149cd83cba77db53c3100: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:16.147543  5137 raft_consensus.cc:2243] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.154644  6272 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.187772  5137 raft_consensus.cc:2272] T 30efe8bf06814d23bf20818fb23139be P 4268023ecea149cd83cba77db53c3100 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.206993  6272 leader_election.cc:304] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.208709 16514 raft_consensus.cc:2749] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:16.209334  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 481d3efac77d41e6bba0961a140626dc with cas_config_opid_index 17: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:29:16.156659 16596 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:16.209410 16515 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:16.217734  6272 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.245797  6272 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:29:16.255184  6272 raft_consensus.cc:2686] T 150319986ee143d5a746f901c5b4adcc P 4268023ecea149cd83cba77db53c3100: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:16.258824  6272 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.251577 16596 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:16.260687 16514 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:16.261633  6272 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.271454 16548 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:16.275985  6272 leader_election.cc:304] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.289839 16515 raft_consensus.cc:2749] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:16.292344  5137 tablet_replica.cc:333] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.293620  5137 raft_consensus.cc:2243] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.294389  5137 raft_consensus.cc:2272] T 55b778a085744d27a1adc35598438230 P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.297479  5137 tablet_replica.cc:333] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.298589  5137 raft_consensus.cc:2243] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.299535  5137 raft_consensus.cc:2272] T f24bf2d0bc904aa495cdb5995b470fb0 P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.304543  5137 tablet_replica.cc:333] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.305502  5137 raft_consensus.cc:2243] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.306348  5137 raft_consensus.cc:2272] T f53146508e564f8aae279a61e6696e80 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.311767  5137 tablet_replica.cc:333] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.315007  5137 raft_consensus.cc:2243] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.316286  5137 raft_consensus.cc:2272] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.322448  6272 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46556.3us
I20260709 12:29:16.323987  6272 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.332165 16514 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:16.342872  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 88425.8us
I20260709 12:29:16.355437  5137 tablet_replica.cc:333] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.360980  6272 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.374096 16514 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:16.374440  6272 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:16.375967  6272 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4268023ecea149cd83cba77db53c3100; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.376061 16514 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:16.376812 16596 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:16.389801  5137 raft_consensus.cc:2243] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.392196  5137 raft_consensus.cc:2272] T a96cfc92d6c6455eae7d1204353604ec P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.407893  5137 tablet_replica.cc:333] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.411142  5137 raft_consensus.cc:2243] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.418982  5137 raft_consensus.cc:2272] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.427997  5137 tablet_replica.cc:333] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.428925  5137 raft_consensus.cc:2243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.429674  5137 raft_consensus.cc:2272] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.432547  5137 tablet_replica.cc:333] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.437304  5137 raft_consensus.cc:2243] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.447690  5137 raft_consensus.cc:2272] T 034bd5c36e71466da14c6a4850a920cb P 4268023ecea149cd83cba77db53c3100 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.479247  5137 tablet_replica.cc:333] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.480986  5137 raft_consensus.cc:2243] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.494926  5137 raft_consensus.cc:2272] T dd3cb6d1db5d481b9589e84d04e83d8e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.510453  5137 tablet_replica.cc:333] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.537272  5137 raft_consensus.cc:2243] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.538969  5137 raft_consensus.cc:2272] T 2483d7801887403e873adc62429fe046 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.543048  5137 tablet_replica.cc:333] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.551126  5137 raft_consensus.cc:2243] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.551808  5137 raft_consensus.cc:2272] T 4e75b2c5204044d2847e78c187388fd3 P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.557860  5137 tablet_replica.cc:333] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.558723  5137 raft_consensus.cc:2243] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.559361  5137 raft_consensus.cc:2272] T 3b2f25afefb44300a1d6297e2cc140f5 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.564837  5137 tablet_replica.cc:333] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.565703  5137 raft_consensus.cc:2243] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.566351  5137 raft_consensus.cc:2272] T b776445e68c14533bcde1f784caa703a P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.579620  5137 tablet_replica.cc:333] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.580505  5137 raft_consensus.cc:2243] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.581719  5137 raft_consensus.cc:2272] T 2e4301d25f8649e496bcbcb9f954c74e P 4268023ecea149cd83cba77db53c3100 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.609956  5137 tablet_replica.cc:333] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.611032  5137 raft_consensus.cc:2243] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.612172  5137 raft_consensus.cc:2272] T 6ea0c2d7699e4559823a28a280a0e713 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.619059  5137 tablet_replica.cc:333] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.621536  5137 raft_consensus.cc:2243] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.656916  5137 raft_consensus.cc:2272] T c413b06caa444496aecb13ac2142c246 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.667508  5137 tablet_replica.cc:333] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.668279  5137 raft_consensus.cc:2243] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.668830  5137 raft_consensus.cc:2272] T 62bd177b33be425d81796cc414f86f76 P 4268023ecea149cd83cba77db53c3100 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.685185  5137 tablet_replica.cc:333] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.685979  5137 raft_consensus.cc:2243] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.686765  5137 raft_consensus.cc:2272] T 481d3efac77d41e6bba0961a140626dc P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.690153  5137 tablet_replica.cc:333] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.690951  5137 raft_consensus.cc:2243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.691490  5137 pending_rounds.cc:70] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Trying to abort 1 pending ops.
I20260709 12:29:16.691721  5137 pending_rounds.cc:77] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100: Aborting op as it isn't in flight: id { term: 31 index: 19 } timestamp: 7305626226713665536 op_type: NO_OP noop_request { }
I20260709 12:29:16.692065  5137 raft_consensus.cc:2889] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:16.692410  5137 raft_consensus.cc:2272] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.694923  5137 tablet_replica.cc:333] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.695662  5137 raft_consensus.cc:2243] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.696513  5137 raft_consensus.cc:2272] T 252dcff751be4a3d948b0f83cb081100 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.731511  5137 tablet_replica.cc:333] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.732326  5137 raft_consensus.cc:2243] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.733419  5137 raft_consensus.cc:2272] T cdd6e8c2e3c642dc806079ce6070d975 P 4268023ecea149cd83cba77db53c3100 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.743813  5137 tablet_replica.cc:333] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.744566  5137 raft_consensus.cc:2243] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.745096  5137 raft_consensus.cc:2272] T eede017c955d4746bbbd40d9384932e7 P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.753185  5137 tablet_replica.cc:333] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.758775  5137 raft_consensus.cc:2243] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.759444  5137 raft_consensus.cc:2272] T 540a67ae2f134850bfa9955841e5f56a P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.762231  5137 tablet_replica.cc:333] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.762982  5137 raft_consensus.cc:2243] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.763509  5137 raft_consensus.cc:2272] T 1b749ec31e454255bbbba4a4bb493f28 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.771703  5137 tablet_replica.cc:333] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.772457  5137 raft_consensus.cc:2243] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.782258  5137 raft_consensus.cc:2272] T cd67024481794a7db3a98df0cd400c9d P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.785035  5137 tablet_replica.cc:333] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.785722  5137 raft_consensus.cc:2243] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.786330  5137 raft_consensus.cc:2272] T e44e2bcbe27f4a9192cc9acd16b966ad P 4268023ecea149cd83cba77db53c3100 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.786525 16527 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:16.787343 16527 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:16.789000  5137 tablet_replica.cc:333] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.789829  5137 raft_consensus.cc:2243] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.790383  5137 raft_consensus.cc:2272] T 3a0563d158eb400f86c11eb8e04fc8fc P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.790771 16527 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:16.793023  5137 tablet_replica.cc:333] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.793541  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:16.807144  5137 raft_consensus.cc:2243] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.795819  6351 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:16.807991  5137 raft_consensus.cc:2272] T bdf58974e308489d9ae9a07e0a775c71 P 4268023ecea149cd83cba77db53c3100 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.808476  6348 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:16.809935 16527 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:16.810926  5137 tablet_replica.cc:333] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.811780  5137 raft_consensus.cc:2243] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.812353  5137 raft_consensus.cc:2272] T dda9448d18794a44a75d0b570e998c3a P 4268023ecea149cd83cba77db53c3100 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.815048  5137 tablet_replica.cc:333] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.824504  5137 raft_consensus.cc:2243] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.825616  5137 raft_consensus.cc:2272] T d53ab19d424b4a85af2153ef24520678 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.828598  5137 tablet_replica.cc:333] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.829336  5137 raft_consensus.cc:2243] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.830579  5137 raft_consensus.cc:2272] T f4d316e983e24c05a1bda96f3c053d46 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.833103  5137 tablet_replica.cc:333] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
W20260709 12:29:16.833844  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:16.838752  5137 raft_consensus.cc:2243] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.839916  5137 raft_consensus.cc:2272] T f745d20b9f8c4aca9adf20ed14f509ac P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.852186  5137 tablet_replica.cc:333] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.853029  5137 raft_consensus.cc:2243] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.854272  5137 raft_consensus.cc:2272] T 40665848f81d4ce9a753d2417536ae0a P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.857081  5137 tablet_replica.cc:333] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.857824  5137 raft_consensus.cc:2243] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.858388  5137 raft_consensus.cc:2272] T 0fe2196990ce4d60adc195d394d4c01c P 4268023ecea149cd83cba77db53c3100 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.860874  5137 tablet_replica.cc:333] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.865598  5137 raft_consensus.cc:2243] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.866698  5137 raft_consensus.cc:2272] T f73da77e7bb0404e9f79f06e3dd200f4 P 4268023ecea149cd83cba77db53c3100 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:16.881453  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:16.891953  5137 tablet_replica.cc:333] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.892848  5137 raft_consensus.cc:2243] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.893886  5137 raft_consensus.cc:2272] T f0250a69b3914cb38265a7457b73a158 P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.905956  5137 tablet_replica.cc:333] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.906821  5137 raft_consensus.cc:2243] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.916826  5137 raft_consensus.cc:2272] T 402a1967982740658c70049fd62be44e P 4268023ecea149cd83cba77db53c3100 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.919804  5137 tablet_replica.cc:333] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.928553  5137 raft_consensus.cc:2243] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.934257  5137 raft_consensus.cc:2272] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4268023ecea149cd83cba77db53c3100 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.944880  5137 tablet_replica.cc:333] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.945654  5137 raft_consensus.cc:2243] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.946416  5137 raft_consensus.cc:2272] T 64dfdc039c8140e5bb288a15c2e827ce P 4268023ecea149cd83cba77db53c3100 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.950743  5137 tablet_replica.cc:333] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.951663  5137 raft_consensus.cc:2243] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.952349  5137 raft_consensus.cc:2272] T 28020383d3c344fda935a94bca07ad82 P 4268023ecea149cd83cba77db53c3100 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.959215  5137 tablet_replica.cc:333] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.961370  5137 raft_consensus.cc:2243] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:16.968629  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:16.969233  5137 raft_consensus.cc:2272] T 0ddf1d667df04473a598939605e43a1e P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.972085  5137 tablet_replica.cc:333] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.972903  5137 raft_consensus.cc:2243] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:16.973846  5137 raft_consensus.cc:2272] T b18717d2a8534f54b725363ce75d49ed P 4268023ecea149cd83cba77db53c3100 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:16.976683  5137 tablet_replica.cc:333] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100: stopping tablet replica
I20260709 12:29:16.977334  5137 raft_consensus.cc:2243] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:29:16.978426  5137 raft_consensus.cc:2272] T 7878a824e1e94db6a0b83ab47738f4b0 P 4268023ecea149cd83cba77db53c3100 [term 36 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:17.009825  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.030123  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.041417  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.056425  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.072666  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.073158  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.086548  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): ChangeConfig:REMOVE_PEER RPC failed for tablet f4d316e983e24c05a1bda96f3c053d46: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:29:17.117383  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.158368  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.163976  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.324640  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.325624  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.330374  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.338315  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.352255  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:17.403537 16600 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.404121 16600 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.411192  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:17.413440 16600 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:17.415099  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:17.423108  6425 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.423784  6425 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:17.425644 16529 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:17.448791  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.493889  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.499315  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.515869  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.518412  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 63417.7us
W20260709 12:29:17.544345  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 88472.1us
W20260709 12:29:17.588933  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:17.590806 16609 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.591421 16609 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.594069 16609 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:17.595018  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:17.595846  6459 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:29:17.597940  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:17.600314  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.602141  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.605005  6351 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.609560  6351 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:17.610375 16609 raft_consensus.cc:2749] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:17.625994  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.698753  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.710248  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.737069  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.741331  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:17.746402  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:17.790824 16617 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.791567 16617 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:17.809754  7288 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.813220  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:17.813942  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:29:17.814160  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:17.814962  6459 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:29:17.821521  7287 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:17.824007 16565 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:17.824429 16565 raft_consensus.cc:493] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.824769 16565 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:29:17.854949  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:17.865196 16617 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:17.887629  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:17.890862 16565 raft_consensus.cc:515] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.901224  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:17.901852  6381 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:17.903283  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a96cfc92d6c6455eae7d1204353604ec" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:17.904047  6459 raft_consensus.cc:3060] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:29:17.908824  7288 leader_election.cc:336] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.912209  6381 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:17.912359 16609 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.913141 16609 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:17.915692 16609 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:17.917753 16529 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.918979 16600 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.919466  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:17.919596 16600 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:17.921850 16600 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:17.918388 16529 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:17.922374 16600 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.923038 16600 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:17.924197 16529 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:17.925045 16600 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:17.929800 16617 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.934450 16630 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.934314 16617 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.935078 16630 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.937337 16630 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:17.938720  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:17.941454 16630 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.942063 16630 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:17.944195  7288 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.944716  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:17.943666 16629 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.945647 16621 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:17.946153 16629 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:17.947253 16609 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.947358 16529 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.948025 16529 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:17.948585 16629 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:17.947837 16609 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:17.944218  6351 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.962110  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:17.962235 16529 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:17.963261 16628 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:17.964706 16627 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.965265 16627 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.967442 16627 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:17.969200  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:17.951406  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:17.951808  6425 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:17.971293  6425 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:17.971920  6425 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:17.959789  7288 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:17.981268  6425 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:17.985438  6351 leader_election.cc:336] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.986013  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:17.986840 16627 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:17.988147  6351 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:17.988579  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:17.989812 16529 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.990411 16529 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:17.992637 16529 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:17.997841 16631 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:17.999303  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.008203  6453 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:18.008014 16631 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.009917  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.003600 16617 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.010731 16627 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:18.010838 16621 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:18.011495 16621 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.011904 16621 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.016361 16630 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.017444 16631 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.022917 16565 leader_election.cc:290] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.001236  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.027150  6454 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:18.018199 16627 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.028160 16627 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.030273 16627 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.031445  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:18.031450  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.032346 16633 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:18.032879 16633 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:29:18.005865  6425 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.033227 16633 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.025038  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.033934  6453 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:29:18.034103 16621 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.035230  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:18.035984 16629 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:29:18.036386 16629 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.036727 16629 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:18.038524  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:18.039680  7288 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.039866  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:18.007058 16609 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.044651  6459 raft_consensus.cc:2468] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:18.040992 16633 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.047873  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.063268  7317 raft_consensus.cc:2393] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:18.063433 16629 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.064275 16633 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.048530  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.049104  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.065734  6382 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 23.
I20260709 12:29:18.050886  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.053318 16631 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.053507  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.066661  6380 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:29:18.055285  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:18.066668 16631 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.067453  6453 raft_consensus.cc:3060] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.067556  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:18.068142  6454 raft_consensus.cc:3060] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:29:18.069013 16631 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.069762  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:18.070348  6458 raft_consensus.cc:3060] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.057418 16621 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.064719  6381 raft_consensus.cc:2393] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 28: Already voted for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in this term.
I20260709 12:29:18.076612  6454 raft_consensus.cc:2468] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 28.
I20260709 12:29:18.077133  6458 raft_consensus.cc:2468] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:29:18.077513 16621 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.093654  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.096874 16621 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.097934  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.066334 16629 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.100114 16621 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.101248  6348 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:18.087406  7285 leader_election.cc:304] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:29:18.089640  6425 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.095667 16631 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.103175 16633 raft_consensus.cc:2804] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:18.103794 16633 raft_consensus.cc:697] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:18.103644 16631 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.103296  7285 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:18.104758 16633 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.078848  6453 raft_consensus.cc:2468] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:29:18.101293  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.106380 16631 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.106997 16634 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.107755 16629 raft_consensus.cc:2804] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:29:18.107610 16634 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.105312 16621 raft_consensus.cc:2804] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:18.110316  6424 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:18.111811  6351 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.109493 16529 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:18.110584 16629 raft_consensus.cc:697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:18.117131 16629 consensus_queue.cc:237] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.118705 16621 raft_consensus.cc:697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:18.119717 16529 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:29:18.118273  6351 leader_election.cc:336] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.119658 16621 consensus_queue.cc:237] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.120385 16529 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.122673 16529 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.123440  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:18.123804  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:18.106514  7285 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:29:18.124588  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:18.125326  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.125900 16630 raft_consensus.cc:2804] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:18.126958 16628 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:18.127243 16631 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:18.125240  7288 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:18.134884  7288 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.134532 16529 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:18.142953 16630 raft_consensus.cc:697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:18.146893 16529 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:18.147353 16529 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.147771 16529 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:18.147518 16630 consensus_queue.cc:237] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.148388  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:18.149087  7317 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:29:18.150297  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.125533 16632 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:29:18.152727  6351 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.156798 16634 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.160219 16632 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.160861 16632 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:18.161198  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.162106  6454 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:29:18.169334 16632 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.171972 16632 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:18.172911  7288 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.179957  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.178005 16527 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.180828 16527 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.184146 16527 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:18.171697  6425 leader_election.cc:336] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:18.185192  6425 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.198235  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:29:18.201716  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 76481.4us
I20260709 12:29:18.204914  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.208102  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:18.211560  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.212378  6381 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:29:18.214186  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.214885  6378 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 28.
I20260709 12:29:18.197844 16565 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:29:18.218674 16637 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:18.220892 16627 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.230772 16637 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.231324 16637 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.227255  7317 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 36.
I20260709 12:29:18.218218 16600 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:18.231320 16627 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.240180 16627 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.225404  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:18.261184  6381 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:18.252686  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.263827  6454 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:18.255303  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:18.265202  7317 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 26.
I20260709 12:29:18.266916  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.260375  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:18.225555 16527 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:29:18.269450  6351 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.269114 16527 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.270102 16627 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:29:18.270257 16632 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:18.283988 16632 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.284530 16632 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.285211 16527 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:18.287889 16529 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:18.289875  6425 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.290130 16529 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:18.293510  7288 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.293991  7288 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.294771 16565 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:18.270601  5137 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 6331)
I20260709 12:29:18.271574 16609 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.297163 16627 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.296990 16609 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.297803 16627 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:29:18.299197 16609 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.305573  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:18.306342  6454 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 24.
I20260709 12:29:18.297017  6199 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 27 to 28, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 28 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:18.274869  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:29:18.310647  6425 leader_election.cc:336] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.314729 16529 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:18.315323 16529 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.315786 16529 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.323910 16565 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.324617 16565 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.309072  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
W20260709 12:29:18.334561  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 65134.9us
I20260709 12:29:18.336344  6381 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 24.
I20260709 12:29:18.353335  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:29:18.355397  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64841.7us
I20260709 12:29:18.354156  6376 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:29:18.358462  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:18.371443  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:18.379448 16529 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.385208 16529 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.378947 16633 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.393401 16633 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.394630 16634 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:18.395129 16634 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.395576 16634 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:18.395776 16627 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 37 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.379573 16637 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.400856 16637 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.401060 16628 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:18.401564 16628 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.401927 16628 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:18.345878 16565 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.335988 16527 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:29:18.403122 16527 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
I20260709 12:29:18.403643 16527 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:18.406113  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 94211.8us
I20260709 12:29:18.408200  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:18.409078  6376 raft_consensus.cc:3060] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:18.409883  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:18.410509  6381 raft_consensus.cc:3060] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.423979  6381 raft_consensus.cc:2468] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:29:18.413996  6424 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100
W20260709 12:29:18.436591  6193 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 208559us
I20260709 12:29:18.441679  6376 raft_consensus.cc:2468] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 30.
I20260709 12:29:18.443215 16633 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.464296 16641 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:18.467743  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:18.468611  6376 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:29:18.469822  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
W20260709 12:29:18.478919  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 117957us
I20260709 12:29:18.484247  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:18.501323 16634 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.517218  7318 raft_consensus.cc:2393] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:18.519215 16634 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.523355  6205 catalog_manager.cc:5697] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 27 to 28, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 28 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.490809 16649 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:29:18.533860  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 87528.4us
I20260709 12:29:18.533481  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:18.535738  7317 raft_consensus.cc:3060] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Advancing to term 37
W20260709 12:29:18.534814  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65094.2us
I20260709 12:29:18.534669  6424 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:18.539983  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
W20260709 12:29:18.540642  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): ChangeConfig:REMOVE_PEER RPC failed for tablet f4d316e983e24c05a1bda96f3c053d46: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.540683  6376 raft_consensus.cc:3060] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:18.543347 16653 raft_consensus.cc:2804] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:18.550836 16653 raft_consensus.cc:697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:18.552006 16653 consensus_queue.cc:237] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:18.552853  6351 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.537257  6424 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:18.562577  6376 raft_consensus.cc:2468] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
I20260709 12:29:18.495635 16641 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:18.566390 16646 raft_consensus.cc:2804] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:18.573302 16632 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.575886 16632 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:18.577082 16627 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.582118  6376 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:18.582345  7287 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:18.588025  7317 raft_consensus.cc:2468] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 37.
W20260709 12:29:18.565932  6425 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.533118 16649 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4268023ecea149cd83cba77db53c3100)
W20260709 12:29:18.593037  7288 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.502969 16527 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.594287  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:18.643716  6376 raft_consensus.cc:3060] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.591734 16628 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.601666 16649 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:29:18.651845  6454 raft_consensus.cc:1275] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:29:18.653244 16628 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:18.587544 16646 raft_consensus.cc:697] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:18.655440 16646 consensus_queue.cc:237] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.659873  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:18.660630  6454 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:18.661108  6351 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:18.593711  6425 leader_election.cc:336] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.616032 16641 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:18.643720  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 52514.8us
W20260709 12:29:18.661901  6351 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.666702  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:18.662597  6376 raft_consensus.cc:2468] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
W20260709 12:29:18.662645  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 73059.7us
W20260709 12:29:18.665539  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 70499.9us
W20260709 12:29:18.671865  6425 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.661298 16657 raft_consensus.cc:2804] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:29:18.676059  6351 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.683199  6424 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:18.683635 16647 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:18.690914 16655 raft_consensus.cc:2804] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:18.692109 16655 raft_consensus.cc:697] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:18.693814  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:18.694550  7329 raft_consensus.cc:3060] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:18.695941  6351 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.704017  7329 raft_consensus.cc:2468] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 29.
I20260709 12:29:18.711107  6454 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:29:18.717919 16647 raft_consensus.cc:2804] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:29:18.678548 16617 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.037s
W20260709 12:29:18.719941  6425 leader_election.cc:336] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.727435  6453 raft_consensus.cc:1275] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:29:18.733542 16527 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:18.737922  7288 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:18.739014  7288 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:18.740093  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:18.743233 16609 consensus_queue.cc:1048] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:29:18.747152  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:18.747879  6454 raft_consensus.cc:3060] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:18.755649  6425 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.763959 16529 raft_consensus.cc:2804] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:18.775017  6454 raft_consensus.cc:2468] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 25.
I20260709 12:29:18.777024  6348 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6; no voters: 
I20260709 12:29:18.754688 16647 raft_consensus.cc:697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 37 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:29:18.784899  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 115307us
I20260709 12:29:18.780568 16645 raft_consensus.cc:2804] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:29:18.714941 16655 consensus_queue.cc:237] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:18.816128  6351 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:18.828298  6458 raft_consensus.cc:1275] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:29:18.786646 16657 raft_consensus.cc:697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:18.849716  6451 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:29:18.839257 16529 raft_consensus.cc:697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:18.845988 16645 raft_consensus.cc:697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:18.874840  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:18.877846  7329 raft_consensus.cc:3060] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:18.839493 16633 consensus_queue.cc:1048] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.030s
I20260709 12:29:18.888885  7329 raft_consensus.cc:2468] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 27.
I20260709 12:29:18.870534 16647 consensus_queue.cc:237] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 36.24, Last appended by leader: 24, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.910796 16657 consensus_queue.cc:237] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.909704 16529 consensus_queue.cc:237] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:18.919836  6351 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100
I20260709 12:29:18.918115 16670 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.006s
W20260709 12:29:18.963605  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 43839.3us
I20260709 12:29:18.906739 16649 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:19.064188  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 150722us
I20260709 12:29:18.927311 16645 consensus_queue.cc:237] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:18.903443 16617 consensus_queue.cc:1048] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
W20260709 12:29:19.109176  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 82811us
W20260709 12:29:19.113572  6351 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:19.127620 16649 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Requested vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:19.115047 16628 raft_consensus.cc:2804] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:29:19.164319  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 248117us
W20260709 12:29:19.176990  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 179140us
I20260709 12:29:19.191771  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:19.192499  6376 raft_consensus.cc:3060] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:29:19.192588  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:19.193253  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 293640us
I20260709 12:29:19.203922  6382 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:29:19.276400  6376 raft_consensus.cc:2468] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
W20260709 12:29:19.296324  5137 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 6331)
I20260709 12:29:19.281231 16628 raft_consensus.cc:697] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Becoming Leader. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:19.279673 16656 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:29:19.307600 16628 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:19.347689 16669 log.cc:927] Time spent T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.224s	user 0.005s	sys 0.000s
W20260709 12:29:19.379935  6425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 352 similar messages]
I20260709 12:29:19.404268  6376 raft_consensus.cc:1275] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:29:19.407099  6381 raft_consensus.cc:1275] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:29:19.414173  7287 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:19.415983 16656 consensus_queue.cc:1048] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:19.416409 16631 consensus_queue.cc:1048] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:29:19.416006 16634 raft_consensus.cc:2804] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:29:19.418339 16634 raft_consensus.cc:697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:19.419416 16634 consensus_queue.cc:237] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:19.422045  6206 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 23 to 24, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 24 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:19.449506  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 60414.9us
W20260709 12:29:19.455035  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 334131us
W20260709 12:29:19.522989  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 141074us
W20260709 12:29:19.534109  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:19.537271  7288 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:19.581247  7317 raft_consensus.cc:1275] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
W20260709 12:29:19.605530  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:19.606583  6425 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:19.587539 16631 consensus_queue.cc:1048] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.041s
I20260709 12:29:19.649035  6380 raft_consensus.cc:1275] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:29:19.659139  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:19.669931  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:19.679024 16688 consensus_queue.cc:1048] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:29:19.693145  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:19.700815  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:19.737689  7317 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:29:19.746114  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:19.788792  6455 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:29:19.828202  6455 raft_consensus.cc:1275] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:29:19.797886 16679 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:29:19.851464 16527 consensus_queue.cc:1048] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:19.870720 16658 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:29:19.889458  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:19.912683  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:19.953651  7318 raft_consensus.cc:1275] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 36 index: 24. Preceding OpId from leader: term: 37 index: 25. (index mismatch)
W20260709 12:29:19.966276  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:19.967512  6205 catalog_manager.cc:5697] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 24 to 25, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 25 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.969340  6205 catalog_manager.cc:5697] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 29 to 30, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 30 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:19.970757  6205 catalog_manager.cc:5697] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 36 to 37, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 37 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.972544  6205 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 26 to 27, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66). New cstate: current_term: 27 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:19.974259  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:19.994153  6205 catalog_manager.cc:5697] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: term changed from 28 to 29, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 29 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } }
I20260709 12:29:20.014070 16627 consensus_queue.cc:1048] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260709 12:29:20.032274  6199 catalog_manager.cc:5697] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 28 to 29, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 29 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:20.039819  6199 catalog_manager.cc:5697] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 28 to 29, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 29 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:20.042208  6199 catalog_manager.cc:5697] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 26 to 27, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 27 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:20.051822  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:20.058738  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.059548  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.061388  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:20.122099  6382 raft_consensus.cc:1275] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:29:20.128435  6382 raft_consensus.cc:1275] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:29:20.152149  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:20.156586  6206 catalog_manager.cc:5697] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 29, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 29 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:20.161931 16662 consensus_queue.cc:1048] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:29:20.164638  6206 catalog_manager.cc:5697] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 28 to 29, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67). New cstate: current_term: 29 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:20.187942  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:20.188688  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:20.191465 16717 consensus_queue.cc:1048] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.010s
W20260709 12:29:20.239928  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.277034  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.277735  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.286317  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.277209  6205 catalog_manager.cc:5697] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 23 to 24, leader changed from 4268023ecea149cd83cba77db53c3100 (127.5.4.65) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 24 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:20.296972  5137 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:20.308962  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.374848  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.380829  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.501579  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.539937  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.546052  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.560760  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.573933  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.587371  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.625970  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.633805  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 45579ad5e67d4843a6f90bf59939a2d7: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:20.639250  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.662999  6384 consensus_queue.cc:237] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:20.673218  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.685869  7317 raft_consensus.cc:1275] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:29:20.692310  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.692982  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.700261  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.700075 16647 consensus_queue.cc:1048] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:29:20.707434  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:20.707978  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:20.712598  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.713338  7314 consensus_queue.cc:237] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:20.732757  6384 raft_consensus.cc:1275] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:29:20.733160  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:20.734345 16621 consensus_queue.cc:1048] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:29:20.736831 16628 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 16 to 18, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:20.756163  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.760928  7314 raft_consensus.cc:2955] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 16 to 18, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:20.775385  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:20.782249 16649 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Committing config change with OpId 28.20: config changed from index 18 to 20, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:20.863492  6192 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f4d316e983e24c05a1bda96f3c053d46 with cas_config_opid_index 16: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
I20260709 12:29:20.864399  6377 raft_consensus.cc:2955] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Committing config change with OpId 28.20: config changed from index 18 to 20, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:20.946552  6193 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a96cfc92d6c6455eae7d1204353604ec with cas_config_opid_index 18: ChangeConfig:REMOVE_PEER succeeded (attempt 9)
W20260709 12:29:20.989837  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 46188.3us
W20260709 12:29:21.006768  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:21.007405  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 64082.7us
I20260709 12:29:21.022629  6205 catalog_manager.cc:5697] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 16 to 18, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 27 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:21.045346  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:21.156510  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:21.303937  5137 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 6331)
I20260709 12:29:21.367926  6442 tablet_service.cc:1558] Processing DeleteTablet for tablet f4d316e983e24c05a1bda96f3c053d46 with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:42278
I20260709 12:29:21.386458  6199 catalog_manager.cc:5697] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 18 to 20, VOTER 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) evicted. New cstate: current_term: 28 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:21.397743  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:21.424003  6446 tablet_service.cc:1558] Processing DeleteTablet for tablet a96cfc92d6c6455eae7d1204353604ec with delete_type TABLET_DATA_TOMBSTONED (TS 4e0e521cb08c420f981a29576eee90a6 not found in new config with opid_index 20) from {username='slave'} at 127.0.0.1:42278
I20260709 12:29:21.468122 16736 tablet_replica.cc:333] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:21.470085 16736 raft_consensus.cc:2243] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.475330 16736 raft_consensus.cc:2272] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:21.480101 16736 ts_tablet_manager.cc:1916] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:29:21.623103 16736 ts_tablet_manager.cc:1929] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.17
I20260709 12:29:21.623667 16736 log.cc:1199] T f4d316e983e24c05a1bda96f3c053d46 P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/f4d316e983e24c05a1bda96f3c053d46
I20260709 12:29:21.641274  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet f4d316e983e24c05a1bda96f3c053d46 (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
I20260709 12:29:21.648205 16736 tablet_replica.cc:333] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:21.663187 16736 raft_consensus.cc:2243] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:21.666004 16736 raft_consensus.cc:2272] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:21.696373  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:21.711678 16736 ts_tablet_manager.cc:1916] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:29:21.735420  6425 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:21.821985 16736 ts_tablet_manager.cc:1929] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 28.19
I20260709 12:29:21.827521 16736 log.cc:1199] T a96cfc92d6c6455eae7d1204353604ec P 4e0e521cb08c420f981a29576eee90a6: Deleting WAL directory at /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/wals/a96cfc92d6c6455eae7d1204353604ec
I20260709 12:29:21.829510  6190 catalog_manager.cc:5028] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): tablet a96cfc92d6c6455eae7d1204353604ec (table test-workload [id=caa276ebe4814c8197f43c3434fd5228]) successfully deleted
W20260709 12:29:22.014205  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.080154  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.081278  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.083953  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.202389  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.209127  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.258765  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.268266  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.268968  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.309981  5137 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:22.918309  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.919281  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.919696  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.922794  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.927011  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.936939  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:22.969614  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:22.999194  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.034463  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.065097  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.098732  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.121641  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:23.161056  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.165860  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.166661  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.167411  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.167922  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.168272  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:23.249543  6453 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:29:23.283984  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e75b2c5204044d2847e78c187388fd3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:23.285264  6379 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:29:23.288574 16755 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:29:23.295576  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:23.311232  5137 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:23.318607  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.320870 16755 raft_consensus.cc:2955] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:23.324160  6377 raft_consensus.cc:2955] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Committing config change with OpId 23.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:23.346798  6453 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:23.365621  6199 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:23.382472  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.441407  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.474387  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.480719  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.481099  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.481751  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.483603  6377 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
W20260709 12:29:23.486303  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 28020383d3c344fda935a94bca07ad82. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:23.487358 16755 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
W20260709 12:29:23.497812  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:23.515301  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.519790 16732 raft_consensus.cc:2955] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 LEADER]: Committing config change with OpId 32.19: config changed from index -1 to 19, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:29:23.523994  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:23.524907  6377 raft_consensus.cc:2955] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Committing config change with OpId 32.19: config changed from index -1 to 19, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:29:23.531134  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800172760 after lost signal to thread 5140
W20260709 12:29:23.532312  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800307bc0 after lost signal to thread 6240
W20260709 12:29:23.533720  6237 debug-util.cc:398] Leaking SignalData structure 0x7b08001f2160 after lost signal to thread 6409
W20260709 12:29:23.537310  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800309200 after lost signal to thread 6483
W20260709 12:29:23.540124  6237 debug-util.cc:398] Leaking SignalData structure 0x7b0800304e80 after lost signal to thread 7371
W20260709 12:29:23.549947  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:23.578620  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.609858  6377 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:29:23.617568  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.617441  6199 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 19, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 32 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:23.623101  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.634325  6453 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:29:23.636188  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.638239 16746 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:23.645624  7318 raft_consensus.cc:3060] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 15 LEARNER]: Advancing to term 25
W20260709 12:29:23.646556  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:23.665603  7318 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:29:23.669150  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.673595 16746 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:29:23.678479 16746 consensus_queue.cc:1243] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer 561a4e21d8b344a79af44568ca80a3c5 log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:29:23.703627  7318 pending_rounds.cc:85] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Aborting all ops after (but not including) 8
I20260709 12:29:23.704000  7318 pending_rounds.cc:107] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 14.9
I20260709 12:29:23.704312  7318 raft_consensus.cc:2917] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEARNER]: Aborting config change with OpId 14.9: Aborted: Op aborted by new leader
I20260709 12:29:23.704778  7318 raft_consensus.cc:2889] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:23.721513 16647 raft_consensus.cc:2955] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Committing config change with OpId 25.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:23.740240  6459 raft_consensus.cc:2955] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Committing config change with OpId 25.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:23.810864  6206 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 25 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
W20260709 12:29:23.828298  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 56373.1us
W20260709 12:29:23.860941  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 70054.1us
I20260709 12:29:23.863859  6459 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:23.894541  7314 raft_consensus.cc:2955] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEARNER]: Committing config change with OpId 25.15: config changed from index -1 to 15, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:23.909235  6380 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:29:23.912581  6425 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:23.918187  6425 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625): Couldn't send request to peer 561a4e21d8b344a79af44568ca80a3c5. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 44fa024e8a7d48a6bc67d49f348bc0c0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:23.925962 16764 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
W20260709 12:29:23.932073  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:23.961015 16787 ts_tablet_manager.cc:933] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:23.985730 16780 raft_consensus.cc:1064] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:23.988984 16780 consensus_queue.cc:237] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 13, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:24.013995  6459 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:29:24.017989 16780 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:24.021997 16787 tablet_copy_client.cc:323] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:29:24.024302  7314 raft_consensus.cc:1275] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEARNER]: Refusing update from remote peer 0bdd5f9c03b84f7289e750ebaab2fda6: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:29:24.026433 16786 log.cc:927] Time spent T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:29:24.054456 16764 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:24.060768 16784 consensus_queue.cc:1048] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:29:24.071789  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:24.065521  6377 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Committing config change with OpId 24.14: config changed from index -1 to 14, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:24.080124  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 4e75b2c5204044d2847e78c187388fd3 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:29:24.080929  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 4e75b2c5204044d2847e78c187388fd3 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-4e75b2c5204044d2847e78c187388fd3
I20260709 12:29:24.091154  6469 tablet_copy_source_session.cc:215] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:24.097749 16647 raft_consensus.cc:2955] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Committing config change with OpId 25.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:24.100335  7329 raft_consensus.cc:2955] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:24.104072 16787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e75b2c5204044d2847e78c187388fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:24.108268 16791 ts_tablet_manager.cc:933] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:24.128262  6451 raft_consensus.cc:2955] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Committing config change with OpId 25.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:24.127966  6206 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index -1 to 14, NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) added. New cstate: current_term: 24 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } }
I20260709 12:29:24.172453  6199 catalog_manager.cc:5697] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 25 leader_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:24.218783 16791 tablet_copy_client.cc:323] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:29:24.221431  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 28020383d3c344fda935a94bca07ad82 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:29:24.222030  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 28020383d3c344fda935a94bca07ad82 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-28020383d3c344fda935a94bca07ad82
I20260709 12:29:24.237051  6469 tablet_copy_source_session.cc:215] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:24.255925 16791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 28020383d3c344fda935a94bca07ad82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:24.257766 16787 tablet_copy_client.cc:806] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:24.272306 16787 tablet_copy_client.cc:670] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:24.277557 16787 tablet_copy_client.cc:538] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:29:24.277225 16791 tablet_copy_client.cc:806] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:24.295049 16791 tablet_copy_client.cc:670] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:24.299671 16791 tablet_copy_client.cc:538] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 12:29:24.311755  5137 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 6331)
I20260709 12:29:24.313845 16787 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:29:24.332422 16791 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
W20260709 12:29:24.382382  6425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 448 similar messages]
I20260709 12:29:24.457634 16793 ts_tablet_manager.cc:933] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: Initiating tablet copy from peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:24.495090 16793 tablet_copy_client.cc:323] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Beginning tablet copy session from remote peer at address 127.5.4.67:45591
I20260709 12:29:24.499027  6469 tablet_copy_service.cc:140] P 4e0e521cb08c420f981a29576eee90a6: Received BeginTabletCopySession request for tablet 44fa024e8a7d48a6bc67d49f348bc0c0 from peer 561a4e21d8b344a79af44568ca80a3c5 ({username='slave'} at 127.0.0.1:38272)
I20260709 12:29:24.499706  6469 tablet_copy_service.cc:161] P 4e0e521cb08c420f981a29576eee90a6: Beginning new tablet copy session on tablet 44fa024e8a7d48a6bc67d49f348bc0c0 from peer 561a4e21d8b344a79af44568ca80a3c5 at {username='slave'} at 127.0.0.1:38272: session id = 561a4e21d8b344a79af44568ca80a3c5-44fa024e8a7d48a6bc67d49f348bc0c0
W20260709 12:29:24.504040  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:24.508980  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:24.524286  6469 tablet_copy_source_session.cc:215] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:29:24.542517 16793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44fa024e8a7d48a6bc67d49f348bc0c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:24.582772 16787 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:24.595340 16787 tablet_bootstrap.cc:492] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:24.610195 16787 ts_tablet_manager.cc:1403] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.297s	user 0.052s	sys 0.012s
I20260709 12:29:24.605113 16793 tablet_copy_client.cc:806] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 0 data blocks...
I20260709 12:29:24.615167 16787 raft_consensus.cc:359] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:24.620051 16787 raft_consensus.cc:740] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:24.629953 16787 consensus_queue.cc:260] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:24.633785 16787 ts_tablet_manager.cc:1434] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:29:24.627743 16793 tablet_copy_client.cc:670] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Starting download of 1 WAL segments...
I20260709 12:29:24.657307  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4e75b2c5204044d2847e78c187388fd3 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:29:24.657874  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-4e75b2c5204044d2847e78c187388fd3 on tablet 4e75b2c5204044d2847e78c187388fd3 with peer 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:24.694931 16791 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:24.695863 16793 tablet_copy_client.cc:538] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:29:24.708158 16793 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap starting.
I20260709 12:29:24.708959 16791 tablet_bootstrap.cc:492] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:24.710745 16791 ts_tablet_manager.cc:1403] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.379s	user 0.061s	sys 0.012s
I20260709 12:29:24.725883 16791 raft_consensus.cc:359] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
W20260709 12:29:24.731424  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:24.735023 16791 raft_consensus.cc:740] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:24.742801 16791 consensus_queue.cc:260] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:24.753731 16791 ts_tablet_manager.cc:1434] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
W20260709 12:29:24.765625  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:24.771050  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-28020383d3c344fda935a94bca07ad82 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:29:24.771535  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-28020383d3c344fda935a94bca07ad82 on tablet 28020383d3c344fda935a94bca07ad82 with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:29:24.788969  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 45579ad5e67d4843a6f90bf59939a2d7: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:24.867835  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:24.885016  7318 raft_consensus.cc:1217] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEARNER]: Deduplicated request from leader. Original: 23.14->[23.15-23.15]   Dedup: 23.15->[]
W20260709 12:29:24.954216  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:24.976861 16793 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:24.978101 16793 tablet_bootstrap.cc:492] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: Bootstrap complete.
I20260709 12:29:24.996006 16793 ts_tablet_manager.cc:1403] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent bootstrapping tablet: real 0.288s	user 0.036s	sys 0.029s
W20260709 12:29:25.004515  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:25.015408 16793 raft_consensus.cc:359] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:25.016535 16793 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEARNER]: Becoming Follower/Learner. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Initialized, Role: LEARNER
I20260709 12:29:25.017326 16793 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } }
I20260709 12:29:25.034708 16793 ts_tablet_manager.cc:1434] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: Time spent starting tablet: real 0.038s	user 0.012s	sys 0.000s
I20260709 12:29:25.044103  6469 tablet_copy_service.cc:342] P 4e0e521cb08c420f981a29576eee90a6: Request end of tablet copy session 561a4e21d8b344a79af44568ca80a3c5-44fa024e8a7d48a6bc67d49f348bc0c0 received from {username='slave'} at 127.0.0.1:38272
I20260709 12:29:25.044566  6469 tablet_copy_service.cc:434] P 4e0e521cb08c420f981a29576eee90a6: ending tablet copy session 561a4e21d8b344a79af44568ca80a3c5-44fa024e8a7d48a6bc67d49f348bc0c0 on tablet 44fa024e8a7d48a6bc67d49f348bc0c0 with peer 561a4e21d8b344a79af44568ca80a3c5
W20260709 12:29:25.084118  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:25.093955  7318 raft_consensus.cc:1217] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEARNER]: Deduplicated request from leader. Original: 32.18->[32.19-32.19]   Dedup: 32.19->[]
W20260709 12:29:25.108975  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:25.120215  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.125173  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:25.151304  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:25.279327  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.312248  5137 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 6331)
I20260709 12:29:25.320225 16766 raft_consensus.cc:1064] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
W20260709 12:29:25.321956  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:25.323602 16766 consensus_queue.cc:237] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:25.385370  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.389945  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:25.392654  7329 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:29:25.395741 16755 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:25.449544  6380 raft_consensus.cc:1275] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:29:25.466391  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.467088  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.472095 16804 consensus_queue.cc:1048] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:29:25.479192  7329 raft_consensus.cc:1217] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEARNER]: Deduplicated request from leader. Original: 24.13->[24.14-24.14]   Dedup: 24.14->[]
W20260709 12:29:25.496039  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.507124  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.527639  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.604776  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:25.607295  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.622108  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:25.623363 16766 raft_consensus.cc:2955] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:25.633080  6377 raft_consensus.cc:2955] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:25.653288  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.662478  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:25.685389  6206 catalog_manager.cc:5697] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 23 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:25.707093  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 44675.6us
I20260709 12:29:25.709574 16764 raft_consensus.cc:1064] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:25.713131 16764 consensus_queue.cc:237] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 17, Current term: 32, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:25.722158  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:25.722872  7329 raft_consensus.cc:2955] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 15 to 16, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:25.741997  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:25.742794  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.771975  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.772768  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:25.788671  7329 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:29:25.791124  6377 raft_consensus.cc:1275] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:29:25.791577 16804 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:29:25.793291 16801 consensus_queue.cc:1048] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:29:25.815340  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:25.836117 16801 raft_consensus.cc:2955] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 LEADER]: Committing config change with OpId 32.20: config changed from index 19 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:25.867282  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:25.866843  6377 raft_consensus.cc:2955] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Committing config change with OpId 32.20: config changed from index 19 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:25.912269  6206 catalog_manager.cc:5697] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 reported cstate change: config changed from index 19 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 32 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:25.913918  7329 raft_consensus.cc:2955] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Committing config change with OpId 32.20: config changed from index 19 to 20, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:25.989531  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:25.963184  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:26.001227  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:26.029362  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:26.030665  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:26.078845  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:26.119900 16803 raft_consensus.cc:1064] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: attempting to promote NON_VOTER 561a4e21d8b344a79af44568ca80a3c5 to VOTER
I20260709 12:29:26.134690 16803 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 12, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:26.138792  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:26.147455  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:26.167219  6377 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:26.168161  7329 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEARNER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:29:26.169107 16801 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:29:26.171973 16764 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:29:26.191347  6425 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:26.222476 16766 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Committing config change with OpId 24.15: config changed from index 14 to 15, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:26.225634  7329 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:26.258095  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:26.277804  6206 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 14 to 15, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New cstate: current_term: 24 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:26.283478  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:26.288425  6377 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Committing config change with OpId 24.15: config changed from index 14 to 15, 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } attrs { replace: true } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:26.305557  6459 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:26.312812  5137 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:26.320760  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:26.350091  7329 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:26.357090 16766 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:29:26.365707  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:29:26.375905 16803 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Committing config change with OpId 24.16: config changed from index 15 to 16, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:26.414754  7329 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 15 to 16, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:26.437078  6380 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:29:26.441432 16801 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:29:26.473294  6380 raft_consensus.cc:2955] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Committing config change with OpId 24.16: config changed from index 15 to 16, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
I20260709 12:29:26.649616  6190 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 44fa024e8a7d48a6bc67d49f348bc0c0 with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:29:26.655946  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 53144us
I20260709 12:29:26.885972  6206 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: config changed from index 15 to 16, VOTER 4268023ecea149cd83cba77db53c3100 (127.5.4.65) evicted. New cstate: current_term: 24 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:26.897037  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:26.901129  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.018572  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.077498  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:27.223834  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.320014  5137 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:27.320529  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.439308  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.473052  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.478348  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.488372  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.510819  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.641335  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.668172  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.714740  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800381180 after lost signal to thread 5140
W20260709 12:29:27.716149  6406 debug-util.cc:398] Leaking SignalData structure 0x7b08002d6dc0 after lost signal to thread 6240
W20260709 12:29:27.717787  6406 debug-util.cc:398] Leaking SignalData structure 0x7b080016a1c0 after lost signal to thread 6409
W20260709 12:29:27.735081  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.738315  6406 debug-util.cc:398] Leaking SignalData structure 0x7b08001697a0 after lost signal to thread 6483
W20260709 12:29:27.740707  6406 debug-util.cc:398] Leaking SignalData structure 0x7b0800235920 after lost signal to thread 7371
W20260709 12:29:27.751040  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.792749  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.803929  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:27.829818  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.842747  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:27.978890  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:28.024369  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.037050  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.098414  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:28.143941  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.156013  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:28.174979  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:28.183993  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.195763  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.245409  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.293061  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.301625  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.302979  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.321030  5137 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:28.385385  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.388542  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:28.389603  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:28.399994  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.436488  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.496850  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.515334  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.563525  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48232.7us
W20260709 12:29:28.575022  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.634517  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.640491  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.652181  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:28.694100  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.742290  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.796219  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:28.822352  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:29.062399  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:29.258095  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:29.321779  5137 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:29.341215  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:29.387843  6425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 472 similar messages]
W20260709 12:29:29.489805  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:29.675395  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:29.862084  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:29.912985  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:29.930744  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:29.988617  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.021036  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.073372  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.092872  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.140184  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.171672  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.191084  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.276693  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.322333  5137 thread.cc:527] Waited for 13000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:30.452363  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.454766  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.462630  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:30.508632  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.531572  6348 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 75304.8us
W20260709 12:29:30.546250  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.563084  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.549762  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.606518  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.608418  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.670759  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.677310  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.749953  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.765756  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.765200  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.776326  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:30.834360  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.922863  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.949101  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.956027  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:30.975397  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:30.988641  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.022830  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.187348  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:31.229516  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 91006.4us
W20260709 12:29:31.236537  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.280198  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.310194  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.312415  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.310794  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 59796.5us
W20260709 12:29:31.356129  5137 thread.cc:527] Waited for 14000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:31.401269  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.401984  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.426200  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.456463  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.460373  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:31.629377  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:31.657202  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:31.659044  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:32.356667  5137 thread.cc:527] Waited for 15000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:32.478243  6190 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2483d7801887403e873adc62429fe046 with cas_config_opid_index 8: ChangeConfig:REMOVE_PEER failed with leader 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 4e0e521cb08c420f981a29576eee90a6 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 25 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } } }
W20260709 12:29:32.496447  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:32.497099  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:32.524001  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:32.533962  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:32.579373  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:32.580044  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:32.602790  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:32.626309  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:32.822503  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:32.839157  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:32.869895  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:32.974103  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.002099  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.063073  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 82810.5us
W20260709 12:29:33.077287  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:33.080392  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.080928  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.081418  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.103749  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.104513  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.110478  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 45579ad5e67d4843a6f90bf59939a2d7: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:33.136124  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.144182  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.152782  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.157682  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.233229  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.257229  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.278843  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080021d8a0 after lost signal to thread 5140
W20260709 12:29:33.280264  6480 debug-util.cc:398] Leaking SignalData structure 0x7b0800341fe0 after lost signal to thread 6240
W20260709 12:29:33.281848  6480 debug-util.cc:398] Leaking SignalData structure 0x7b0800405f00 after lost signal to thread 6409
W20260709 12:29:33.283200  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080003b000 after lost signal to thread 6483
W20260709 12:29:33.284590  6480 debug-util.cc:398] Leaking SignalData structure 0x7b080021d9c0 after lost signal to thread 7371
W20260709 12:29:33.294173  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.294788  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.316380  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.357157  5137 thread.cc:527] Waited for 16000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:33.378693  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.380987  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.390296  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:33.487707  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.613770  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.637512  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.639555  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.643797  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.650712  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.881206  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.903640  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:33.936923  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:33.990832  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:34.048166  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:34.053279  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:34.094007  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:34.109018  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:34.194228  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:34.357652  5137 thread.cc:527] Waited for 17000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:34.391515  7288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 452 similar messages]
W20260709 12:29:34.499135  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 48261.7us
W20260709 12:29:34.714752  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:34.831485  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:34.876433  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:34.974611  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.012344  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.114653  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.124203  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.205730  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.237483  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.358203  5137 thread.cc:527] Waited for 18000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:35.368904  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.391757  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.393559  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.442658  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.455292  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.531402  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.562918  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.619402  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.620416  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:35.702749  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.717876  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.798887  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.832056  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.848829  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 135443us
W20260709 12:29:35.853693  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.862903  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:35.872685  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.878016  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.878720  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.893319  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.894213  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.894742  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:35.905763  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:35.960470  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:36.089093  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:36.224476  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:36.359069  5137 thread.cc:527] Waited for 19000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:36.463322  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:36.490597  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:36.494874  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:36.644116  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:36.790457  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:36.829591  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:36.993359  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 95532.5us
W20260709 12:29:36.999501  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:37.041415  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:37.042445  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:29:37.043435  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:37.239316  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:37.269531  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:37.359629  5137 thread.cc:527] Waited for 20000ms trying to join with diag-logger (tid 6331)
W20260709 12:29:37.410207  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:37.446178  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:37.442584  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:37.488492  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:37.643816  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:37.702255  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:37.827483  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:37.864308  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:37.877043  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 47869.2us
W20260709 12:29:37.888425  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:37.976982  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:37.990015  6351 consensus_peers.cc:597] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:38.043666  6351 consensus_peers.cc:597] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.044296  6351 consensus_peers.cc:597] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.140522  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.156769  6351 consensus_peers.cc:597] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.159556  6351 consensus_peers.cc:597] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.191032  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:38.264065  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.275100  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:38.284652  6351 consensus_peers.cc:597] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:38.308531  6351 consensus_peers.cc:597] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.316943  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.372618  6351 consensus_peers.cc:597] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.379513  6351 consensus_peers.cc:597] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:38.456156  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.645469  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.684549  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:38.729086  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.756865  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.773855  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:38.826894  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:38.837391  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:40.244158  5142 debug-util.cc:398] Leaking SignalData structure 0x7b08003ad0a0 after lost signal to thread 7285
W20260709 12:29:40.244910  5142 kernel_stack_watchdog.cc:198] Thread 7285 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 334ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260709 12:29:40.436128  6351 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 389 similar messages]
W20260709 12:29:40.490149  7285 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.581s	user 0.025s	sys 0.026s
W20260709 12:29:40.492865  7285 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.584s	user 0.025s	sys 0.026s
W20260709 12:29:40.500025  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 63940us
I20260709 12:29:40.505306 16898 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.506345 16898 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.555630 16898 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.575297 16912 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.576092 16912 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.577723 16898 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.578428 16898 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.580533 16898 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.580570 16906 raft_consensus.cc:493] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.581323 16906 raft_consensus.cc:515] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.584136 16912 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.584510 16906 leader_election.cc:290] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.585071 16903 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.587584 16905 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.589320 16911 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.603277 16911 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.604666 16903 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.607553 16903 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:40.607656 16911 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.603410 16905 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.610692 16904 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.612378 16905 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:40.611676 16904 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.624641 16904 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.632336 16913 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.633240 16913 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.636795 16913 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.638166 16912 raft_consensus.cc:493] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.638927 16912 raft_consensus.cc:515] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.641513 16912 leader_election.cc:290] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.642081 16912 raft_consensus.cc:493] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.642918 16912 raft_consensus.cc:515] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.645493 16912 leader_election.cc:290] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.646803 16901 raft_consensus.cc:493] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.647449 16901 raft_consensus.cc:515] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.650095 16901 leader_election.cc:290] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.650969 16901 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.651727 16901 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.655285 16901 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.655388 16925 raft_consensus.cc:493] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.656299 16898 raft_consensus.cc:493] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.665166 16902 raft_consensus.cc:493] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.666006 16905 raft_consensus.cc:493] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.666431 16908 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.668044 16918 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.668098 16904 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.668753 16926 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.676409 16920 raft_consensus.cc:493] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.670713 16919 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.681474 16898 raft_consensus.cc:515] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.681645 16904 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.680253 16925 raft_consensus.cc:515] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.672837 16906 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.695077 16927 raft_consensus.cc:493] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.688195 16913 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.681511 16908 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.690665 16920 raft_consensus.cc:515] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.681501 16926 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.698666 16898 leader_election.cc:290] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.722232 16868 raft_consensus.cc:493] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.723099 16868 raft_consensus.cc:515] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.725692 16868 leader_election.cc:290] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.713601 16927 raft_consensus.cc:515] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.679931 16905 raft_consensus.cc:515] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.737237 16916 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.738042 16916 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.678921 16902 raft_consensus.cc:515] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.688732 16919 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.751773 16904 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.752008 16923 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.712479 16906 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.755657 16868 raft_consensus.cc:493] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.740782 16916 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.756953 16914 raft_consensus.cc:493] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.752851 16917 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.680055 16918 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.752965 16935 raft_consensus.cc:493] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.739114 16934 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.767141 16868 raft_consensus.cc:515] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.718319 16913 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.741317 16933 raft_consensus.cc:493] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.753070 16908 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.753700 16901 raft_consensus.cc:493] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.785171 16937 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.804764 16868 leader_election.cc:290] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.805929 16905 leader_election.cc:290] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.806620 16933 raft_consensus.cc:515] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.807628 16928 raft_consensus.cc:493] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.808023 16910 raft_consensus.cc:493] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.808342 16928 raft_consensus.cc:515] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.808596 16910 raft_consensus.cc:515] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.809262 16933 leader_election.cc:290] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.811178 16906 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:40.811122 16916 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.811807 16908 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.812268 16916 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.812434 16908 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.813032 16920 leader_election.cc:290] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.814002 16919 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:40.815094  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 75203.8us
I20260709 12:29:40.811101 16921 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.812848 16910 leader_election.cc:290] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.775913 16912 raft_consensus.cc:493] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.814764 16926 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.814651 16916 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.807672 16927 leader_election.cc:290] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.812465 16918 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.782895 16898 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.780913 16935 raft_consensus.cc:515] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.889868 16928 leader_election.cc:290] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.898326 16926 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.837524 16901 raft_consensus.cc:515] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.898507 16929 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.776866 16923 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.898854 16919 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.914973  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:40.915302 16919 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.916225 16942 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.917109 16913 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.916920 16942 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.917560 16919 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.918131 16918 raft_consensus.cc:493] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.918551 16902 leader_election.cc:290] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.918715 16918 raft_consensus.cc:515] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.920797 16923 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.921088 16918 leader_election.cc:290] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.765743 16914 raft_consensus.cc:515] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.882246 16921 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.926857 16942 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.909595  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c413b06caa444496aecb13ac2142c246" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:40.896885 16933 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.912309 16935 leader_election.cc:290] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.906925 16908 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.907691 16925 leader_election.cc:290] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.950018 16925 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.913188 16926 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.950696 16925 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.952279 16919 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.952991 16924 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.952817 16919 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.914611 16927 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.953543 16924 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.953979 16927 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.954682 16905 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.955324 16905 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.950346  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:40.803480 16934 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.956964 16898 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.957556 16902 raft_consensus.cc:493] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.958317 16902 raft_consensus.cc:515] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.915091 16929 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.908090 16868 raft_consensus.cc:493] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.899151 16922 raft_consensus.cc:493] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.923298  6450 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:40.896584 16910 raft_consensus.cc:493] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:40.935835  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:40.945119  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.946125  7335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "150319986ee143d5a746f901c5b4adcc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.946429  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.947885 16904 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.978955 16904 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.889797 16912 raft_consensus.cc:515] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.948149  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "62bd177b33be425d81796cc414f86f76" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.985591  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f745d20b9f8c4aca9adf20ed14f509ac" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.986828 16932 raft_consensus.cc:493] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.987121  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.987766  7314 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bdd5f9c03b84f7289e750ebaab2fda6 in term 23.
I20260709 12:29:40.987557 16932 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.947901  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.899386 16906 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.974679 16868 raft_consensus.cc:515] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.026448 16906 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:41.027737 16868 leader_election.cc:290] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.975152 16922 raft_consensus.cc:515] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.028620 16906 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:40.964684 16905 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:40.976411 16910 raft_consensus.cc:515] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:40.964140 16921 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.984088 16941 raft_consensus.cc:493] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 561a4e21d8b344a79af44568ca80a3c5)
I20260709 12:29:40.979568  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.986158 16912 leader_election.cc:290] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.986117 16902 leader_election.cc:290] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.023903 16916 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:40.986981  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:40.839828 16937 raft_consensus.cc:515] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.025779 16932 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.832665 16917 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.044620 16926 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.044963  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:40.989781  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.035538  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd67024481794a7db3a98df0cd400c9d" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.030426 16922 leader_election.cc:290] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.037140 16942 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:41.038810 16916 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.039546 16898 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.030525 16929 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.039855 16910 leader_election.cc:290] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.041172 16925 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.041531 16937 leader_election.cc:290] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.042062 16919 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:41.046161  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:29:41.066887  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.050696 16917 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:41.087457 16942 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.087771  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.089079  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2ea11ad8cada4bb682c147f8a26cceb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.062817  7285 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:41.089522 16942 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:41.112779  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.050222  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b2f25afefb44300a1d6297e2cc140f5" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.067656  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "40665848f81d4ce9a753d2417536ae0a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.050321  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.052974  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.071861  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.052039  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.056404 16904 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:41.054199 16941 raft_consensus.cc:515] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.124778  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:41.086373  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:29:41.078562  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4d316e983e24c05a1bda96f3c053d46" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.052175  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.053462  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.045647  6458 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:29:41.128213 16941 leader_election.cc:290] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:40.986227 16901 leader_election.cc:290] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.054299  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:40.962509 16933 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:40.956979 16927 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:41.101291 16916 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:41.071080 16914 leader_election.cc:290] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:40.980818 16924 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:41.063686  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.131901  6454 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:41.107013  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.132913  6383 raft_consensus.cc:2468] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:29:41.134893  7285 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:41.135891 16916 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:41.136286 16916 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:41.136662 16916 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.136816  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.138072  6377 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 31.
I20260709 12:29:41.139859 16933 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:41.141209  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.108872  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.109380  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.111114  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e75b2c5204044d2847e78c187388fd3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.146237  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e44e2bcbe27f4a9192cc9acd16b966ad" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.112427  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:40.958925 16934 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.148317  6456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.113462  6378 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 25.
I20260709 12:29:41.123543  6451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "540a67ae2f134850bfa9955841e5f56a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.152405  6450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.129439  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.102735 16911 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:41.142423  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b749ec31e454255bbbba4a4bb493f28" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.142673  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe2196990ce4d60adc195d394d4c01c" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.142726  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "034bd5c36e71466da14c6a4850a920cb" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.143564  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7878a824e1e94db6a0b83ab47738f4b0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 38 candidate_status { last_received { term: 37 index: 25 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.154709  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f53146508e564f8aae279a61e6696e80" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.155922  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b776445e68c14533bcde1f784caa703a" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.156987  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdf58974e308489d9ae9a07e0a775c71" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:41.157136  7288 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.145512  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.157825  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d797adfa10541918dfc0ebf49b90784" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.144054  6459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "402a1967982740658c70049fd62be44e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.154042 16911 raft_consensus.cc:493] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:41.159029 16911 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:29:41.159219  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:41.159767  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:41.160259  6425 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:41.160727  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:29:41.161113  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:41.161876  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:41.162461  6425 leader_election.cc:336] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.162772  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:41.163054  6425 leader_election.cc:336] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.147210  6453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:41.163620  6425 leader_election.cc:336] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:41.164119  6425 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.151355  6452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d53ab19d424b4a85af2153ef24520678" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.149983  6382 raft_consensus.cc:2468] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 23.
I20260709 12:29:41.157752  7288 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.165695 16934 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.153517  6454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30efe8bf06814d23bf20818fb23139be" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:41.166975  7288 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:41.167590  7288 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.167956  7288 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:41.164604  6425 leader_election.cc:336] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.150605  6385 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 24.
W20260709 12:29:41.169286  7288 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.169728  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.172689  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a0563d158eb400f86c11eb8e04fc8fc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.174006 16934 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:29:41.176003  6425 leader_election.cc:336] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:41.176841  6425 leader_election.cc:336] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:41.177361  6425 leader_election.cc:336] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.178472 16945 raft_consensus.cc:493] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:41.179124 16945 raft_consensus.cc:515] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:41.183050 16945 leader_election.cc:290] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.189246  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2483d7801887403e873adc62429fe046" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.185947 16911 raft_consensus.cc:515] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.194553  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a0ddc1149fd4b03af2abd36909591d0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.194711  6376 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7db3e095e8b44b23b1a439899d305bd7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 36 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.195837  6378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.196116  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.196669  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e8c2e3c642dc806079ce6070d975" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.197604  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "906fbea0458f4a28bfad3b9dde8c6b35" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.197893  6380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cde8a89c1124a31810e39cb4a6c2518" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.198585  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:41.199923  6457 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.200752  6377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.201062  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.198858  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "252dcff751be4a3d948b0f83cb081100" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.199119  6379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "676a90ff0b8a4bb5ba0028f73a5d589f" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.202273  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.199368  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64dfdc039c8140e5bb288a15c2e827ce" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.200035  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.203945 16942 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.206641 16916 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.208137  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ddf1d667df04473a598939605e43a1e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.224233  7317 raft_consensus.cc:1240] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:29:41.224325  6457 raft_consensus.cc:2468] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:29:41.224573 16916 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.225143  6454 raft_consensus.cc:1240] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Rejecting Update request from peer 0bdd5f9c03b84f7289e750ebaab2fda6 for earlier term 28. Current term is 29. Ops: []
I20260709 12:29:41.225184  6424 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.226579 16927 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.197621 16911 leader_election.cc:290] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.209115  6381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
W20260709 12:29:41.204129  7288 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.228236  7288 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.209980  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4301d25f8649e496bcbcb9f954c74e" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.210739  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" is_pre_election: true
I20260709 12:29:41.231263 16938 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:41.242965  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0250a69b3914cb38265a7457b73a158" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.244294  7319 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.244652  7316 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.245318  7319 raft_consensus.cc:2393] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 29: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:41.245236  7314 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f24bf2d0bc904aa495cdb5995b470fb0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.244199  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:41.245819  7288 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.211390  7287 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.246426  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.246611  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8729b74dbaf94d7092d811f0a55205f2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.217468  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.222585  6425 leader_election.cc:304] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.244452  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4db5d1ae5f524ad799ccb78373ce3ff7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.248612  6425 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.248862 16927 raft_consensus.cc:2749] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:41.249002  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.250025 16937 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:29:41.250257 16917 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:29:41.250607  6425 leader_election.cc:304] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.250756 16917 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:41.213231  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:41.251150 16917 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:29:41.252210  6425 leader_election.cc:304] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.253428 16925 raft_consensus.cc:2749] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.255587  6424 leader_election.cc:304] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.248131 16938 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:41.257156 16929 raft_consensus.cc:2749] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.256923  6424 leader_election.cc:304] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.257956  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd3cb6d1db5d481b9589e84d04e83d8e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:41.260703  6425 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.258764 16925 raft_consensus.cc:2749] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:29:41.259805  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.252118 16919 raft_consensus.cc:2749] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:29:41.258615  6424 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.263118  7287 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.269155 16919 raft_consensus.cc:2749] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:41.269189  6458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:41.270021 16916 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:41.270427  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.271158 16919 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.272277  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.274360  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.275122 16919 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:41.275616 16916 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:41.275636  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e600ce2bd05948cf8ce5a5f292f57f28" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.276283 16916 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:41.276857 16916 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:29:41.277491  6425 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.277638  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f73da77e7bb0404e9f79f06e3dd200f4" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:41.278020  6425 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.278465  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.278817  6351 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.272383  7317 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.279378  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c2d3ed2ffee2408ebcb407323a99660b" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.280524  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.281056 16919 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:41.286535 16901 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:41.287174  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.282070  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
W20260709 12:29:41.281692  6425 leader_election.cc:336] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.283376  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "28020383d3c344fda935a94bca07ad82" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.286638 16925 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.287588  7287 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.282157  7329 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5012c92e5bb4fe188dca8f60efceeb0" candidate_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:41.288504  6425 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.288559  7285 leader_election.cc:304] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:41.289124 16904 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.289284 16919 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:41.289619 16938 raft_consensus.cc:2804] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:41.290581 16919 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:41.290463  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.290405 16938 raft_consensus.cc:697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:41.291740 16919 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:41.291961  6425 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.292387  6425 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.293078 16919 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.292397 16938 consensus_queue.cc:237] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:41.293273  6425 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.302255 16916 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.350781  6424 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.263775 16917 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:41.246381  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:41.367151  7288 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 120614us
I20260709 12:29:41.348449  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ee2be3ef27a4ec7bd5946b81b8ee8c3" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:41.364408 16919 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:41.369267  7288 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.370000  7288 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.364640  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:41.300648  7287 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.374678 16916 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.375571 16934 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:41.377023 16917 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:41.377504 16934 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:29:41.348563  6351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:43625, user_credentials={real_user=slave}} blocked reactor thread for 69579.4us
I20260709 12:29:41.378696 16934 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
W20260709 12:29:41.374095  6425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:37395, user_credentials={real_user=slave}} blocked reactor thread for 80712.1us
I20260709 12:29:41.378110  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
W20260709 12:29:41.380020  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 79299.5us
I20260709 12:29:41.380318 16934 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:29:41.385308  7288 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.395629  7288 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.380363  6351 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.398794  6384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:41.404151 16945 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.404695  6382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:41.405601  6383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:41.397583 16941 consensus_queue.cc:1059] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:29:41.408392 16941 raft_consensus.cc:3055] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:41.408967 16941 raft_consensus.cc:740] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
I20260709 12:29:41.412151 16941 consensus_queue.cc:260] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.416635 16941 raft_consensus.cc:3060] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.420600 16917 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:29:41.421746  7288 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.428835  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.425752  7287 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.431486  6351 leader_election.cc:304] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.433041 16901 raft_consensus.cc:2749] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:29:41.440481  6351 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.441715 16901 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.452337 16923 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:41.453517 16934 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:41.454810  7288 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.455914  7288 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:41.454895 16917 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:29:41.468360 16923 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:29:41.475385  7287 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.477133 16917 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:29:41.476958 16933 consensus_queue.cc:1059] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:29:41.479066  7287 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.488009 16933 raft_consensus.cc:3055] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:29:41.488600 16933 raft_consensus.cc:740] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 0bdd5f9c03b84f7289e750ebaab2fda6, State: Running, Role: LEADER
W20260709 12:29:41.491426  7366 debug-util.cc:398] Leaking SignalData structure 0x7b08003ebdc0 after lost signal to thread 5140
W20260709 12:29:41.494516  7366 debug-util.cc:398] Leaking SignalData structure 0x7b0800383940 after lost signal to thread 6240
I20260709 12:29:41.500284 16933 consensus_queue.cc:260] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.494004 16923 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.503118 16933 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.511310  6351 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.528079 16901 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:29:41.515110  7366 debug-util.cc:398] Leaking SignalData structure 0x7b08002bcd80 after lost signal to thread 6409
I20260709 12:29:41.516601  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:41.520308  6385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6"
I20260709 12:29:41.526072 16934 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:41.544721  7366 debug-util.cc:398] Leaking SignalData structure 0x7b080032fdc0 after lost signal to thread 6483
W20260709 12:29:41.546810  7366 debug-util.cc:398] Leaking SignalData structure 0x7b08003c02a0 after lost signal to thread 7371
I20260709 12:29:41.557514  6351 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.558776 16941 raft_consensus.cc:2749] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:29:41.560034  6351 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.561216 16901 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.566823  7287 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.569041 16923 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:29:41.573448  7329 raft_consensus.cc:1240] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 25. Current term is 26. Ops: []
I20260709 12:29:41.586733  6351 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.589262 16933 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.592358  6351 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.593477  6351 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.598750 16933 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:29:41.556747  6199 catalog_manager.cc:5697] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 28 to 29, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 29 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:41.602556 16901 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:41.609769  6351 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.610472  6351 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.611403 16933 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:41.613255  6351 leader_election.cc:336] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.613739  6351 leader_election.cc:304] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.614611 16933 raft_consensus.cc:2749] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:41.615945  6351 leader_election.cc:336] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.616441  7329 raft_consensus.cc:1240] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 31. Current term is 32. Ops: []
I20260709 12:29:41.616513  6351 leader_election.cc:304] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.625281 16933 raft_consensus.cc:2749] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:41.626798  6351 leader_election.cc:336] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.627332  6351 leader_election.cc:304] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.630801 16933 raft_consensus.cc:2749] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:29:41.633838  6351 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.635881  6425 leader_election.cc:304] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.641279  7331 raft_consensus.cc:1240] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Rejecting Update request from peer 4e0e521cb08c420f981a29576eee90a6 for earlier term 24. Current term is 25. Ops: []
I20260709 12:29:41.643421 16943 raft_consensus.cc:2749] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:41.646389  6425 leader_election.cc:304] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.647495 16943 raft_consensus.cc:2749] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:41.650713  6351 leader_election.cc:336] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.655108  6351 leader_election.cc:304] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.649530  6425 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.656373 16943 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:41.665832  6425 leader_election.cc:304] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.668946 16927 raft_consensus.cc:2749] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:41.666225  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
I20260709 12:29:41.666720 16933 raft_consensus.cc:2749] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:41.667902  6351 consensus_peers.cc:597] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:41.670992  6351 leader_election.cc:336] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.673914  6351 leader_election.cc:304] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.674950 16933 raft_consensus.cc:2749] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:41.676936  6351 leader_election.cc:336] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.677517  6351 leader_election.cc:304] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.673230  6425 leader_election.cc:304] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.678884 16927 raft_consensus.cc:2749] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.679894  6425 leader_election.cc:304] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.680559 16933 raft_consensus.cc:2749] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:29:41.683591 16927 raft_consensus.cc:2749] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:29:41.681990  6351 leader_election.cc:336] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.685141  6425 leader_election.cc:304] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.685421  6351 leader_election.cc:304] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.669344 16943 raft_consensus.cc:2749] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:41.686892  6425 leader_election.cc:304] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.687124 16933 raft_consensus.cc:2749] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:41.687894 16937 raft_consensus.cc:2749] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:29:41.698858  6351 leader_election.cc:336] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.699765  6351 leader_election.cc:304] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.700830 16933 raft_consensus.cc:2749] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:29:41.702309 16919 consensus_queue.cc:1059] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:29:41.703891 16919 raft_consensus.cc:3055] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:29:41.704262 16919 raft_consensus.cc:740] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:41.705461 16919 consensus_queue.cc:260] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.707008 16919 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:41.716267 16920 consensus_queue.cc:1059] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:29:41.717487 16920 raft_consensus.cc:3055] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:29:41.717868 16920 raft_consensus.cc:740] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:41.686856 16943 raft_consensus.cc:2749] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:29:41.721060 16920 consensus_queue.cc:260] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.722867 16920 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:29:41.736781  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:29:41.737636 16927 consensus_queue.cc:1059] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:29:41.738924 16927 raft_consensus.cc:3055] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:29:41.739254 16927 raft_consensus.cc:740] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:41.740324 16927 consensus_queue.cc:260] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:41.741848 16927 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:29:41.754180  6382 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:29:41.756405 16934 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:29:41.771446  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:41.781208  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:29:41.781965  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:29:41.782269  6457 raft_consensus.cc:1275] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:29:41.785260 16938 consensus_queue.cc:1048] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:29:41.853386  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:41.868494  6351 leader_election.cc:336] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.871009  6351 leader_election.cc:304] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.872076 16901 raft_consensus.cc:2749] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:41.872722  6351 leader_election.cc:336] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.873344  6351 leader_election.cc:304] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.874027 16901 raft_consensus.cc:2749] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:29:41.874454  6351 leader_election.cc:336] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.884938  6351 leader_election.cc:304] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.903015 16901 raft_consensus.cc:2749] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:41.903373  6351 leader_election.cc:336] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.903949  6351 leader_election.cc:304] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:41.904721 16901 raft_consensus.cc:2749] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:41.905038  6351 leader_election.cc:336] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.905531  6351 leader_election.cc:304] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
W20260709 12:29:41.906853  6351 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.907358 16901 raft_consensus.cc:2749] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:29:41.907987  6351 leader_election.cc:336] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:41.908811  6351 leader_election.cc:304] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0bdd5f9c03b84f7289e750ebaab2fda6; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:41.909538 16901 raft_consensus.cc:2749] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:29:41.931063  6351 consensus_peers.cc:597] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:41.931676  6351 consensus_peers.cc:597] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:41.940408  6351 consensus_peers.cc:597] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:42.076340  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:42.101356  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:42.171968  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:42.181106  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
W20260709 12:29:42.201557  6424 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.209857  7287 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.210696  7287 consensus_peers.cc:597] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.221642  7287 consensus_peers.cc:597] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.248703  7287 consensus_peers.cc:597] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.261119  6424 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.280256  6351 consensus_peers.cc:597] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:42.291682  7287 consensus_peers.cc:597] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.306690  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:42.309548  6424 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.317509  7287 consensus_peers.cc:597] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.332002  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:42.341769  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:42.343317  5137 tablet_replica.cc:333] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.344622  5137 raft_consensus.cc:2243] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.346865  5137 raft_consensus.cc:2272] T 2ddf475f266c47adb1a50aa986a0c44c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.348853  7287 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.351011  6424 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.354348  5137 tablet_replica.cc:333] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.355407  5137 raft_consensus.cc:2243] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.357379  5137 raft_consensus.cc:2272] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.360402  6424 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.369472  5137 tablet_replica.cc:333] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
W20260709 12:29:42.370541  6351 consensus_peers.cc:597] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:42.373698  6424 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.371791  5137 raft_consensus.cc:2243] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.375918  5137 raft_consensus.cc:2272] T 4cde8a89c1124a31810e39cb4a6c2518 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.385541  5137 tablet_replica.cc:333] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.386670  5137 raft_consensus.cc:2243] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.387861  5137 raft_consensus.cc:2272] T 7db3e095e8b44b23b1a439899d305bd7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.393760  5137 tablet_replica.cc:333] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.394855  5137 raft_consensus.cc:2243] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.395910  5137 raft_consensus.cc:2272] T 4a0ddc1149fd4b03af2abd36909591d0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.396677  6424 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.399964  5137 tablet_replica.cc:333] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.401125  5137 raft_consensus.cc:2243] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.406033  5137 raft_consensus.cc:2272] T 45579ad5e67d4843a6f90bf59939a2d7 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.407019  6424 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.409149  5137 tablet_replica.cc:333] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.409899  5137 raft_consensus.cc:2243] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.410534  5137 raft_consensus.cc:2272] T 30efe8bf06814d23bf20818fb23139be P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.415369  5137 tablet_replica.cc:333] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
W20260709 12:29:42.416177  7287 consensus_peers.cc:597] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.416412  5137 raft_consensus.cc:2243] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.417762  5137 raft_consensus.cc:2272] T 034bd5c36e71466da14c6a4850a920cb P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.421245  5137 tablet_replica.cc:333] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.422263  5137 raft_consensus.cc:2243] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.424081  5137 raft_consensus.cc:2272] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.427660  5137 tablet_replica.cc:333] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.428455  5137 raft_consensus.cc:2243] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.429563  5137 raft_consensus.cc:2272] T a96cfc92d6c6455eae7d1204353604ec P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.432121  5137 tablet_replica.cc:333] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.432852  5137 raft_consensus.cc:2243] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.433585  5137 raft_consensus.cc:2272] T b776445e68c14533bcde1f784caa703a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.436439  5137 tablet_replica.cc:333] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.437093  5137 raft_consensus.cc:2243] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.438241  5137 raft_consensus.cc:2272] T 2e4301d25f8649e496bcbcb9f954c74e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.441064  5137 tablet_replica.cc:333] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.441939  5137 raft_consensus.cc:2243] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.442591  5137 raft_consensus.cc:2272] T f53146508e564f8aae279a61e6696e80 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.445611  5137 tablet_replica.cc:333] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.446533  5137 raft_consensus.cc:2243] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.447911  5137 raft_consensus.cc:2272] T 676a90ff0b8a4bb5ba0028f73a5d589f P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.450877  5137 tablet_replica.cc:333] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.451759  5137 raft_consensus.cc:2243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.452636  5137 raft_consensus.cc:2272] T 2ea11ad8cada4bb682c147f8a26cceb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.452939  6424 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.455798  5137 tablet_replica.cc:333] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.456899  5137 raft_consensus.cc:2243] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.457607  5137 raft_consensus.cc:2272] T 2483d7801887403e873adc62429fe046 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.460222  6425 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:42.460848  6425 consensus_peers.cc:597] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:29:42.462163  5137 tablet_replica.cc:333] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.463693  5137 raft_consensus.cc:2243] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.465006  5137 raft_consensus.cc:2272] T 4e75b2c5204044d2847e78c187388fd3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.467971  5137 tablet_replica.cc:333] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.468796  5137 raft_consensus.cc:2243] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.469344  5137 raft_consensus.cc:2272] T 481d3efac77d41e6bba0961a140626dc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.472270  5137 tablet_replica.cc:333] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.473063  5137 raft_consensus.cc:2243] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.473805  5137 raft_consensus.cc:2272] T 4d797adfa10541918dfc0ebf49b90784 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.476711  5137 tablet_replica.cc:333] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.477649  5137 raft_consensus.cc:2243] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.479779  5137 raft_consensus.cc:2272] T 540a67ae2f134850bfa9955841e5f56a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.483003  5137 tablet_replica.cc:333] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.483767  5137 raft_consensus.cc:2243] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.485011  5137 raft_consensus.cc:2272] T 6ea0c2d7699e4559823a28a280a0e713 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.487829  5137 tablet_replica.cc:333] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.488937  5137 raft_consensus.cc:2243] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.489622  5137 raft_consensus.cc:2272] T dd3cb6d1db5d481b9589e84d04e83d8e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.493582  5137 tablet_replica.cc:333] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.494443  5137 raft_consensus.cc:2243] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.495126  5137 raft_consensus.cc:2272] T a5012c92e5bb4fe188dca8f60efceeb0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.498172  5137 tablet_replica.cc:333] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.498951  5137 raft_consensus.cc:2243] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.500357  5137 raft_consensus.cc:2272] T 7878a824e1e94db6a0b83ab47738f4b0 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.503284  5137 tablet_replica.cc:333] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.504158  5137 raft_consensus.cc:2243] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.504930  5137 raft_consensus.cc:2272] T c2d3ed2ffee2408ebcb407323a99660b P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.509200  5137 tablet_replica.cc:333] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.510133  5137 raft_consensus.cc:2243] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:29:42.510814  7287 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.511989  5137 raft_consensus.cc:2272] T f4d316e983e24c05a1bda96f3c053d46 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.515200  6424 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.515194  5137 tablet_replica.cc:333] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.516659  5137 raft_consensus.cc:2243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.518034  5137 raft_consensus.cc:2272] T 906fbea0458f4a28bfad3b9dde8c6b35 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.521278  5137 tablet_replica.cc:333] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.522127  5137 raft_consensus.cc:2243] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.523257  5137 raft_consensus.cc:2272] T 252dcff751be4a3d948b0f83cb081100 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.527773  5137 tablet_replica.cc:333] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.528609  5137 raft_consensus.cc:2243] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.530055  5137 raft_consensus.cc:2272] T cdd6e8c2e3c642dc806079ce6070d975 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.535552  6424 consensus_peers.cc:597] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.536121  5137 tablet_replica.cc:333] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.537077  5137 raft_consensus.cc:2243] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.538312  5137 raft_consensus.cc:2272] T dda9448d18794a44a75d0b570e998c3a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.541220  5137 tablet_replica.cc:333] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.542089  5137 raft_consensus.cc:2243] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.543321  5137 raft_consensus.cc:2272] T 8729b74dbaf94d7092d811f0a55205f2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.546430  5137 tablet_replica.cc:333] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.547319  5137 raft_consensus.cc:2243] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:42.547397  6425 consensus_peers.cc:597] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:29:42.548453  5137 raft_consensus.cc:2272] T d53ab19d424b4a85af2153ef24520678 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.554595  5137 tablet_replica.cc:333] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
W20260709 12:29:42.554653  6424 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.555733  5137 raft_consensus.cc:2243] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.556547  5137 raft_consensus.cc:2272] T 40665848f81d4ce9a753d2417536ae0a P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.559659  5137 tablet_replica.cc:333] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.563279  5137 raft_consensus.cc:2243] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 LEADER]: Raft consensus shutting down.
W20260709 12:29:42.564428  6424 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.566309  5137 raft_consensus.cc:2272] T 1b749ec31e454255bbbba4a4bb493f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.570817  5137 tablet_replica.cc:333] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.571815  5137 raft_consensus.cc:2243] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.572459  5137 raft_consensus.cc:2272] T e44e2bcbe27f4a9192cc9acd16b966ad P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.574396  6351 consensus_peers.cc:597] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:42.575562  5137 tablet_replica.cc:333] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.577756  5137 raft_consensus.cc:2243] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.579552  5137 raft_consensus.cc:2272] T 3a0563d158eb400f86c11eb8e04fc8fc P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.583118  5137 tablet_replica.cc:333] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.583981  5137 raft_consensus.cc:2243] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.584689  5137 raft_consensus.cc:2272] T bdf58974e308489d9ae9a07e0a775c71 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.586903  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:29:42.587678  5137 tablet_replica.cc:333] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.588368  5137 raft_consensus.cc:2243] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.588946  5137 raft_consensus.cc:2272] T 34bae875d7104f298673c223da6caf2e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.591324  5137 tablet_replica.cc:333] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.591897  5137 raft_consensus.cc:2243] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.592833  5137 raft_consensus.cc:2272] T 0fe2196990ce4d60adc195d394d4c01c P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.595825  5137 tablet_replica.cc:333] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.596647  5137 raft_consensus.cc:2243] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:42.597952  5137 raft_consensus.cc:2272] T 64dfdc039c8140e5bb288a15c2e827ce P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.600945  5137 tablet_replica.cc:333] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.601806  5137 raft_consensus.cc:2243] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.602986  5137 raft_consensus.cc:2272] T 28020383d3c344fda935a94bca07ad82 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.606019  5137 tablet_replica.cc:333] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.606858  5137 raft_consensus.cc:2243] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:42.607169  7287 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.608083  5137 raft_consensus.cc:2272] T e600ce2bd05948cf8ce5a5f292f57f28 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.611827  5137 tablet_replica.cc:333] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.612660  5137 raft_consensus.cc:2243] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.613292  5137 raft_consensus.cc:2272] T a2f5503246494eed8b91ac10cdf742e2 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.616281  5137 tablet_replica.cc:333] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.617111  5137 raft_consensus.cc:2243] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.617990  5137 raft_consensus.cc:2272] T f745d20b9f8c4aca9adf20ed14f509ac P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.621560  5137 tablet_replica.cc:333] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.622365  5137 raft_consensus.cc:2243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.623005  5137 raft_consensus.cc:2272] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.627266  5137 tablet_replica.cc:333] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.628077  5137 raft_consensus.cc:2243] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.628684  5137 raft_consensus.cc:2272] T f73da77e7bb0404e9f79f06e3dd200f4 P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.631124  6424 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:42.631667  5137 tablet_replica.cc:333] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.632476  5137 raft_consensus.cc:2243] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.633193  5137 raft_consensus.cc:2272] T 0ddf1d667df04473a598939605e43a1e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.636080  5137 tablet_replica.cc:333] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.636839  5137 raft_consensus.cc:2243] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.637473  5137 raft_consensus.cc:2272] T b18717d2a8534f54b725363ce75d49ed P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:42.640626  5137 tablet_replica.cc:333] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6: stopping tablet replica
I20260709 12:29:42.641383  5137 raft_consensus.cc:2243] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:42.642100  5137 raft_consensus.cc:2272] T 402a1967982740658c70049fd62be44e P 0bdd5f9c03b84f7289e750ebaab2fda6 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:42.647310  7287 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.697547  6424 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.720942  7287 consensus_peers.cc:597] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.737001  7287 consensus_peers.cc:597] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:42.772358  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:29:42.960634 16919 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:42.961206 16919 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:42.963593 16919 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:42.969640  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:42.970391  7331 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
W20260709 12:29:42.980119  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:42.982939  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:42.985461 16919 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:29:42.985857 16919 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:42.986202 16919 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:42.993734 16919 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:43.008307  6424 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.016526 16919 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:43.020977  6424 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.024585  7331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44fa024e8a7d48a6bc67d49f348bc0c0" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5"
I20260709 12:29:43.025355  7331 raft_consensus.cc:3060] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:29:43.033751  7331 raft_consensus.cc:2468] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 26.
W20260709 12:29:43.034138  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:43.048802  6425 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:43.049561 16919 raft_consensus.cc:2804] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:29:43.050233 16919 raft_consensus.cc:697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Becoming Leader. State: Replica: 4e0e521cb08c420f981a29576eee90a6, State: Running, Role: LEADER
I20260709 12:29:43.051110 16919 consensus_queue.cc:237] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.073261 16923 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.073817 16923 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.082521  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:43.083204  6457 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 32.
I20260709 12:29:43.085001  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.085798 16934 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:29:43.086177 16934 raft_consensus.cc:493] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:43.086558 16934 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:29:43.088521  6425 consensus_peers.cc:597] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:43.091427  6425 consensus_peers.cc:597] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:43.097676  6205 catalog_manager.cc:5697] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "4e0e521cb08c420f981a29576eee90a6" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:43.101882 16923 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:43.102924  7287 leader_election.cc:336] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.114933 16934 raft_consensus.cc:515] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:43.115934  6425 consensus_peers.cc:597] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:43.121069  6425 consensus_peers.cc:597] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:43.125487  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "34bae875d7104f298673c223da6caf2e" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:43.126080  6457 raft_consensus.cc:3060] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:29:43.128679  7287 leader_election.cc:336] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.130671 16934 leader_election.cc:290] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:43.135639  6457 raft_consensus.cc:2468] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 33.
I20260709 12:29:43.136983  7285 leader_election.cc:304] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6
I20260709 12:29:43.138051 16934 raft_consensus.cc:2804] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:29:43.142912 16934 raft_consensus.cc:697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:43.143800 16934 consensus_queue.cc:237] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:43.163847  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:43.174208  6199 catalog_manager.cc:5697] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 31 to 33, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 33 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:43.182049  6425 consensus_peers.cc:597] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:43.186131  6425 consensus_peers.cc:597] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:43.294373  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:43.426875 16919 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.427433 16919 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.431005  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:43.447225 16917 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.447765 16917 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.449860 16917 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:43.460577  6424 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.461135  6424 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:43.463019 16937 raft_consensus.cc:493] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.463572 16937 raft_consensus.cc:515] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:43.465929 16937 leader_election.cc:290] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.468654  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:43.469496  6457 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:29:43.472419 16911 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.472916 16911 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.474960  7285 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.475706 16934 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:29:43.476054 16934 raft_consensus.cc:493] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:43.476384 16934 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:43.484143 16934 raft_consensus.cc:515] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.486126 16911 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:43.489898 16919 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:43.497843  7287 leader_election.cc:336] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.507058  6457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:43.511641  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 51094us
I20260709 12:29:43.512081 16937 raft_consensus.cc:2749] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:43.510248  6425 leader_election.cc:336] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:43.514359  7288 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.539685  7285 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:43.543768 16917 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:43.553375 16934 leader_election.cc:290] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:43.558065 16919 raft_consensus.cc:493] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.558754 16919 raft_consensus.cc:515] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.561368 16919 leader_election.cc:290] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
W20260709 12:29:43.563125  7288 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:43.563489  7287 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.587077  7287 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 71116.8us
I20260709 12:29:43.592293  7320 raft_consensus.cc:1275] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Refusing update from remote peer 4e0e521cb08c420f981a29576eee90a6: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:29:43.567164 16937 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:43.573675 16962 raft_consensus.cc:493] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.593509 16962 raft_consensus.cc:515] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:43.595641 16962 leader_election.cc:290] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.598770 16937 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.601402 16937 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68:43625)
I20260709 12:29:43.608382 16911 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.608955 16911 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.612203 16911 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:43.576829  6424 leader_election.cc:336] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.565047  7285 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:45591, user_credentials={real_user=slave}} blocked reactor thread for 46942.7us
I20260709 12:29:43.616863  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "481d3efac77d41e6bba0961a140626dc" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:43.617439  6455 raft_consensus.cc:3060] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:29:43.628575  6455 raft_consensus.cc:2468] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 30.
I20260709 12:29:43.634335  7285 leader_election.cc:304] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.636482  6424 leader_election.cc:304] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:43.638688  6425 consensus_peers.cc:597] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:29:43.639201 16923 raft_consensus.cc:2804] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:29:43.646920 16923 raft_consensus.cc:697] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:43.648205 16919 raft_consensus.cc:2749] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:29:43.649251 16937 raft_consensus.cc:493] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.648952 16923 consensus_queue.cc:237] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.649806 16937 raft_consensus.cc:515] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:43.640203 16934 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:43.641922  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 65162.7us
I20260709 12:29:43.652895 16934 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:43.653486  6424 consensus_peers.cc:597] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.645488  7287 leader_election.cc:336] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.655664 16934 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:43.654572  6425 consensus_peers.cc:597] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:43.657168  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
I20260709 12:29:43.657857  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 26.
I20260709 12:29:43.659109  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.659960 16934 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:29:43.660372 16934 raft_consensus.cc:493] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:43.661588 16934 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:29:43.663388 16937 leader_election.cc:290] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.668644  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "521c92260b1b4a6abba1ce9d3c7c5ad2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:43.669247  7320 raft_consensus.cc:2468] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e0e521cb08c420f981a29576eee90a6 in term 25.
W20260709 12:29:43.673868  6425 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.674785 16962 consensus_queue.cc:1048] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:29:43.691644  6425 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:43.692891  6425 consensus_peers.cc:597] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:29:43.694020  6425 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.695604 16934 raft_consensus.cc:515] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.715324 16970 raft_consensus.cc:493] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.715845 16970 raft_consensus.cc:515] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:43.717845 16970 leader_election.cc:290] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:43.718817  6424 leader_election.cc:336] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.719271  6424 leader_election.cc:304] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.720539 16934 leader_election.cc:290] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:43.721686  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:43.725234  6455 raft_consensus.cc:3060] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:29:43.732775 16971 log.cc:927] Time spent T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:29:43.738251  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:43.738782  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.740059  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f5503246494eed8b91ac10cdf742e2" candidate_uuid: "4e0e521cb08c420f981a29576eee90a6" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "561a4e21d8b344a79af44568ca80a3c5" is_pre_election: true
I20260709 12:29:43.740813  7320 raft_consensus.cc:2393] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4e0e521cb08c420f981a29576eee90a6 in current term 27: Already voted for candidate 561a4e21d8b344a79af44568ca80a3c5 in this term.
I20260709 12:29:43.746023 16970 raft_consensus.cc:2749] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:43.746438  6199 catalog_manager.cc:5697] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 28 to 30, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 30 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:43.751883 16970 raft_consensus.cc:493] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.752473 16970 raft_consensus.cc:515] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:43.758080  6425 leader_election.cc:336] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.759101 16970 leader_election.cc:290] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.761443 16972 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.762106 16972 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.766427 16972 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:43.771948  6424 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:34641, user_credentials={real_user=slave}} blocked reactor thread for 53149.1us
W20260709 12:29:43.779728  7287 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.781316  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.784266 16973 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:29:43.783022  6455 raft_consensus.cc:2468] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 27.
W20260709 12:29:43.785017  6425 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.785907  7285 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.786756 16973 raft_consensus.cc:2804] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:29:43.787276 16973 raft_consensus.cc:697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:43.788200 16973 consensus_queue.cc:237] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:43.801895  7288 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:43.804226  7287 leader_election.cc:336] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.814144  7287 consensus_peers.cc:597] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:43.821272  6425 consensus_peers.cc:597] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:43.831678  6424 leader_election.cc:336] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.832163  6424 leader_election.cc:304] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.835023 16973 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.835526 16973 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.836841  6455 raft_consensus.cc:1275] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Refusing update from remote peer 561a4e21d8b344a79af44568ca80a3c5: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:29:43.838502  6199 catalog_manager.cc:5697] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 25 to 27, leader changed from 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 27 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:29:43.848328 16972 consensus_queue.cc:1048] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
W20260709 12:29:43.849704  7287 leader_election.cc:336] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.850627  7287 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.851006  7287 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.851666 16970 raft_consensus.cc:2749] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:43.854665  6424 leader_election.cc:336] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.855067  6424 leader_election.cc:304] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 561a4e21d8b344a79af44568ca80a3c5
I20260709 12:29:43.855741 16970 raft_consensus.cc:2749] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:43.856194  6424 leader_election.cc:336] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.856626  6424 leader_election.cc:304] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.857317 16970 raft_consensus.cc:2749] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:29:43.843904 16973 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.858127 16923 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:43.859330 16972 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.859849 16972 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.861860 16972 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.870857 16973 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.871451 16973 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
W20260709 12:29:43.872896  6425 consensus_peers.cc:597] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:43.877997  7288 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:43.878520  7288 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:43.890329  6424 leader_election.cc:336] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.890844  6424 leader_election.cc:304] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.893138 16923 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.893687 16923 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.895798 16923 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:43.898783  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
I20260709 12:29:43.897033 16923 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.901345 16923 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.902748 16919 raft_consensus.cc:2749] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:43.903935 16923 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.906838  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6" is_pre_election: true
W20260709 12:29:43.908632  7287 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.911597  6424 leader_election.cc:336] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.912052  6424 leader_election.cc:304] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.909133  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.914312 16911 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:29:43.914613  7287 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.915198  7287 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.915961 16911 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:43.916633 16919 raft_consensus.cc:2749] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:43.917481 16919 raft_consensus.cc:493] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.917979 16919 raft_consensus.cc:515] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
I20260709 12:29:43.912448  6455 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 28.
I20260709 12:29:43.920727  7285 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.921823 16934 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:29:43.922216 16934 raft_consensus.cc:493] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.922746 16934 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:29:43.924872 16917 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.925467 16917 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.927785 16917 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.931042 16970 raft_consensus.cc:493] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:43.931550 16970 raft_consensus.cc:515] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:43.933271  7288 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.933562 16970 leader_election.cc:290] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.938069 16919 leader_election.cc:290] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:43.939518  6424 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.943370  6424 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:43.947235  6425 leader_election.cc:336] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.949208  6425 leader_election.cc:304] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.950163 16919 raft_consensus.cc:2749] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:29:43.953944  6425 leader_election.cc:336] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:43.954378  7288 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.954459  6425 leader_election.cc:304] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:43.964857  7288 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:43.967396 16970 raft_consensus.cc:2749] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:29:43.965461 16973 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:43.965840 16934 raft_consensus.cc:515] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:43.973008 16934 leader_election.cc:290] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Requested vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:43.965992  7287 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.975876  6455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "45579ad5e67d4843a6f90bf59939a2d7" candidate_uuid: "561a4e21d8b344a79af44568ca80a3c5" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4e0e521cb08c420f981a29576eee90a6"
I20260709 12:29:43.978772  6455 raft_consensus.cc:3060] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:29:43.984201  7287 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:43.984761  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:43.988615 16934 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:29:43.995568  6455 raft_consensus.cc:2468] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561a4e21d8b344a79af44568ca80a3c5 in term 29.
I20260709 12:29:43.997253  7285 leader_election.cc:304] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6, 561a4e21d8b344a79af44568ca80a3c5; no voters: 
I20260709 12:29:43.998303 16973 raft_consensus.cc:2804] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:29:43.999754 16973 raft_consensus.cc:697] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Becoming Leader. State: Replica: 561a4e21d8b344a79af44568ca80a3c5, State: Running, Role: LEADER
I20260709 12:29:44.000902 16973 consensus_queue.cc:237] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:44.004096 16970 raft_consensus.cc:493] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:44.004884 16970 raft_consensus.cc:515] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } }
W20260709 12:29:44.012598  6424 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:44.015659 16970 leader_election.cc:290] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:44.020047  6425 leader_election.cc:336] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:44.020512  6425 leader_election.cc:304] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4e0e521cb08c420f981a29576eee90a6; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:44.021672 16970 raft_consensus.cc:2749] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:29:44.022241  7287 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:44.023262  7287 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:44.032359  7287 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:44.036708  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:44.033203 16934 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:29:44.039041 16934 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:29:44.040335  7287 leader_election.cc:336] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:44.057088  6199 catalog_manager.cc:5697] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 reported cstate change: term changed from 28 to 29, leader changed from 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66) to 561a4e21d8b344a79af44568ca80a3c5 (127.5.4.68). New cstate: current_term: 29 leader_uuid: "561a4e21d8b344a79af44568ca80a3c5" committed_config { opid_index: 21 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:29:44.156404  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:44.176405  6425 consensus_peers.cc:597] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:44.201344  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
W20260709 12:29:44.207908  7285 consensus_peers.cc:597] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.211390  7285 consensus_peers.cc:597] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.222316  7285 consensus_peers.cc:597] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.231266  7285 consensus_peers.cc:597] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.231859  7285 consensus_peers.cc:597] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.240115  7285 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.243917  7287 consensus_peers.cc:597] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.261221  6425 consensus_peers.cc:597] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:29:44.284325  7285 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.286038  7285 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.340234  7285 consensus_peers.cc:597] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.346011  7287 consensus_peers.cc:597] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.340837  7285 consensus_peers.cc:597] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.364423  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:44.365859  5137 tablet_replica.cc:333] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.367913  5137 raft_consensus.cc:2243] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.368919  5137 raft_consensus.cc:2272] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.393620  7285 consensus_peers.cc:597] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:44.399053  7285 consensus_peers.cc:597] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.396531  5137 tablet_replica.cc:333] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.400496  5137 raft_consensus.cc:2243] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.401312  5137 raft_consensus.cc:2272] T 150319986ee143d5a746f901c5b4adcc P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.404618  5137 tablet_replica.cc:333] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.407511  5137 raft_consensus.cc:2243] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.408459  5137 raft_consensus.cc:2272] T 7db3e095e8b44b23b1a439899d305bd7 P 4e0e521cb08c420f981a29576eee90a6 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.411649  5137 tablet_replica.cc:333] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
W20260709 12:29:44.416965  7285 consensus_peers.cc:597] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.418447  5137 raft_consensus.cc:2243] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.419251  5137 raft_consensus.cc:2272] T 4a0ddc1149fd4b03af2abd36909591d0 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.422567  5137 tablet_replica.cc:333] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.423465  5137 raft_consensus.cc:2243] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.424120  5137 raft_consensus.cc:2272] T 4cde8a89c1124a31810e39cb4a6c2518 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.432507  5137 tablet_replica.cc:333] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
W20260709 12:29:44.434752  6425 consensus_peers.cc:597] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:29:44.435981  5137 raft_consensus.cc:2243] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:44.436339  7285 consensus_peers.cc:597] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.436726  5137 raft_consensus.cc:2272] T 55b778a085744d27a1adc35598438230 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.439883  5137 tablet_replica.cc:333] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
W20260709 12:29:44.440254  7287 consensus_peers.cc:597] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.440762  5137 raft_consensus.cc:2243] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.441923  5137 raft_consensus.cc:2272] T 45579ad5e67d4843a6f90bf59939a2d7 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.445108  5137 tablet_replica.cc:333] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.445990  5137 raft_consensus.cc:2243] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 22 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.447111  5137 raft_consensus.cc:2272] T 30efe8bf06814d23bf20818fb23139be P 4e0e521cb08c420f981a29576eee90a6 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.449867  5137 tablet_replica.cc:333] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.450727  5137 raft_consensus.cc:2243] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.451896  5137 raft_consensus.cc:2272] T 3b2f25afefb44300a1d6297e2cc140f5 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.454866  5137 tablet_replica.cc:333] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.455776  5137 raft_consensus.cc:2243] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.456823  5137 raft_consensus.cc:2272] T b776445e68c14533bcde1f784caa703a P 4e0e521cb08c420f981a29576eee90a6 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.459818  5137 tablet_replica.cc:333] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.460671  5137 raft_consensus.cc:2243] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:44.461601  7285 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.467610  5137 raft_consensus.cc:2272] T 2e4301d25f8649e496bcbcb9f954c74e P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.470722  5137 tablet_replica.cc:333] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.471606  5137 raft_consensus.cc:2243] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.472357  5137 raft_consensus.cc:2272] T f24bf2d0bc904aa495cdb5995b470fb0 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.475208  5137 tablet_replica.cc:333] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.476034  5137 raft_consensus.cc:2243] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.477101  5137 raft_consensus.cc:2272] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.479669  7285 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.481353  5137 tablet_replica.cc:333] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.482309  5137 raft_consensus.cc:2243] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.483188  5137 raft_consensus.cc:2272] T 4d797adfa10541918dfc0ebf49b90784 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.486022  5137 tablet_replica.cc:333] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.486873  5137 raft_consensus.cc:2243] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.488312  5137 raft_consensus.cc:2272] T 2483d7801887403e873adc62429fe046 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.491219  5137 tablet_replica.cc:333] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.492049  5137 raft_consensus.cc:2243] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.493750  5137 raft_consensus.cc:2272] T 4e75b2c5204044d2847e78c187388fd3 P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.496392  5137 tablet_replica.cc:333] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.497237  5137 raft_consensus.cc:2243] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.498229  5137 raft_consensus.cc:2272] T f53146508e564f8aae279a61e6696e80 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.500849  5137 tablet_replica.cc:333] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.501571  5137 raft_consensus.cc:2243] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:44.504794  7288 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:29:44.508987  5137 raft_consensus.cc:2272] T 676a90ff0b8a4bb5ba0028f73a5d589f P 4e0e521cb08c420f981a29576eee90a6 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.511822  5137 tablet_replica.cc:333] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.512641  5137 raft_consensus.cc:2243] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.513662  5137 raft_consensus.cc:2272] T 2ea11ad8cada4bb682c147f8a26cceb0 P 4e0e521cb08c420f981a29576eee90a6 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.516445  5137 tablet_replica.cc:333] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.517308  5137 raft_consensus.cc:2243] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.518527  5137 raft_consensus.cc:2272] T dd3cb6d1db5d481b9589e84d04e83d8e P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.521463  5137 tablet_replica.cc:333] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.522277  5137 raft_consensus.cc:2243] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.523128  5137 raft_consensus.cc:2272] T a5012c92e5bb4fe188dca8f60efceeb0 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.525928  5137 tablet_replica.cc:333] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.526733  5137 raft_consensus.cc:2243] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.527314  5137 raft_consensus.cc:2272] T 906fbea0458f4a28bfad3b9dde8c6b35 P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.530303  5137 tablet_replica.cc:333] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.531172  5137 raft_consensus.cc:2243] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.531785  5137 raft_consensus.cc:2272] T 252dcff751be4a3d948b0f83cb081100 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.534646  5137 tablet_replica.cc:333] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.535401  5137 raft_consensus.cc:2243] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.536067  5137 raft_consensus.cc:2272] T cdd6e8c2e3c642dc806079ce6070d975 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.537675  7285 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.538975  5137 tablet_replica.cc:333] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.539846  5137 raft_consensus.cc:2243] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.540508  5137 raft_consensus.cc:2272] T eede017c955d4746bbbd40d9384932e7 P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.543386  5137 tablet_replica.cc:333] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.544171  5137 raft_consensus.cc:2243] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Raft consensus shutting down.
W20260709 12:29:44.544689  7285 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.545517  5137 raft_consensus.cc:2272] T c413b06caa444496aecb13ac2142c246 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.548658  5137 tablet_replica.cc:333] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.549525  5137 raft_consensus.cc:2243] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.550606  5137 raft_consensus.cc:2272] T 62bd177b33be425d81796cc414f86f76 P 4e0e521cb08c420f981a29576eee90a6 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.552896  7285 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.553495  5137 tablet_replica.cc:333] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.554420  5137 raft_consensus.cc:2243] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.555523  5137 raft_consensus.cc:2272] T 481d3efac77d41e6bba0961a140626dc P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.558599  5137 tablet_replica.cc:333] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.559401  5137 raft_consensus.cc:2243] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.559963  5137 raft_consensus.cc:2272] T 34bae875d7104f298673c223da6caf2e P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.563396  5137 tablet_replica.cc:333] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.564399  5137 raft_consensus.cc:2243] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.565717  5137 raft_consensus.cc:2272] T 40665848f81d4ce9a753d2417536ae0a P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.568715  5137 tablet_replica.cc:333] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.569581  5137 raft_consensus.cc:2243] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.570780  5137 raft_consensus.cc:2272] T 540a67ae2f134850bfa9955841e5f56a P 4e0e521cb08c420f981a29576eee90a6 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.573539  5137 tablet_replica.cc:333] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.574329  5137 raft_consensus.cc:2243] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.575115  5137 raft_consensus.cc:2272] T 8729b74dbaf94d7092d811f0a55205f2 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.577883  5137 tablet_replica.cc:333] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.578657  5137 raft_consensus.cc:2243] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.579950  5137 raft_consensus.cc:2272] T d53ab19d424b4a85af2153ef24520678 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.582608  5137 tablet_replica.cc:333] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.583367  5137 raft_consensus.cc:2243] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.584153  5137 raft_consensus.cc:2272] T cd67024481794a7db3a98df0cd400c9d P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.587044  5137 tablet_replica.cc:333] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.587764  5137 raft_consensus.cc:2243] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.588899  5137 raft_consensus.cc:2272] T e44e2bcbe27f4a9192cc9acd16b966ad P 4e0e521cb08c420f981a29576eee90a6 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.592726  5137 tablet_replica.cc:333] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.594292  5137 raft_consensus.cc:2243] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.595901  5137 raft_consensus.cc:2272] T bdf58974e308489d9ae9a07e0a775c71 P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.599514  5137 tablet_replica.cc:333] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.600420  5137 raft_consensus.cc:2243] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.601224  5137 raft_consensus.cc:2272] T 3a0563d158eb400f86c11eb8e04fc8fc P 4e0e521cb08c420f981a29576eee90a6 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.604480  5137 tablet_replica.cc:333] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.605392  5137 raft_consensus.cc:2243] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.606353  5137 raft_consensus.cc:2272] T c2d3ed2ffee2408ebcb407323a99660b P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.608013  7287 consensus_peers.cc:597] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:44.609686  5137 tablet_replica.cc:333] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.610664  5137 raft_consensus.cc:2243] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.611333  5137 raft_consensus.cc:2272] T f0250a69b3914cb38265a7457b73a158 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.614948  5137 tablet_replica.cc:333] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.615789  5137 raft_consensus.cc:2243] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.616549  5137 raft_consensus.cc:2272] T e600ce2bd05948cf8ce5a5f292f57f28 P 4e0e521cb08c420f981a29576eee90a6 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.619822  5137 tablet_replica.cc:333] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.620647  5137 raft_consensus.cc:2243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.622305  5137 raft_consensus.cc:2272] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.622957  7285 consensus_peers.cc:597] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Couldn't send request to peer 4e0e521cb08c420f981a29576eee90a6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:44.625488  5137 tablet_replica.cc:333] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.626446  5137 raft_consensus.cc:2243] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.627025  5137 raft_consensus.cc:2272] T a2f5503246494eed8b91ac10cdf742e2 P 4e0e521cb08c420f981a29576eee90a6 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.630128  5137 tablet_replica.cc:333] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.630959  5137 raft_consensus.cc:2243] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.631557  5137 raft_consensus.cc:2272] T 64dfdc039c8140e5bb288a15c2e827ce P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.634706  5137 tablet_replica.cc:333] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.635550  5137 raft_consensus.cc:2243] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.637562  5137 raft_consensus.cc:2272] T 28020383d3c344fda935a94bca07ad82 P 4e0e521cb08c420f981a29576eee90a6 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.640847  5137 tablet_replica.cc:333] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.641626  5137 raft_consensus.cc:2243] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.642961  5137 raft_consensus.cc:2272] T 402a1967982740658c70049fd62be44e P 4e0e521cb08c420f981a29576eee90a6 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.646028  5137 tablet_replica.cc:333] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.646819  5137 raft_consensus.cc:2243] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:44.647552  5137 raft_consensus.cc:2272] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 4e0e521cb08c420f981a29576eee90a6 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:44.650382  5137 tablet_replica.cc:333] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6: stopping tablet replica
I20260709 12:29:44.651106  5137 raft_consensus.cc:2243] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:44.652268  5137 raft_consensus.cc:2272] T b18717d2a8534f54b725363ce75d49ed P 4e0e521cb08c420f981a29576eee90a6 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:44.708824  7288 consensus_peers.cc:597] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:44.709390  7288 consensus_peers.cc:597] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:44.745705  7287 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.777222  7287 consensus_peers.cc:597] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.801232  7287 consensus_peers.cc:597] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.814525  7287 consensus_peers.cc:597] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.839630  7288 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:44.852509  7287 consensus_peers.cc:597] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.882978  7288 consensus_peers.cc:597] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:44.905633  7287 consensus_peers.cc:597] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.916023  7287 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.933224  7287 consensus_peers.cc:597] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:45.004837  7288 consensus_peers.cc:597] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 12:29:45.099524  7287 consensus_peers.cc:597] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:45.137847  7287 consensus_peers.cc:597] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:45.138835  7288 consensus_peers.cc:597] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:29:45.403048  7287 consensus_peers.cc:597] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:45.450058  7288 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 607 similar messages]
I20260709 12:29:45.520444 16911 raft_consensus.cc:493] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:45.521147 16911 raft_consensus.cc:515] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.524749 16911 leader_election.cc:290] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:45.531684  7288 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.536237  7287 leader_election.cc:336] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.536888  7287 leader_election.cc:304] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 38 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:45.537782 16911 raft_consensus.cc:2749] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Leader pre-election lost for term 38. Reason: could not achieve majority
I20260709 12:29:45.563853 16934 raft_consensus.cc:493] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:45.564522 16934 raft_consensus.cc:515] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.567032 16934 leader_election.cc:290] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:45.567709  7288 consensus_peers.cc:597] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:45.583757  7288 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.605654  7288 consensus_peers.cc:597] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:45.627918  7287 consensus_peers.cc:597] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Couldn't send request to peer 0bdd5f9c03b84f7289e750ebaab2fda6. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:45.628602  7287 leader_election.cc:336] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.629010  7287 leader_election.cc:304] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:45.640235 16911 raft_consensus.cc:2749] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:29:45.643600 16934 raft_consensus.cc:493] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:45.644167 16934 raft_consensus.cc:515] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.649775 16934 leader_election.cc:290] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:45.657742  7288 consensus_peers.cc:597] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:45.676340  7288 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.678763  7287 leader_election.cc:336] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.679250  7287 leader_election.cc:304] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:45.680042 16934 raft_consensus.cc:2749] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:29:45.763432 16934 raft_consensus.cc:493] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:45.764050 16934 raft_consensus.cc:515] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 20 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.767742 16934 leader_election.cc:290] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:45.774159  7288 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:45.781941 16911 raft_consensus.cc:493] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:45.782691 16911 raft_consensus.cc:515] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.785339 16911 leader_election.cc:290] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:45.794014 16934 raft_consensus.cc:493] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:45.794792 16934 raft_consensus.cc:515] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.796813 16934 leader_election.cc:290] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:45.805766 16911 raft_consensus.cc:493] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:45.806331 16911 raft_consensus.cc:515] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.808526 16911 leader_election.cc:290] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:45.804836  7287 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.812063 16911 raft_consensus.cc:493] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:45.812676 16911 raft_consensus.cc:515] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } attrs { replace: true } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.815729 16911 leader_election.cc:290] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
I20260709 12:29:45.810359  7287 leader_election.cc:304] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:45.819343 16934 raft_consensus.cc:2749] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:29:45.811482  7288 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.832139  7287 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.832617  7287 leader_election.cc:304] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:45.835023 16973 raft_consensus.cc:493] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:45.835632 16973 raft_consensus.cc:515] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.848805 16973 leader_election.cc:290] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:45.849581  7285 leader_election.cc:336] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
W20260709 12:29:45.850764  7288 consensus_peers.cc:597] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 12:29:45.854022  7285 leader_election.cc:336] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
I20260709 12:29:45.855087 16911 raft_consensus.cc:2749] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:45.865854  7285 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
W20260709 12:29:45.867504  7288 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.868111  7288 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.872287  7287 leader_election.cc:336] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.872762  7287 leader_election.cc:304] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:45.878803 16911 raft_consensus.cc:2749] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:29:45.880080  7287 leader_election.cc:336] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:45.881138  7285 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
I20260709 12:29:45.881609  7285 leader_election.cc:304] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:45.882790  7287 leader_election.cc:304] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:45.882400  7288 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:45.885047 16934 raft_consensus.cc:2749] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:29:45.886222  7287 leader_election.cc:336] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.895886 16973 raft_consensus.cc:493] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:45.896522 16973 raft_consensus.cc:515] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.898823 16973 leader_election.cc:290] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
I20260709 12:29:45.889705 16911 raft_consensus.cc:2749] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:29:45.923226  7288 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:29:45.927210  7285 leader_election.cc:336] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
I20260709 12:29:45.933141  7285 leader_election.cc:304] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:45.935976 16911 raft_consensus.cc:2749] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:29:45.938264  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:29:45.943167 16911 raft_consensus.cc:493] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4e0e521cb08c420f981a29576eee90a6)
I20260709 12:29:45.945111 16911 raft_consensus.cc:515] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:45.951133 16911 leader_election.cc:290] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591)
W20260709 12:29:45.957824  7287 leader_election.cc:336] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
I20260709 12:29:45.958317  7287 leader_election.cc:304] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
W20260709 12:29:45.960247  7285 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
I20260709 12:29:45.962004 16934 raft_consensus.cc:2749] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:29:45.967981  7288 leader_election.cc:336] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:45.968511  7288 leader_election.cc:304] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 4268023ecea149cd83cba77db53c3100, 4e0e521cb08c420f981a29576eee90a6
I20260709 12:29:45.969278 16934 raft_consensus.cc:2749] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:29:45.975066  7288 consensus_peers.cc:597] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 -> Peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Couldn't send request to peer 4268023ecea149cd83cba77db53c3100. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 12:29:46.035372 16934 raft_consensus.cc:493] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:46.036001 16934 raft_consensus.cc:515] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:46.039518 16934 leader_election.cc:290] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:46.045576  7287 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:46.047348  7288 leader_election.cc:336] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:46.049213  7288 leader_election.cc:304] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:46.050096 16973 raft_consensus.cc:2749] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:29:46.075618  5137 tablet_server.cc:179] TabletServer@127.5.4.68:0 shutting down...
I20260709 12:29:46.106273 16973 raft_consensus.cc:493] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 0bdd5f9c03b84f7289e750ebaab2fda6)
I20260709 12:29:46.107080 16973 raft_consensus.cc:515] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: false } }
I20260709 12:29:46.109655 16973 leader_election.cc:290] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395), 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641)
W20260709 12:29:46.123147  7287 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 0bdd5f9c03b84f7289e750ebaab2fda6 (127.5.4.66:34641): Network error: Client connection negotiation failed: client connection to 127.5.4.66:34641: connect: Connection refused (error 111)
W20260709 12:29:46.124250  7288 leader_election.cc:336] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
I20260709 12:29:46.124807  7288 leader_election.cc:304] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 561a4e21d8b344a79af44568ca80a3c5; no voters: 0bdd5f9c03b84f7289e750ebaab2fda6, 4268023ecea149cd83cba77db53c3100
I20260709 12:29:46.125739 16973 raft_consensus.cc:2749] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:29:46.144620  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:46.145728  5137 tablet_replica.cc:333] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.146771  5137 raft_consensus.cc:2243] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.147370  5137 raft_consensus.cc:2272] T 44fa024e8a7d48a6bc67d49f348bc0c0 P 561a4e21d8b344a79af44568ca80a3c5 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.150336  5137 tablet_replica.cc:333] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.151176  5137 raft_consensus.cc:2243] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.151811  5137 raft_consensus.cc:2272] T 4e75b2c5204044d2847e78c187388fd3 P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.154708  5137 tablet_replica.cc:333] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.155509  5137 raft_consensus.cc:2243] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.156587  5137 pending_rounds.cc:70] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Trying to abort 1 pending ops.
I20260709 12:29:46.156845  5137 pending_rounds.cc:77] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5: Aborting op as it isn't in flight: id { term: 30 index: 19 } timestamp: 7305626352236036096 op_type: NO_OP noop_request { }
I20260709 12:29:46.157238  5137 raft_consensus.cc:2889] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:46.157562  5137 raft_consensus.cc:2272] T 481d3efac77d41e6bba0961a140626dc P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.160291  5137 tablet_replica.cc:333] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.161123  5137 raft_consensus.cc:2243] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.162356  5137 raft_consensus.cc:2272] T a96cfc92d6c6455eae7d1204353604ec P 561a4e21d8b344a79af44568ca80a3c5 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.165331  5137 tablet_replica.cc:333] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.166105  5137 raft_consensus.cc:2243] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.167218  5137 pending_rounds.cc:70] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Trying to abort 1 pending ops.
I20260709 12:29:46.167462  5137 pending_rounds.cc:77] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5: Aborting op as it isn't in flight: id { term: 29 index: 22 } timestamp: 7305626353809313792 op_type: NO_OP noop_request { }
I20260709 12:29:46.167902  5137 raft_consensus.cc:2889] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:46.168325  5137 raft_consensus.cc:2272] T 45579ad5e67d4843a6f90bf59939a2d7 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.171841  5137 tablet_replica.cc:333] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.172794  5137 raft_consensus.cc:2243] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.173870  5137 raft_consensus.cc:2272] T 1ee2be3ef27a4ec7bd5946b81b8ee8c3 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.176779  5137 tablet_replica.cc:333] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.177596  5137 raft_consensus.cc:2243] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.178535  5137 raft_consensus.cc:2272] T 521c92260b1b4a6abba1ce9d3c7c5ad2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.181600  5137 tablet_replica.cc:333] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.182458  5137 raft_consensus.cc:2243] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.183403  5137 raft_consensus.cc:2272] T 2483d7801887403e873adc62429fe046 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.186249  5137 tablet_replica.cc:333] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.187016  5137 raft_consensus.cc:2243] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.188021  5137 raft_consensus.cc:2272] T 4db5d1ae5f524ad799ccb78373ce3ff7 P 561a4e21d8b344a79af44568ca80a3c5 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.190467  5137 tablet_replica.cc:333] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.191246  5137 raft_consensus.cc:2243] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.192260  5137 raft_consensus.cc:2272] T 0ddf1d667df04473a598939605e43a1e P 561a4e21d8b344a79af44568ca80a3c5 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.194691  5137 tablet_replica.cc:333] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.195412  5137 raft_consensus.cc:2243] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.196372  5137 raft_consensus.cc:2272] T a5012c92e5bb4fe188dca8f60efceeb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.199049  5137 tablet_replica.cc:333] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.199806  5137 raft_consensus.cc:2243] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.200822  5137 raft_consensus.cc:2272] T eede017c955d4746bbbd40d9384932e7 P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.203244  5137 tablet_replica.cc:333] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.204031  5137 raft_consensus.cc:2243] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.204963  5137 raft_consensus.cc:2272] T 6ea0c2d7699e4559823a28a280a0e713 P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.207765  5137 tablet_replica.cc:333] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.208520  5137 raft_consensus.cc:2243] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.209578  5137 raft_consensus.cc:2272] T 62bd177b33be425d81796cc414f86f76 P 561a4e21d8b344a79af44568ca80a3c5 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.212183  5137 tablet_replica.cc:333] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.212989  5137 raft_consensus.cc:2243] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.213740  5137 raft_consensus.cc:2272] T f4d316e983e24c05a1bda96f3c053d46 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.216560  5137 tablet_replica.cc:333] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.217245  5137 raft_consensus.cc:2243] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.218293  5137 raft_consensus.cc:2272] T c2d3ed2ffee2408ebcb407323a99660b P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.221088  5137 tablet_replica.cc:333] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.221863  5137 raft_consensus.cc:2243] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.223057  5137 pending_rounds.cc:70] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Trying to abort 1 pending ops.
I20260709 12:29:46.223300  5137 pending_rounds.cc:77] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5: Aborting op as it isn't in flight: id { term: 27 index: 21 } timestamp: 7305626352818102272 op_type: NO_OP noop_request { }
I20260709 12:29:46.223644  5137 raft_consensus.cc:2889] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:46.223977  5137 raft_consensus.cc:2272] T a2f5503246494eed8b91ac10cdf742e2 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.226462  5137 tablet_replica.cc:333] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.227214  5137 raft_consensus.cc:2243] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.228226  5137 raft_consensus.cc:2272] T 150319986ee143d5a746f901c5b4adcc P 561a4e21d8b344a79af44568ca80a3c5 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.230800  5137 tablet_replica.cc:333] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.231552  5137 raft_consensus.cc:2243] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.232852  5137 raft_consensus.cc:2272] T f0250a69b3914cb38265a7457b73a158 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.235664  5137 tablet_replica.cc:333] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.236413  5137 raft_consensus.cc:2243] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.237141  5137 raft_consensus.cc:2272] T 1b749ec31e454255bbbba4a4bb493f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.240022  5137 tablet_replica.cc:333] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.240818  5137 raft_consensus.cc:2243] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.241427  5137 raft_consensus.cc:2272] T 034bd5c36e71466da14c6a4850a920cb P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.244346  5137 tablet_replica.cc:333] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.245065  5137 raft_consensus.cc:2243] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.245849  5137 raft_consensus.cc:2272] T 0fe2196990ce4d60adc195d394d4c01c P 561a4e21d8b344a79af44568ca80a3c5 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.248457  5137 tablet_replica.cc:333] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.249155  5137 raft_consensus.cc:2243] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.250382  5137 raft_consensus.cc:2272] T 34bae875d7104f298673c223da6caf2e P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.253041  5137 tablet_replica.cc:333] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.253821  5137 raft_consensus.cc:2243] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.254987  5137 raft_consensus.cc:2272] T cd67024481794a7db3a98df0cd400c9d P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.257530  5137 tablet_replica.cc:333] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.258248  5137 raft_consensus.cc:2243] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.259456  5137 raft_consensus.cc:2272] T f745d20b9f8c4aca9adf20ed14f509ac P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.262385  5137 tablet_replica.cc:333] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.263206  5137 raft_consensus.cc:2243] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.264055  5137 raft_consensus.cc:2272] T 7878a824e1e94db6a0b83ab47738f4b0 P 561a4e21d8b344a79af44568ca80a3c5 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.266937  5137 tablet_replica.cc:333] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.267727  5137 raft_consensus.cc:2243] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.268936  5137 raft_consensus.cc:2272] T e600ce2bd05948cf8ce5a5f292f57f28 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.271844  5137 tablet_replica.cc:333] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.272855  5137 raft_consensus.cc:2243] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.273934  5137 raft_consensus.cc:2272] T f73da77e7bb0404e9f79f06e3dd200f4 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.276793  5137 tablet_replica.cc:333] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.277511  5137 raft_consensus.cc:2243] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.278607  5137 raft_consensus.cc:2272] T 8729b74dbaf94d7092d811f0a55205f2 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.281104  5137 tablet_replica.cc:333] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.281855  5137 raft_consensus.cc:2243] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.282418  5137 raft_consensus.cc:2272] T c413b06caa444496aecb13ac2142c246 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.284968  5137 tablet_replica.cc:333] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.285682  5137 raft_consensus.cc:2243] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.286684  5137 raft_consensus.cc:2272] T 2ddf475f266c47adb1a50aa986a0c44c P 561a4e21d8b344a79af44568ca80a3c5 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.289079  5137 tablet_replica.cc:333] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.289757  5137 raft_consensus.cc:2243] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.290787  5137 raft_consensus.cc:2272] T 55b778a085744d27a1adc35598438230 P 561a4e21d8b344a79af44568ca80a3c5 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.293185  5137 tablet_replica.cc:333] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.293879  5137 raft_consensus.cc:2243] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.294451  5137 raft_consensus.cc:2272] T 3b2f25afefb44300a1d6297e2cc140f5 P 561a4e21d8b344a79af44568ca80a3c5 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.296895  5137 tablet_replica.cc:333] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.297565  5137 raft_consensus.cc:2243] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.298576  5137 raft_consensus.cc:2272] T 4d797adfa10541918dfc0ebf49b90784 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.301079  5137 tablet_replica.cc:333] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.301797  5137 raft_consensus.cc:2243] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.302699  5137 raft_consensus.cc:2272] T f24bf2d0bc904aa495cdb5995b470fb0 P 561a4e21d8b344a79af44568ca80a3c5 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.305712  5137 tablet_replica.cc:333] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.306422  5137 raft_consensus.cc:2243] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEARNER]: Raft consensus shutting down.
I20260709 12:29:46.306921  5137 pending_rounds.cc:70] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Trying to abort 1 pending ops.
I20260709 12:29:46.307153  5137 pending_rounds.cc:77] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5: Aborting op as it isn't in flight: id { term: 17 index: 11 } timestamp: 7305625903510306816 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "d53ab19d424b4a85af2153ef24520678" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } } } new_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "4268023ecea149cd83cba77db53c3100" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 37395 } } peers { permanent_uuid: "0bdd5f9c03b84f7289e750ebaab2fda6" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 34641 } } peers { permanent_uuid: "4e0e521cb08c420f981a29576eee90a6" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 45591 } attrs { replace: true } } peers { permanent_uuid: "561a4e21d8b344a79af44568ca80a3c5" member_type: NON_VOTER last_known_addr { host: "127.5.4.68" port: 43625 } attrs { promote: true } } } }
I20260709 12:29:46.308167  5137 raft_consensus.cc:2917] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 LEARNER]: Aborting config change with OpId 17.11: Aborted: Op aborted
I20260709 12:29:46.308566  5137 raft_consensus.cc:2889] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:29:46.308900  5137 raft_consensus.cc:2272] T d53ab19d424b4a85af2153ef24520678 P 561a4e21d8b344a79af44568ca80a3c5 [term 18 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 12:29:46.312058  5137 tablet_replica.cc:333] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.312829  5137 raft_consensus.cc:2243] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.313642  5137 raft_consensus.cc:2272] T dda9448d18794a44a75d0b570e998c3a P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.316372  5137 tablet_replica.cc:333] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.317080  5137 raft_consensus.cc:2243] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:46.317752  5137 raft_consensus.cc:2272] T 28020383d3c344fda935a94bca07ad82 P 561a4e21d8b344a79af44568ca80a3c5 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.320736  5137 tablet_replica.cc:333] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.321401  5137 raft_consensus.cc:2243] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.322546  5137 raft_consensus.cc:2272] T 7db3e095e8b44b23b1a439899d305bd7 P 561a4e21d8b344a79af44568ca80a3c5 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.325371  5137 tablet_replica.cc:333] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5: stopping tablet replica
I20260709 12:29:46.326057  5137 raft_consensus.cc:2243] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:29:46.327136  5137 raft_consensus.cc:2272] T 4a0ddc1149fd4b03af2abd36909591d0 P 561a4e21d8b344a79af44568ca80a3c5 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:46.868975  5137 tablet_server.cc:196] TabletServer@127.5.4.68:0 shutdown complete.
I20260709 12:29:46.956631  5137 master.cc:562] Master@127.5.4.126:35813 shutting down...
W20260709 12:29:47.007709  6254 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:29:47.008240  6254 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:29:47.008460  6254 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:29:49.521623  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 45579ad5e67d4843a6f90bf59939a2d7: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:30:00.578930  6193 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111) [suppressed 112 similar messages]
W20260709 12:30:00.582747  6193 catalog_manager.cc:4755] TS 4268023ecea149cd83cba77db53c3100 (127.5.4.65:37395): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 44fa024e8a7d48a6bc67d49f348bc0c0: Network error: Client connection negotiation failed: client connection to 127.5.4.65:37395: connect: Connection refused (error 111)
W20260709 12:30:05.304666  6190 catalog_manager.cc:4755] TS 4e0e521cb08c420f981a29576eee90a6 (127.5.4.67:45591): ChangeConfig:REMOVE_PEER RPC failed for tablet 2483d7801887403e873adc62429fe046: Network error: Client connection negotiation failed: client connection to 127.5.4.67:45591: connect: Connection refused (error 111)
I20260709 12:30:08.260720  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:08.262117  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:08.262609  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P 08c731ab128f4ef881e4c3c73ff48bd8: stopping tablet replica
I20260709 12:30:08.542122  5137 master.cc:584] Master@127.5.4.126:35813 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (342843 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:30:08.666821  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:38383
I20260709 12:30:08.667954  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:08.674094 16994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:08.676569 16997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:08.676468 16995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:08.677500  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:08.682433  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:08.682698  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:08.682888  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600208682873 us; error 0 us; skew 500 ppm
I20260709 12:30:08.683481  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:08.697317  5137 webserver.cc:533] Webserver started at http://127.5.4.126:46761/ using document root <none> and password file <none>
I20260709 12:30:08.697882  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:08.698073  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:08.698387  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:08.699712  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "ee328dfad2034db3a72398b337bc4cb7"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-lg8f"
I20260709 12:30:08.704406  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:30:08.707906 17002 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:08.708931  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:30:08.709336  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "ee328dfad2034db3a72398b337bc4cb7"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-lg8f"
I20260709 12:30:08.709765  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:08.730427  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:08.731940  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:08.779127  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:38383
I20260709 12:30:08.779244 17053 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:38383 every 8 connection(s)
I20260709 12:30:08.783809 17054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:08.795415 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7: Bootstrap starting.
I20260709 12:30:08.800493 17054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:08.805359 17054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7: No bootstrap required, opened a new log
I20260709 12:30:08.808113 17054 raft_consensus.cc:359] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee328dfad2034db3a72398b337bc4cb7" member_type: VOTER }
I20260709 12:30:08.808632 17054 raft_consensus.cc:385] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:08.808903 17054 raft_consensus.cc:740] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee328dfad2034db3a72398b337bc4cb7, State: Initialized, Role: FOLLOWER
I20260709 12:30:08.809765 17054 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee328dfad2034db3a72398b337bc4cb7" member_type: VOTER }
I20260709 12:30:08.810281 17054 raft_consensus.cc:399] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:08.810566 17054 raft_consensus.cc:493] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:08.810894 17054 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:08.815764 17054 raft_consensus.cc:515] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee328dfad2034db3a72398b337bc4cb7" member_type: VOTER }
I20260709 12:30:08.816304 17054 leader_election.cc:304] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee328dfad2034db3a72398b337bc4cb7; no voters: 
I20260709 12:30:08.817546 17054 leader_election.cc:290] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:08.818145 17057 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:08.819873 17057 raft_consensus.cc:697] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 1 LEADER]: Becoming Leader. State: Replica: ee328dfad2034db3a72398b337bc4cb7, State: Running, Role: LEADER
I20260709 12:30:08.820713 17057 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee328dfad2034db3a72398b337bc4cb7" member_type: VOTER }
I20260709 12:30:08.820869 17054 sys_catalog.cc:565] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:08.829334 17059 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee328dfad2034db3a72398b337bc4cb7. Latest consensus state: current_term: 1 leader_uuid: "ee328dfad2034db3a72398b337bc4cb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee328dfad2034db3a72398b337bc4cb7" member_type: VOTER } }
I20260709 12:30:08.829419 17058 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee328dfad2034db3a72398b337bc4cb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee328dfad2034db3a72398b337bc4cb7" member_type: VOTER } }
I20260709 12:30:08.830127 17059 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:08.830153 17058 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:08.832495 17067 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:08.839025 17067 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:08.840256  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:08.850356 17067 catalog_manager.cc:1383] Generated new cluster ID: 3489870c17a5487d925d2f20df6de81a
I20260709 12:30:08.850817 17067 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:08.872828 17067 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:08.874850 17067 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:08.892021 17067 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7: Generated new TSK 0
I20260709 12:30:08.892984 17067 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:08.907975  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:08.915170 17075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:08.916280 17076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:08.920765  5137 server_base.cc:1061] running on GCE node
W20260709 12:30:08.921516 17078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:08.924461  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:08.924799  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:08.925027  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600208925011 us; error 0 us; skew 500 ppm
I20260709 12:30:08.925664  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:08.928303  5137 webserver.cc:533] Webserver started at http://127.5.4.65:33081/ using document root <none> and password file <none>
I20260709 12:30:08.928840  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:08.929035  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:08.929328  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:08.930590  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "79e92a05527f4a0b98b8352cb68a1010"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-lg8f"
I20260709 12:30:08.935537  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 12:30:08.939608 17083 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:08.940847  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 12:30:08.941359  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "79e92a05527f4a0b98b8352cb68a1010"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-lg8f"
I20260709 12:30:08.941820  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:08.961651  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:08.963119  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:08.964836  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:08.967512  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:08.967698  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:08.968052  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:08.968215  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.021606  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:34625
I20260709 12:30:09.021749 17145 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:34625 every 8 connection(s)
I20260709 12:30:09.028100  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.037123 17150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.038314 17151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.040782 17153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.042418  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:09.042877 17146 heartbeater.cc:344] Connected to a master server at 127.5.4.126:38383
I20260709 12:30:09.043324 17146 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:09.044260 17146 heartbeater.cc:507] Master 127.5.4.126:38383 requested a full tablet report, sending...
I20260709 12:30:09.046639  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:09.046929  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.046867 17019 ts_manager.cc:194] Registered new tserver with Master: 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:09.047518  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600209047475 us; error 0 us; skew 500 ppm
I20260709 12:30:09.048290  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.049327 17019 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60912
I20260709 12:30:09.051259  5137 webserver.cc:533] Webserver started at http://127.5.4.66:39955/ using document root <none> and password file <none>
I20260709 12:30:09.051854  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.052085  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.052373  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.053800  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-lg8f"
I20260709 12:30:09.058738  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:30:09.062650 17158 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.063537  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:09.063869  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-lg8f"
I20260709 12:30:09.064188  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.083993  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.085464  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.087288  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.089757  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.089999  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.090255  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.090435  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.141191  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:44307
I20260709 12:30:09.141319 17220 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:44307 every 8 connection(s)
I20260709 12:30:09.146871  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.155754 17224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.157985 17225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.161778  5137 server_base.cc:1061] running on GCE node
W20260709 12:30:09.162632 17227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.162915 17221 heartbeater.cc:344] Connected to a master server at 127.5.4.126:38383
I20260709 12:30:09.163405 17221 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:09.164019  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:30:09.164256 17221 heartbeater.cc:507] Master 127.5.4.126:38383 requested a full tablet report, sending...
W20260709 12:30:09.164486  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.164829  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600209164810 us; error 0 us; skew 500 ppm
I20260709 12:30:09.165637  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.166911 17019 ts_manager.cc:194] Registered new tserver with Master: c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:09.168596 17019 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60920
I20260709 12:30:09.168731  5137 webserver.cc:533] Webserver started at http://127.5.4.67:36627/ using document root <none> and password file <none>
I20260709 12:30:09.169407  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.169656  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.169958  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.171526  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "6c1eb70ad94940b89118a32a370a4310"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-lg8f"
I20260709 12:30:09.177206  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:30:09.181236 17232 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.182072  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:30:09.182411  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "6c1eb70ad94940b89118a32a370a4310"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-lg8f"
I20260709 12:30:09.182812  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.202764  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.204216  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.205891  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.208266  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.208498  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.208776  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.208956  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.259224  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:43265
I20260709 12:30:09.259346 17294 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:43265 every 8 connection(s)
I20260709 12:30:09.264746  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:09.273684 17298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.274333 17299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:09.277570 17301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:09.277956  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:09.279572  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:09.279877  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:09.280121  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600209280099 us; error 0 us; skew 500 ppm
I20260709 12:30:09.280117 17295 heartbeater.cc:344] Connected to a master server at 127.5.4.126:38383
I20260709 12:30:09.280568 17295 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:09.280754  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:09.281368 17295 heartbeater.cc:507] Master 127.5.4.126:38383 requested a full tablet report, sending...
I20260709 12:30:09.283582 17019 ts_manager.cc:194] Registered new tserver with Master: 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:09.284099  5137 webserver.cc:533] Webserver started at http://127.5.4.68:38057/ using document root <none> and password file <none>
I20260709 12:30:09.284714  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:09.284936  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:09.285223  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:09.285285 17019 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60924
I20260709 12:30:09.286788  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-3-root/instance:
uuid: "7c0f44cba62f491392d41ca89376139a"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-lg8f"
I20260709 12:30:09.291611  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:30:09.295405 17306 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.296241  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.004s
I20260709 12:30:09.296582  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-3-root
uuid: "7c0f44cba62f491392d41ca89376139a"
format_stamp: "Formatted at 2026-07-09 12:30:09 on dist-test-slave-lg8f"
I20260709 12:30:09.296904  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:09.315773  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:09.317241  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:09.318913  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:09.322968  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:09.323359  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:30:09.323743  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:09.324031  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:09.376816  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.68:38585
I20260709 12:30:09.376953 17368 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.68:38585 every 8 connection(s)
I20260709 12:30:09.392820 17369 heartbeater.cc:344] Connected to a master server at 127.5.4.126:38383
I20260709 12:30:09.393285 17369 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:09.394196 17369 heartbeater.cc:507] Master 127.5.4.126:38383 requested a full tablet report, sending...
I20260709 12:30:09.396430 17019 ts_manager.cc:194] Registered new tserver with Master: 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:09.397192  5137 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016564789s
I20260709 12:30:09.398069 17019 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60932
I20260709 12:30:09.431277 17019 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60942:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:30:09.719565 17185 tablet_service.cc:1511] Processing CreateTablet for tablet 52b7abb65c454a8f9dca603b58f2cc0a (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:30:09.720731 17183 tablet_service.cc:1511] Processing CreateTablet for tablet 5746c5442b934f38910321caaa1db9a8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:30:09.721534 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52b7abb65c454a8f9dca603b58f2cc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.722285 17183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5746c5442b934f38910321caaa1db9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.731593 17334 tablet_service.cc:1511] Processing CreateTablet for tablet 675c55f0ab604b0cb4f3205d8bedf323 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:30:09.733378 17334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 675c55f0ab604b0cb4f3205d8bedf323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.733037 17332 tablet_service.cc:1511] Processing CreateTablet for tablet 0148e46ec1b2484e902b01edfda2976c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:30:09.723922 17182 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2404ce16f84a8a9429e9b50a14d97b (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:30:09.735543 17182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2404ce16f84a8a9429e9b50a14d97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.735984 17110 tablet_service.cc:1511] Processing CreateTablet for tablet 52b7abb65c454a8f9dca603b58f2cc0a (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:30:09.723925 17184 tablet_service.cc:1511] Processing CreateTablet for tablet 0148e46ec1b2484e902b01edfda2976c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:30:09.739039 17184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0148e46ec1b2484e902b01edfda2976c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.738633 17331 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2404ce16f84a8a9429e9b50a14d97b (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:30:09.741355 17109 tablet_service.cc:1511] Processing CreateTablet for tablet e9b921f32d04442e89c1b7ea73df020f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:30:09.742808 17180 tablet_service.cc:1511] Processing CreateTablet for tablet 058259c8483642bba034c3bd3f5ba90e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:30:09.743860 17330 tablet_service.cc:1511] Processing CreateTablet for tablet 44c804e5f1864c799cd5d1d49734db98 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:30:09.744462 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058259c8483642bba034c3bd3f5ba90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.746471 17329 tablet_service.cc:1511] Processing CreateTablet for tablet fd26acf9f75043088e62624ad9f61c2e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:30:09.747839 17179 tablet_service.cc:1511] Processing CreateTablet for tablet 55a24caeb8124f77a6a9d21f8bed1831 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:30:09.749418 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55a24caeb8124f77a6a9d21f8bed1831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.749099 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 5746c5442b934f38910321caaa1db9a8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:30:09.751665 17107 tablet_service.cc:1511] Processing CreateTablet for tablet 9d2404ce16f84a8a9429e9b50a14d97b (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:30:09.753791 17178 tablet_service.cc:1511] Processing CreateTablet for tablet 0af900dfc2784f8781502931a17eaa79 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:30:09.755298 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af900dfc2784f8781502931a17eaa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.755470 17106 tablet_service.cc:1511] Processing CreateTablet for tablet 44c804e5f1864c799cd5d1d49734db98 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:30:09.725349 17181 tablet_service.cc:1511] Processing CreateTablet for tablet fd26acf9f75043088e62624ad9f61c2e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:30:09.729509 17333 tablet_service.cc:1511] Processing CreateTablet for tablet e9b921f32d04442e89c1b7ea73df020f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:30:09.751818 17177 tablet_service.cc:1511] Processing CreateTablet for tablet fd33570a634744e89bbf929e83441909 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:30:09.760677 17181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd26acf9f75043088e62624ad9f61c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.760810 17328 tablet_service.cc:1511] Processing CreateTablet for tablet 55a24caeb8124f77a6a9d21f8bed1831 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:30:09.762705 17104 tablet_service.cc:1511] Processing CreateTablet for tablet 55a24caeb8124f77a6a9d21f8bed1831 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:30:09.726832 17186 tablet_service.cc:1511] Processing CreateTablet for tablet 675c55f0ab604b0cb4f3205d8bedf323 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:30:09.764128 17103 tablet_service.cc:1511] Processing CreateTablet for tablet 13be6c315f194afc80d89f01777e105a (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:30:09.731626 17111 tablet_service.cc:1511] Processing CreateTablet for tablet 675c55f0ab604b0cb4f3205d8bedf323 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:30:09.766655 17102 tablet_service.cc:1511] Processing CreateTablet for tablet fd33570a634744e89bbf929e83441909 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:30:09.767771 17110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52b7abb65c454a8f9dca603b58f2cc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.767993 17105 tablet_service.cc:1511] Processing CreateTablet for tablet 058259c8483642bba034c3bd3f5ba90e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:30:09.770174 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058259c8483642bba034c3bd3f5ba90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.770959 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c804e5f1864c799cd5d1d49734db98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.771097 17177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd33570a634744e89bbf929e83441909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.771843 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd33570a634744e89bbf929e83441909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.774593 17103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13be6c315f194afc80d89f01777e105a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.777077 17186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 675c55f0ab604b0cb4f3205d8bedf323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.777130 17332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0148e46ec1b2484e902b01edfda2976c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.783038 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 0af900dfc2784f8781502931a17eaa79 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:30:09.784564 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af900dfc2784f8781502931a17eaa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.785447 17331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2404ce16f84a8a9429e9b50a14d97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.781700 17327 tablet_service.cc:1511] Processing CreateTablet for tablet 13be6c315f194afc80d89f01777e105a (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:30:09.787531 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13be6c315f194afc80d89f01777e105a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.788215 17333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b921f32d04442e89c1b7ea73df020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.777971 17111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 675c55f0ab604b0cb4f3205d8bedf323. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.789973 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5746c5442b934f38910321caaa1db9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.790825 17109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b921f32d04442e89c1b7ea73df020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.791600 17104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55a24caeb8124f77a6a9d21f8bed1831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.792512 17107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d2404ce16f84a8a9429e9b50a14d97b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.794083 17330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c804e5f1864c799cd5d1d49734db98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.799069 17328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55a24caeb8124f77a6a9d21f8bed1831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.806898 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd26acf9f75043088e62624ad9f61c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.806552 17325 tablet_service.cc:1511] Processing CreateTablet for tablet ecf98a8bc05f404f8fb4149773b1c384 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:30:09.808275 17325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf98a8bc05f404f8fb4149773b1c384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.825591 17260 tablet_service.cc:1511] Processing CreateTablet for tablet 52b7abb65c454a8f9dca603b58f2cc0a (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:30:09.827847 17259 tablet_service.cc:1511] Processing CreateTablet for tablet e9b921f32d04442e89c1b7ea73df020f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:30:09.829483 17259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e9b921f32d04442e89c1b7ea73df020f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.831898 17258 tablet_service.cc:1511] Processing CreateTablet for tablet 0148e46ec1b2484e902b01edfda2976c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
W20260709 12:30:09.832794 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:09.833401 17258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0148e46ec1b2484e902b01edfda2976c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.835911 17257 tablet_service.cc:1511] Processing CreateTablet for tablet 5746c5442b934f38910321caaa1db9a8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:30:09.838680 17256 tablet_service.cc:1511] Processing CreateTablet for tablet 44c804e5f1864c799cd5d1d49734db98 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:30:09.841544 17255 tablet_service.cc:1511] Processing CreateTablet for tablet fd26acf9f75043088e62624ad9f61c2e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:30:09.843935 17254 tablet_service.cc:1511] Processing CreateTablet for tablet 058259c8483642bba034c3bd3f5ba90e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:30:09.847124 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 13be6c315f194afc80d89f01777e105a (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:30:09.849988 17252 tablet_service.cc:1511] Processing CreateTablet for tablet 0af900dfc2784f8781502931a17eaa79 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:30:09.914858 17251 tablet_service.cc:1511] Processing CreateTablet for tablet fd33570a634744e89bbf929e83441909 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:30:09.916615 17257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5746c5442b934f38910321caaa1db9a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.919163 17256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44c804e5f1864c799cd5d1d49734db98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.929847 17185 tablet_service.cc:1511] Processing CreateTablet for tablet ecf98a8bc05f404f8fb4149773b1c384 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:30:09.951733 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13be6c315f194afc80d89f01777e105a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.955811 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd33570a634744e89bbf929e83441909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.959371 17254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 058259c8483642bba034c3bd3f5ba90e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.961062 17255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd26acf9f75043088e62624ad9f61c2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.961766 17252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0af900dfc2784f8781502931a17eaa79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.966310 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52b7abb65c454a8f9dca603b58f2cc0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.971350 17183 tablet_service.cc:1511] Processing CreateTablet for tablet 795b495c58894c1eb861e953546a6e6d (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:30:09.972801 17183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 795b495c58894c1eb861e953546a6e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:09.975461 17389 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:09.982367 17182 tablet_service.cc:1511] Processing CreateTablet for tablet 901d47a673fa40b7a58a7bea321288aa (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:30:09.984223 17182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 901d47a673fa40b7a58a7bea321288aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.001158 17179 tablet_service.cc:1511] Processing CreateTablet for tablet f9af739be394434c9d30f15785e5e692 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:30:10.008651 17110 tablet_service.cc:1511] Processing CreateTablet for tablet ecf98a8bc05f404f8fb4149773b1c384 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:30:10.010267 17110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf98a8bc05f404f8fb4149773b1c384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.018061 17331 tablet_service.cc:1511] Processing CreateTablet for tablet fb995bc24cf54980b8977e69501ac133 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:30:10.021808 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9af739be394434c9d30f15785e5e692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.025987 17389 tablet_bootstrap.cc:654] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.055132 17331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb995bc24cf54980b8977e69501ac133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.058746 17146 heartbeater.cc:499] Master 127.5.4.126:38383 was elected leader, sending a full tablet report...
I20260709 12:30:10.060103 17180 tablet_service.cc:1511] Processing CreateTablet for tablet 60f10e5fb23f4dc08b18d7e805d0a0f4 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:30:10.061597 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f10e5fb23f4dc08b18d7e805d0a0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.064113 17330 tablet_service.cc:1511] Processing CreateTablet for tablet f9af739be394434c9d30f15785e5e692 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:30:10.065546 17330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9af739be394434c9d30f15785e5e692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.076464 17178 tablet_service.cc:1511] Processing CreateTablet for tablet 09bd2f3de39744c28ad21f75b37293b9 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:30:10.078114 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09bd2f3de39744c28ad21f75b37293b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.079639 17325 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca4d4100a154e058fd6b5525d3e280f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:30:10.079493 17107 tablet_service.cc:1511] Processing CreateTablet for tablet fb995bc24cf54980b8977e69501ac133 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:30:10.084694 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 60f10e5fb23f4dc08b18d7e805d0a0f4 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:30:10.087882 17105 tablet_service.cc:1511] Processing CreateTablet for tablet f9af739be394434c9d30f15785e5e692 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:30:10.096451 17106 tablet_service.cc:1511] Processing CreateTablet for tablet 901d47a673fa40b7a58a7bea321288aa (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:30:10.100795 17109 tablet_service.cc:1511] Processing CreateTablet for tablet 09bd2f3de39744c28ad21f75b37293b9 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:30:10.117537 17254 tablet_service.cc:1511] Processing CreateTablet for tablet fb995bc24cf54980b8977e69501ac133 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:30:10.118953 17254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb995bc24cf54980b8977e69501ac133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.119403 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 901d47a673fa40b7a58a7bea321288aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.123137 17107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb995bc24cf54980b8977e69501ac133. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.127689 17332 tablet_service.cc:1511] Processing CreateTablet for tablet 795b495c58894c1eb861e953546a6e6d (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:30:10.137660 17391 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:10.134162 17181 tablet_service.cc:1511] Processing CreateTablet for tablet ec4e7b8f0ed341e0b42e5d3872c70010 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:30:10.145138 17109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09bd2f3de39744c28ad21f75b37293b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.098366 17103 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca4d4100a154e058fd6b5525d3e280f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:30:10.146735 17325 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca4d4100a154e058fd6b5525d3e280f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.150534 17390 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:10.177855 17221 heartbeater.cc:499] Master 127.5.4.126:38383 was elected leader, sending a full tablet report...
I20260709 12:30:10.175181 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ecf98a8bc05f404f8fb4149773b1c384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.190330 17332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 795b495c58894c1eb861e953546a6e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.193545 17257 tablet_service.cc:1511] Processing CreateTablet for tablet 1ca4d4100a154e058fd6b5525d3e280f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:30:10.195158 17257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca4d4100a154e058fd6b5525d3e280f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.204732 17251 tablet_service.cc:1511] Processing CreateTablet for tablet 795b495c58894c1eb861e953546a6e6d (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:30:10.207417 17329 tablet_service.cc:1511] Processing CreateTablet for tablet ec4e7b8f0ed341e0b42e5d3872c70010 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:30:10.208969 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e7b8f0ed341e0b42e5d3872c70010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.209062 17252 tablet_service.cc:1511] Processing CreateTablet for tablet 901d47a673fa40b7a58a7bea321288aa (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:30:10.210474 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9af739be394434c9d30f15785e5e692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.210669 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 795b495c58894c1eb861e953546a6e6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.214797 17328 tablet_service.cc:1511] Processing CreateTablet for tablet d85bc0ab53bf4a66af3a1b42ff166216 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:30:10.218976 17102 tablet_service.cc:1511] Processing CreateTablet for tablet ec4e7b8f0ed341e0b42e5d3872c70010 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:30:10.220968 17104 tablet_service.cc:1511] Processing CreateTablet for tablet d85bc0ab53bf4a66af3a1b42ff166216 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:30:10.227170 17181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e7b8f0ed341e0b42e5d3872c70010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.229517 17391 tablet_bootstrap.cc:654] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.235494 17389 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:10.235253 17259 tablet_service.cc:1511] Processing CreateTablet for tablet 60f10e5fb23f4dc08b18d7e805d0a0f4 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:30:10.235611 17186 tablet_service.cc:1511] Processing CreateTablet for tablet 2d81d3402a7848f886f20c683e7874f3 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:30:10.244376 17389 ts_tablet_manager.cc:1403] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.269s	user 0.015s	sys 0.028s
I20260709 12:30:10.248548 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f10e5fb23f4dc08b18d7e805d0a0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.253791 17252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 901d47a673fa40b7a58a7bea321288aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.258168 17393 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:10.269083 17389 raft_consensus.cc:359] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:10.269968 17389 raft_consensus.cc:385] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.269275 17328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d85bc0ab53bf4a66af3a1b42ff166216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.273156 17390 tablet_bootstrap.cc:654] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.251925 17103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1ca4d4100a154e058fd6b5525d3e280f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.277247 17256 tablet_service.cc:1511] Processing CreateTablet for tablet 09bd2f3de39744c28ad21f75b37293b9 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:30:10.278681 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4e7b8f0ed341e0b42e5d3872c70010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.279239 17393 tablet_bootstrap.cc:654] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.283813 17104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d85bc0ab53bf4a66af3a1b42ff166216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.288206 17295 heartbeater.cc:499] Master 127.5.4.126:38383 was elected leader, sending a full tablet report...
I20260709 12:30:10.270330 17389 raft_consensus.cc:740] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.294378 17389 consensus_queue.cc:260] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:10.297382 17259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60f10e5fb23f4dc08b18d7e805d0a0f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.302879 17256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 09bd2f3de39744c28ad21f75b37293b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.300338 17258 tablet_service.cc:1511] Processing CreateTablet for tablet d85bc0ab53bf4a66af3a1b42ff166216 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:30:10.304380 17258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d85bc0ab53bf4a66af3a1b42ff166216. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.310688 17186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d81d3402a7848f886f20c683e7874f3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:10.328819 17003 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 45106.2us
I20260709 12:30:10.356453 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 2d81d3402a7848f886f20c683e7874f3 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:30:10.358014 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d81d3402a7848f886f20c683e7874f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.361203 17327 tablet_service.cc:1511] Processing CreateTablet for tablet 2d81d3402a7848f886f20c683e7874f3 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:30:10.366319 17178 tablet_service.cc:1511] Processing CreateTablet for tablet 0eac394a27ae4bd88c0948a7c1ad1868 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:30:10.367703 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2d81d3402a7848f886f20c683e7874f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.367738 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eac394a27ae4bd88c0948a7c1ad1868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.379707 17255 tablet_service.cc:1511] Processing CreateTablet for tablet 0eac394a27ae4bd88c0948a7c1ad1868 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:30:10.381137 17255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eac394a27ae4bd88c0948a7c1ad1868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.390697 17103 tablet_service.cc:1511] Processing CreateTablet for tablet 0eac394a27ae4bd88c0948a7c1ad1868 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:30:10.393746 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 47941e6ecfe34dfeb4aeb7f68e020eb5 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:30:10.395494 17181 tablet_service.cc:1511] Processing CreateTablet for tablet 47941e6ecfe34dfeb4aeb7f68e020eb5 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:30:10.396900 17181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47941e6ecfe34dfeb4aeb7f68e020eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.397953 17391 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:10.401618 17369 heartbeater.cc:499] Master 127.5.4.126:38383 was elected leader, sending a full tablet report...
I20260709 12:30:10.402539 17330 tablet_service.cc:1511] Processing CreateTablet for tablet b6a9d0d27df74b748881bc3ef8245a62 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:30:10.402773 17103 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eac394a27ae4bd88c0948a7c1ad1868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.416184 17390 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:10.416749 17390 ts_tablet_manager.cc:1403] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.267s	user 0.008s	sys 0.013s
I20260709 12:30:10.420054 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47941e6ecfe34dfeb4aeb7f68e020eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.419929 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 35817ffe9ce34abd82d949dc55a8ae78 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:30:10.401182 17389 ts_tablet_manager.cc:1434] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.149s	user 0.010s	sys 0.011s
I20260709 12:30:10.414387 17330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a9d0d27df74b748881bc3ef8245a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.422909 17389 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:10.423208 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35817ffe9ce34abd82d949dc55a8ae78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.413532 17391 ts_tablet_manager.cc:1403] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.276s	user 0.011s	sys 0.013s
I20260709 12:30:10.426069 17106 tablet_service.cc:1511] Processing CreateTablet for tablet cd19f1dc408f4801b7e7d672696228cc (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:30:10.427598 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd19f1dc408f4801b7e7d672696228cc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:10.428864 17222 tablet.cc:2406] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:30:10.429807 17185 tablet_service.cc:1511] Processing CreateTablet for tablet 35817ffe9ce34abd82d949dc55a8ae78 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:30:10.432260 17102 tablet_service.cc:1511] Processing CreateTablet for tablet b6a9d0d27df74b748881bc3ef8245a62 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:30:10.436959 17329 tablet_service.cc:1511] Processing CreateTablet for tablet 47941e6ecfe34dfeb4aeb7f68e020eb5 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:30:10.491497 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a9d0d27df74b748881bc3ef8245a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.443004 17251 tablet_service.cc:1511] Processing CreateTablet for tablet b6a9d0d27df74b748881bc3ef8245a62 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:30:10.492700 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b6a9d0d27df74b748881bc3ef8245a62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.476737 17178 tablet_service.cc:1511] Processing CreateTablet for tablet cd19f1dc408f4801b7e7d672696228cc (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:30:10.491497 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47941e6ecfe34dfeb4aeb7f68e020eb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.499276 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd19f1dc408f4801b7e7d672696228cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.499603 17106 tablet_service.cc:1511] Processing CreateTablet for tablet 27bba673d23842c790eb0140a01cee0c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:30:10.478169 17257 tablet_service.cc:1511] Processing CreateTablet for tablet 35817ffe9ce34abd82d949dc55a8ae78 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:30:10.479202 17254 tablet_service.cc:1511] Processing CreateTablet for tablet cd19f1dc408f4801b7e7d672696228cc (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:30:10.500857 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bba673d23842c790eb0140a01cee0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.480664 17332 tablet_service.cc:1511] Processing CreateTablet for tablet 80ed704c39774ac88e3d32548f26b242 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:30:10.481695 17180 tablet_service.cc:1511] Processing CreateTablet for tablet 80ed704c39774ac88e3d32548f26b242 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:30:10.504048 17393 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:10.503281 17257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35817ffe9ce34abd82d949dc55a8ae78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.525978 17254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd19f1dc408f4801b7e7d672696228cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.503448 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35817ffe9ce34abd82d949dc55a8ae78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.527251 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ed704c39774ac88e3d32548f26b242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.511808 17332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ed704c39774ac88e3d32548f26b242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.531497 17389 tablet_bootstrap.cc:654] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.517580 17393 ts_tablet_manager.cc:1403] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.260s	user 0.006s	sys 0.017s
I20260709 12:30:10.537803 17393 raft_consensus.cc:359] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.487438 17260 tablet_service.cc:1511] Processing CreateTablet for tablet 27bba673d23842c790eb0140a01cee0c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:30:10.555353 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bba673d23842c790eb0140a01cee0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.489806 17328 tablet_service.cc:1511] Processing CreateTablet for tablet 27bba673d23842c790eb0140a01cee0c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:30:10.558357 17104 tablet_service.cc:1511] Processing CreateTablet for tablet 6e37e6c15d9f4c37b87d6f494bdc66bf (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:30:10.559295 17328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27bba673d23842c790eb0140a01cee0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.559720 17104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e37e6c15d9f4c37b87d6f494bdc66bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.560271 17257 tablet_service.cc:1511] Processing CreateTablet for tablet 6e37e6c15d9f4c37b87d6f494bdc66bf (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:30:10.491223 17179 tablet_service.cc:1511] Processing CreateTablet for tablet 1711901c61714bc1bbb4f2bec64dadb3 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:30:10.561787 17257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e37e6c15d9f4c37b87d6f494bdc66bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.561892 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1711901c61714bc1bbb4f2bec64dadb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.439496 17390 raft_consensus.cc:359] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.566707 17390 raft_consensus.cc:385] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.566571 17327 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec669aa1bf9434f9b198f6d8e768348 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:30:10.567106 17390 raft_consensus.cc:740] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.567646 17106 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec669aa1bf9434f9b198f6d8e768348 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:30:10.568001 17390 consensus_queue.cc:260] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.568943 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec669aa1bf9434f9b198f6d8e768348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.505388 17391 raft_consensus.cc:359] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.574712 17391 raft_consensus.cc:385] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.575073 17391 raft_consensus.cc:740] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.575882 17391 consensus_queue.cc:260] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.540761 17256 tablet_service.cc:1511] Processing CreateTablet for tablet 1711901c61714bc1bbb4f2bec64dadb3 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:30:10.578295 17256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1711901c61714bc1bbb4f2bec64dadb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.568029 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec669aa1bf9434f9b198f6d8e768348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.482131 17105 tablet_service.cc:1511] Processing CreateTablet for tablet 80ed704c39774ac88e3d32548f26b242 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:30:10.543713 17329 tablet_service.cc:1511] Processing CreateTablet for tablet 1711901c61714bc1bbb4f2bec64dadb3 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:30:10.583328 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ed704c39774ac88e3d32548f26b242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.583499 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1711901c61714bc1bbb4f2bec64dadb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.589030 17391 ts_tablet_manager.cc:1434] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.163s	user 0.007s	sys 0.000s
I20260709 12:30:10.590222 17391 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:10.600294 17185 tablet_service.cc:1511] Processing CreateTablet for tablet 6e37e6c15d9f4c37b87d6f494bdc66bf (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:30:10.601744 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e37e6c15d9f4c37b87d6f494bdc66bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.602205 17180 tablet_service.cc:1511] Processing CreateTablet for tablet 6ec669aa1bf9434f9b198f6d8e768348 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:30:10.604214 17393 raft_consensus.cc:385] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.604588 17393 raft_consensus.cc:740] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.629776 17330 tablet_service.cc:1511] Processing CreateTablet for tablet 9614ba0ca1894975b6696b3724157ee6 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:30:10.631428 17330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9614ba0ca1894975b6696b3724157ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.631531 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ec669aa1bf9434f9b198f6d8e768348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.635144 17393 consensus_queue.cc:260] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.635581 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 9614ba0ca1894975b6696b3724157ee6 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:30:10.636955 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9614ba0ca1894975b6696b3724157ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.638016 17390 ts_tablet_manager.cc:1434] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.221s	user 0.017s	sys 0.002s
I20260709 12:30:10.638350 17389 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:10.639010 17389 ts_tablet_manager.cc:1403] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.216s	user 0.011s	sys 0.006s
I20260709 12:30:10.639082 17391 tablet_bootstrap.cc:654] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.643558 17389 raft_consensus.cc:359] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:10.657004 17390 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:10.667157 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 9614ba0ca1894975b6696b3724157ee6 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:30:10.668805 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9614ba0ca1894975b6696b3724157ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.673590 17390 tablet_bootstrap.cc:654] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.663820 17389 raft_consensus.cc:385] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.681871 17178 tablet_service.cc:1511] Processing CreateTablet for tablet ce19f767e060412c8bc15f391d74b87f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:30:10.693109 17396 raft_consensus.cc:493] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.718636 17389 raft_consensus.cc:740] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.719055 17396 raft_consensus.cc:515] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:10.719831 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce19f767e060412c8bc15f391d74b87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.721949 17396 leader_election.cc:290] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:10.695184 17260 tablet_service.cc:1511] Processing CreateTablet for tablet ce19f767e060412c8bc15f391d74b87f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:30:10.700930 17328 tablet_service.cc:1511] Processing CreateTablet for tablet ce19f767e060412c8bc15f391d74b87f (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:30:10.703388 17393 ts_tablet_manager.cc:1434] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.169s	user 0.006s	sys 0.009s
I20260709 12:30:10.706388 17102 tablet_service.cc:1511] Processing CreateTablet for tablet 44f0f66af5d044fdb87c6320a84cbf8e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:30:10.708707 17179 tablet_service.cc:1511] Processing CreateTablet for tablet 44f0f66af5d044fdb87c6320a84cbf8e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:30:10.709885 17400 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.716532 17326 tablet_service.cc:1511] Processing CreateTablet for tablet d5b31c8286684684ae24797a44fc5403 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:30:10.716990 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 44f0f66af5d044fdb87c6320a84cbf8e (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:30:10.729082 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce19f767e060412c8bc15f391d74b87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.729257 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f0f66af5d044fdb87c6320a84cbf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.719892 17389 consensus_queue.cc:260] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:10.733991 17400 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.735617 17105 tablet_service.cc:1511] Processing CreateTablet for tablet d5b31c8286684684ae24797a44fc5403 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:30:10.745245 17393 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:10.746660 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f0f66af5d044fdb87c6320a84cbf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.740804 17400 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:10.765974 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b31c8286684684ae24797a44fc5403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.765144 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44f0f66af5d044fdb87c6320a84cbf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.772435 17329 tablet_service.cc:1511] Processing CreateTablet for tablet 56a32f727d4a47368863c010ce8508c8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:30:10.767580 17328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce19f767e060412c8bc15f391d74b87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.800516 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a32f727d4a47368863c010ce8508c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.816063 17393 tablet_bootstrap.cc:654] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.826009 17391 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:10.831158 17391 ts_tablet_manager.cc:1403] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.241s	user 0.034s	sys 0.042s
I20260709 12:30:10.825223 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b31c8286684684ae24797a44fc5403. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:10.836443 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:10.838989 17389 ts_tablet_manager.cc:1434] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.199s	user 0.009s	sys 0.001s
I20260709 12:30:10.840232 17389 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:10.854867 17401 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.855119 17391 raft_consensus.cc:359] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:10.855538 17401 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.856040 17391 raft_consensus.cc:385] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.856422 17391 raft_consensus.cc:740] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.857328 17391 consensus_queue.cc:260] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:10.859786 17391 ts_tablet_manager.cc:1434] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.028s	user 0.000s	sys 0.008s
I20260709 12:30:10.878412 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 56a32f727d4a47368863c010ce8508c8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:30:10.880278 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a32f727d4a47368863c010ce8508c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.885870 17251 tablet_service.cc:1511] Processing CreateTablet for tablet 419b4941c01a4ffd8c32d71f505b8cde (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:30:10.887495 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 419b4941c01a4ffd8c32d71f505b8cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.887951 17401 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:10.892526 17327 tablet_service.cc:1511] Processing CreateTablet for tablet 419b4941c01a4ffd8c32d71f505b8cde (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:30:10.893216 17390 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:10.893810 17390 ts_tablet_manager.cc:1403] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.237s	user 0.022s	sys 0.007s
I20260709 12:30:10.893922 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 419b4941c01a4ffd8c32d71f505b8cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.888588 17389 tablet_bootstrap.cc:654] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:10.902060 17260 tablet_service.cc:1511] Processing CreateTablet for tablet 56a32f727d4a47368863c010ce8508c8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:30:10.903879 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56a32f727d4a47368863c010ce8508c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.913287 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:10.916525 17180 tablet_service.cc:1511] Processing CreateTablet for tablet d5b31c8286684684ae24797a44fc5403 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:30:10.917963 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5b31c8286684684ae24797a44fc5403. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.917799 17105 tablet_service.cc:1511] Processing CreateTablet for tablet cc02a3ce008e456caafa4f8ba81dcead (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:30:10.919368 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc02a3ce008e456caafa4f8ba81dcead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.928306 17399 raft_consensus.cc:493] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:10.928975 17399 raft_consensus.cc:515] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:10.931226 17185 tablet_service.cc:1511] Processing CreateTablet for tablet 419b4941c01a4ffd8c32d71f505b8cde (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:30:10.932762 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 419b4941c01a4ffd8c32d71f505b8cde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.937069 17390 raft_consensus.cc:359] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:10.937914 17390 raft_consensus.cc:385] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.944875 17399 leader_election.cc:290] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:10.945537 17393 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:10.946266 17393 ts_tablet_manager.cc:1403] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.201s	user 0.021s	sys 0.000s
I20260709 12:30:10.950057 17393 raft_consensus.cc:359] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:10.951174 17393 raft_consensus.cc:385] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:10.951471 17393 raft_consensus.cc:740] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.952157 17393 consensus_queue.cc:260] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:10.955032 17393 ts_tablet_manager.cc:1434] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:10.956167 17393 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:10.938232 17390 raft_consensus.cc:740] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:10.959208 17390 consensus_queue.cc:260] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:10.961638 17390 ts_tablet_manager.cc:1434] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.005s
I20260709 12:30:10.965601 17391 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:10.977212 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5746c5442b934f38910321caaa1db9a8" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:10.979910 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5746c5442b934f38910321caaa1db9a8" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:10.980749 17270 raft_consensus.cc:2468] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:10.981775 17389 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:10.984594 17389 ts_tablet_manager.cc:1403] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.145s	user 0.013s	sys 0.013s
W20260709 12:30:10.982709 17086 leader_election.cc:343] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:10.986995 17179 tablet_service.cc:1511] Processing CreateTablet for tablet cc02a3ce008e456caafa4f8ba81dcead (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:30:10.974706 17390 tablet_bootstrap.cc:492] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:10.992816 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 2f61a82e0eef4a83ad047d6dd2ca75f8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:30:10.997285 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc02a3ce008e456caafa4f8ba81dcead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:10.999044 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f61a82e0eef4a83ad047d6dd2ca75f8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.008219 17086 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 44859.2us
I20260709 12:30:11.009815 17393 tablet_bootstrap.cc:654] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.011466 17260 tablet_service.cc:1511] Processing CreateTablet for tablet cc02a3ce008e456caafa4f8ba81dcead (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:30:11.012050 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 2f61a82e0eef4a83ad047d6dd2ca75f8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:30:11.015559 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:11.016506 17270 raft_consensus.cc:2468] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:11.017786 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f61a82e0eef4a83ad047d6dd2ca75f8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.019070 17159 leader_election.cc:343] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.019922 17159 leader_election.cc:304] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:11.025027 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc02a3ce008e456caafa4f8ba81dcead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.021567 17396 raft_consensus.cc:493] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.028573 17396 raft_consensus.cc:515] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.033388 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:11.035318 17235 leader_election.cc:343] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.043078 17396 leader_election.cc:290] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:11.043766 17396 raft_consensus.cc:2804] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:11.044114 17396 raft_consensus.cc:493] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:11.044462 17396 raft_consensus.cc:3060] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.051102 17084 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:11.052814 17400 raft_consensus.cc:2804] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:11.059685 17400 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:11.060061 17400 raft_consensus.cc:3060] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.051254 17389 raft_consensus.cc:359] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.063117 17389 raft_consensus.cc:385] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.063501 17389 raft_consensus.cc:740] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.073374 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.085134 17391 tablet_bootstrap.cc:654] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.086577 17253 tablet_service.cc:1511] Processing CreateTablet for tablet b7ffe1eaf86b4aa8bfa40bb0defe1cfd (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:30:11.088127 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ffe1eaf86b4aa8bfa40bb0defe1cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.093250 17390 tablet_bootstrap.cc:654] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.078374 17389 consensus_queue.cc:260] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.098702 17399 raft_consensus.cc:493] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.098582 17326 tablet_service.cc:1511] Processing CreateTablet for tablet b7ffe1eaf86b4aa8bfa40bb0defe1cfd (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:30:11.099249 17105 tablet_service.cc:1511] Processing CreateTablet for tablet b7ffe1eaf86b4aa8bfa40bb0defe1cfd (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:30:11.099321 17399 raft_consensus.cc:515] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.100096 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ffe1eaf86b4aa8bfa40bb0defe1cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.100778 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7ffe1eaf86b4aa8bfa40bb0defe1cfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.101894 17399 leader_election.cc:290] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:11.103979 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.104959 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:11.104667 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.108399 17393 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:11.108322 17389 ts_tablet_manager.cc:1434] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.113s	user 0.009s	sys 0.000s
W20260709 12:30:11.113718 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 93871.3us
I20260709 12:30:11.115442 17396 raft_consensus.cc:515] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.123373 17389 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:11.117020 17393 ts_tablet_manager.cc:1403] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.161s	user 0.028s	sys 0.004s
I20260709 12:30:11.129093 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:11.132442 17393 raft_consensus.cc:359] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.133704 17393 raft_consensus.cc:385] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.134084 17393 raft_consensus.cc:740] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.136483 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.137236 17119 raft_consensus.cc:2468] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
W20260709 12:30:11.138741 17307 leader_election.cc:343] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.140026 17307 leader_election.cc:343] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.140925 17307 leader_election.cc:343] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.141011 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:11.141638 17307 leader_election.cc:304] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:11.141947 17196 raft_consensus.cc:2468] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:11.142573 17185 tablet_service.cc:1511] Processing CreateTablet for tablet 2f61a82e0eef4a83ad047d6dd2ca75f8 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:30:11.143967 17308 leader_election.cc:304] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:11.144356 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f61a82e0eef4a83ad047d6dd2ca75f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.144861 17400 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.145476 17399 raft_consensus.cc:2804] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:11.146054 17399 raft_consensus.cc:493] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:11.146883 17121 raft_consensus.cc:2393] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:11.149382 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 440c7aeff35f4f3988c3eebfef1c6acf (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:30:11.150854 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440c7aeff35f4f3988c3eebfef1c6acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.146440 17399 raft_consensus.cc:3060] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.151774 17329 tablet_service.cc:1511] Processing CreateTablet for tablet 440c7aeff35f4f3988c3eebfef1c6acf (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:30:11.153115 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440c7aeff35f4f3988c3eebfef1c6acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.154718 17233 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:11.156056 17401 raft_consensus.cc:3060] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.155725 17180 tablet_service.cc:1511] Processing CreateTablet for tablet 440c7aeff35f4f3988c3eebfef1c6acf (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:30:11.159353 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 440c7aeff35f4f3988c3eebfef1c6acf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.161764 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5746c5442b934f38910321caaa1db9a8" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:11.161921 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5746c5442b934f38910321caaa1db9a8" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:11.162803 17270 raft_consensus.cc:3060] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.163548 17391 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:11.164137 17391 ts_tablet_manager.cc:1403] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.263s	user 0.009s	sys 0.023s
I20260709 12:30:11.167457 17159 leader_election.cc:304] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:11.168007 17401 raft_consensus.cc:2749] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.170544 17389 tablet_bootstrap.cc:654] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.172676 17270 raft_consensus.cc:2468] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:11.174700 17418 raft_consensus.cc:2804] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:11.175163 17418 raft_consensus.cc:493] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:11.175529 17418 raft_consensus.cc:3060] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.176206 17396 leader_election.cc:290] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:11.181758 17102 tablet_service.cc:1511] Processing CreateTablet for tablet 3374bfc110c642518353f59ecdcd8a05 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
W20260709 12:30:11.182901 17159 leader_election.cc:343] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.183215 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3374bfc110c642518353f59ecdcd8a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.184037 17159 leader_election.cc:304] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:11.173561 17393 consensus_queue.cc:260] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.167367 17391 raft_consensus.cc:359] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.185087 17391 raft_consensus.cc:385] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.185438 17391 raft_consensus.cc:740] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.185758 17399 raft_consensus.cc:515] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.186347 17391 consensus_queue.cc:260] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.173058 17390 tablet_bootstrap.cc:492] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:11.187795 17400 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:11.188378 17390 ts_tablet_manager.cc:1403] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.222s	user 0.010s	sys 0.009s
I20260709 12:30:11.189127 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
W20260709 12:30:11.191859 17086 leader_election.cc:343] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.193010 17390 raft_consensus.cc:359] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.193140 17327 tablet_service.cc:1511] Processing CreateTablet for tablet 458688757584439eb735f41c9b0e9eaa (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:30:11.193514 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 3374bfc110c642518353f59ecdcd8a05 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:30:11.194638 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458688757584439eb735f41c9b0e9eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.195030 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3374bfc110c642518353f59ecdcd8a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.196334 17393 ts_tablet_manager.cc:1434] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.067s	user 0.007s	sys 0.000s
I20260709 12:30:11.199097 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:11.199415 17396 raft_consensus.cc:2804] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:11.204161 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:11.204851 17196 raft_consensus.cc:3060] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.205783 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 458688757584439eb735f41c9b0e9eaa (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:30:11.209617 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458688757584439eb735f41c9b0e9eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.193847 17390 raft_consensus.cc:385] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.213488 17390 raft_consensus.cc:740] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.214464 17390 consensus_queue.cc:260] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.215106 17196 raft_consensus.cc:2468] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:11.216699 17308 leader_election.cc:304] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
W20260709 12:30:11.216929 17159 leader_election.cc:343] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.218623 17393 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:11.219523 17179 tablet_service.cc:1511] Processing CreateTablet for tablet 3374bfc110c642518353f59ecdcd8a05 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:30:11.220973 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3374bfc110c642518353f59ecdcd8a05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.224408 17428 raft_consensus.cc:2749] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.225575 17428 raft_consensus.cc:2804] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:11.226004 17428 raft_consensus.cc:697] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:11.226197 17178 tablet_service.cc:1511] Processing CreateTablet for tablet 458688757584439eb735f41c9b0e9eaa (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:30:11.226923 17428 consensus_queue.cc:237] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.227564 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 458688757584439eb735f41c9b0e9eaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.233345 17399 leader_election.cc:290] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:11.233276 17396 raft_consensus.cc:697] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:11.241997 17429 raft_consensus.cc:493] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.242636 17429 raft_consensus.cc:515] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.244765 17389 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:11.245395 17389 ts_tablet_manager.cc:1403] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.122s	user 0.016s	sys 0.000s
I20260709 12:30:11.245970 17429 leader_election.cc:290] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:11.247965 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:11.248919 17389 raft_consensus.cc:359] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:11.249432 17308 leader_election.cc:343] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.249703 17389 raft_consensus.cc:385] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.254942 17260 tablet_service.cc:1511] Processing CreateTablet for tablet 0b709ddd01784d588100b08085474cd7 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:30:11.256366 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b709ddd01784d588100b08085474cd7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.241782 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 100894us
I20260709 12:30:11.260409 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:11.261109 17121 raft_consensus.cc:3060] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.270136 17121 raft_consensus.cc:2468] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:11.273092 17270 raft_consensus.cc:2468] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:11.274575 17084 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:11.276023 17425 raft_consensus.cc:2804] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:11.276791 17425 raft_consensus.cc:697] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:11.277705 17425 consensus_queue.cc:237] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.278378 17393 tablet_bootstrap.cc:654] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.279496 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:11.250080 17389 raft_consensus.cc:740] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.286096 17327 tablet_service.cc:1511] Processing CreateTablet for tablet 0b709ddd01784d588100b08085474cd7 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:30:11.287544 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b709ddd01784d588100b08085474cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.289719 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 0b709ddd01784d588100b08085474cd7 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:30:11.289737 17181 tablet_service.cc:1511] Processing CreateTablet for tablet bc38664bb0494fe080bd97ce12c2481b (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:30:11.291145 17389 consensus_queue.cc:260] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.291953 17181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc38664bb0494fe080bd97ce12c2481b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.291688 17251 tablet_service.cc:1511] Processing CreateTablet for tablet bc38664bb0494fe080bd97ce12c2481b (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 12:30:11.294977 17307 leader_election.cc:343] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.295470 17307 leader_election.cc:304] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:11.296411 17418 raft_consensus.cc:515] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.280608 17396 consensus_queue.cc:237] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.298713 17424 raft_consensus.cc:2749] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.299580 17418 leader_election.cc:290] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:11.295399 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc38664bb0494fe080bd97ce12c2481b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.306800 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
W20260709 12:30:11.308787 17159 leader_election.cc:343] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.309345 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:11.310021 17121 raft_consensus.cc:3060] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.310429 17389 ts_tablet_manager.cc:1434] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.065s	user 0.007s	sys 0.002s
I20260709 12:30:11.307062 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b709ddd01784d588100b08085474cd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.311192 17102 tablet_service.cc:1511] Processing CreateTablet for tablet d06ba733fa2c4e5385d5839f67039a97 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:30:11.316013 17180 tablet_service.cc:1511] Processing CreateTablet for tablet d06ba733fa2c4e5385d5839f67039a97 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:30:11.317466 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ba733fa2c4e5385d5839f67039a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.323081 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ba733fa2c4e5385d5839f67039a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.325069 17018 catalog_manager.cc:5697] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:11.314000 17389 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:11.329790 17391 ts_tablet_manager.cc:1434] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.165s	user 0.020s	sys 0.006s
I20260709 12:30:11.331552 17391 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:11.331645 17260 tablet_service.cc:1511] Processing CreateTablet for tablet d06ba733fa2c4e5385d5839f67039a97 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:30:11.333001 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ba733fa2c4e5385d5839f67039a97. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.331622 17329 tablet_service.cc:1511] Processing CreateTablet for tablet bc38664bb0494fe080bd97ce12c2481b (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:30:11.336651 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc38664bb0494fe080bd97ce12c2481b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.338706 17390 ts_tablet_manager.cc:1434] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.150s	user 0.006s	sys 0.004s
I20260709 12:30:11.340068 17401 raft_consensus.cc:493] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.340626 17401 raft_consensus.cc:515] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.341014 17390 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:11.344596 17401 leader_election.cc:290] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:11.345384 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.347263 17393 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:11.347914 17393 ts_tablet_manager.cc:1403] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.151s	user 0.011s	sys 0.003s
I20260709 12:30:11.349287 17418 raft_consensus.cc:493] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.349891 17418 raft_consensus.cc:515] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.351680 17019 catalog_manager.cc:5697] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:11.353067 17418 leader_election.cc:290] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:11.356437 17233 leader_election.cc:343] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.374171 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:11.361120 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:11.376210 17159 leader_election.cc:343] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.366314 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 9ba13b6455354f64a81d0be356cf1982 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:30:11.376467 17389 tablet_bootstrap.cc:654] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.377180 17121 raft_consensus.cc:2468] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:11.369570 17105 tablet_service.cc:1511] Processing CreateTablet for tablet 9ba13b6455354f64a81d0be356cf1982 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:30:11.377499 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ba13b6455354f64a81d0be356cf1982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.371855 17181 tablet_service.cc:1511] Processing CreateTablet for tablet 9ba13b6455354f64a81d0be356cf1982 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:30:11.373265 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:11.382665 17181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ba13b6455354f64a81d0be356cf1982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.380853 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ba13b6455354f64a81d0be356cf1982. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.386772 17161 leader_election.cc:343] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.389634 17161 leader_election.cc:304] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
W20260709 12:30:11.391010 17234 leader_election.cc:343] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.391330 17159 leader_election.cc:304] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310
I20260709 12:30:11.391604 17234 leader_election.cc:304] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:11.395391 17418 raft_consensus.cc:2804] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:11.395962 17418 raft_consensus.cc:697] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:11.396842 17418 consensus_queue.cc:237] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.394781 17396 raft_consensus.cc:2749] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.394052 17401 raft_consensus.cc:2749] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.400215 17391 tablet_bootstrap.cc:654] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.401718 17017 catalog_manager.cc:5697] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:11.416074 17393 raft_consensus.cc:359] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.417399 17393 raft_consensus.cc:385] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.417814 17393 raft_consensus.cc:740] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.424647 17253 tablet_service.cc:1511] Processing CreateTablet for tablet a100ea0ba8a74d14bdfc772d6059f39c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:30:11.426074 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a100ea0ba8a74d14bdfc772d6059f39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.426132 17390 tablet_bootstrap.cc:654] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.448706 17393 consensus_queue.cc:260] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.461314 17401 raft_consensus.cc:493] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.461284 17326 tablet_service.cc:1511] Processing CreateTablet for tablet a100ea0ba8a74d14bdfc772d6059f39c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:30:11.461987 17401 raft_consensus.cc:515] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.462802 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a100ea0ba8a74d14bdfc772d6059f39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.466264 17401 leader_election.cc:290] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:11.467232 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.467573 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:11.469231 17233 leader_election.cc:343] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.470137 17234 leader_election.cc:343] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.470788 17234 leader_election.cc:304] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:11.473774 17102 tablet_service.cc:1511] Processing CreateTablet for tablet a100ea0ba8a74d14bdfc772d6059f39c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:30:11.475629 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a100ea0ba8a74d14bdfc772d6059f39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.478636 17401 raft_consensus.cc:2749] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.478183 17260 tablet_service.cc:1511] Processing CreateTablet for tablet e58121712c0045238ff325078dcf0653 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:30:11.480221 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58121712c0045238ff325078dcf0653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.481426 17393 ts_tablet_manager.cc:1434] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.133s	user 0.012s	sys 0.001s
I20260709 12:30:11.482682 17393 tablet_bootstrap.cc:492] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:11.481526 17180 tablet_service.cc:1511] Processing CreateTablet for tablet e58121712c0045238ff325078dcf0653 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:30:11.490751 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58121712c0045238ff325078dcf0653. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.489434 17393 tablet_bootstrap.cc:654] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.531672 17389 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:11.532399 17389 ts_tablet_manager.cc:1403] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.221s	user 0.014s	sys 0.015s
I20260709 12:30:11.553959 17326 tablet_service.cc:1511] Processing CreateTablet for tablet e58121712c0045238ff325078dcf0653 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:30:11.555397 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e58121712c0045238ff325078dcf0653. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:30:11.541734 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 72498.1us
I20260709 12:30:11.558400 17390 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:11.559132 17390 ts_tablet_manager.cc:1403] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.218s	user 0.015s	sys 0.005s
I20260709 12:30:11.567571 17390 raft_consensus.cc:359] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.568220 17390 raft_consensus.cc:385] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.568498 17390 raft_consensus.cc:740] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.569192 17390 consensus_queue.cc:260] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.571643 17390 ts_tablet_manager.cc:1434] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 12:30:11.572427 17391 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:11.572695 17390 tablet_bootstrap.cc:492] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:11.573108 17391 ts_tablet_manager.cc:1403] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.242s	user 0.013s	sys 0.010s
I20260709 12:30:11.579619 17389 raft_consensus.cc:359] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.581638 17389 raft_consensus.cc:385] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.582257 17389 raft_consensus.cc:740] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.586025 17393 tablet_bootstrap.cc:492] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:11.586725 17393 ts_tablet_manager.cc:1403] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.104s	user 0.009s	sys 0.012s
I20260709 12:30:11.586695 17389 consensus_queue.cc:260] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.590184 17393 raft_consensus.cc:359] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.587971 17391 raft_consensus.cc:359] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.593246 17390 tablet_bootstrap.cc:654] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.593499 17393 raft_consensus.cc:385] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.593628 17391 raft_consensus.cc:385] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.594257 17393 raft_consensus.cc:740] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.594344 17391 raft_consensus.cc:740] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.595188 17393 consensus_queue.cc:260] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.595321 17391 consensus_queue.cc:260] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.597653 17393 ts_tablet_manager.cc:1434] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:30:11.598744 17393 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:11.589745 17389 ts_tablet_manager.cc:1434] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.057s	user 0.011s	sys 0.001s
I20260709 12:30:11.640838 17389 tablet_bootstrap.cc:492] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:11.658361 17181 tablet_service.cc:1511] Processing CreateTablet for tablet 9b33f790d3fc4b0099ea91c45cc6fd11 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:30:11.661198 17181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b33f790d3fc4b0099ea91c45cc6fd11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.667861 17180 tablet_service.cc:1511] Processing CreateTablet for tablet ec4250a950114da1bce4b46571dc97cc (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:30:11.669431 17180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4250a950114da1bce4b46571dc97cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.646003 17105 tablet_service.cc:1511] Processing CreateTablet for tablet 9b33f790d3fc4b0099ea91c45cc6fd11 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:30:11.669641 17326 tablet_service.cc:1511] Processing CreateTablet for tablet ec4250a950114da1bce4b46571dc97cc (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:30:11.675056 17105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b33f790d3fc4b0099ea91c45cc6fd11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.675406 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4250a950114da1bce4b46571dc97cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.684234 17179 tablet_service.cc:1511] Processing CreateTablet for tablet d62bfda640da40f9ab6e7b1da9479a5c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:30:11.690567 17179 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d62bfda640da40f9ab6e7b1da9479a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.694645 17391 ts_tablet_manager.cc:1434] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:30:11.696591 17391 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:11.698792 17260 tablet_service.cc:1511] Processing CreateTablet for tablet 9b33f790d3fc4b0099ea91c45cc6fd11 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:30:11.699368 17102 tablet_service.cc:1511] Processing CreateTablet for tablet ec4250a950114da1bce4b46571dc97cc (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:30:11.699905 17329 tablet_service.cc:1511] Processing CreateTablet for tablet 51de8b9a56eb439e9c25052d3d9587f9 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:30:11.700631 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b33f790d3fc4b0099ea91c45cc6fd11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.700798 17102 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec4250a950114da1bce4b46571dc97cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.701687 17329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51de8b9a56eb439e9c25052d3d9587f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.716265 17327 tablet_service.cc:1511] Processing CreateTablet for tablet d62bfda640da40f9ab6e7b1da9479a5c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:30:11.716998 17391 tablet_bootstrap.cc:654] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.717999 17327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d62bfda640da40f9ab6e7b1da9479a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.724004 17253 tablet_service.cc:1511] Processing CreateTablet for tablet 51de8b9a56eb439e9c25052d3d9587f9 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:30:11.725626 17253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51de8b9a56eb439e9c25052d3d9587f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.728130 17251 tablet_service.cc:1511] Processing CreateTablet for tablet d62bfda640da40f9ab6e7b1da9479a5c (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:30:11.729825 17251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d62bfda640da40f9ab6e7b1da9479a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.731047 17393 tablet_bootstrap.cc:654] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.728119 17108 tablet_service.cc:1511] Processing CreateTablet for tablet 51de8b9a56eb439e9c25052d3d9587f9 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:30:11.735810 17108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51de8b9a56eb439e9c25052d3d9587f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.741355 17106 tablet_service.cc:1511] Processing CreateTablet for tablet 6c70b2a46ca14b57b55e8eaec58cd038 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:30:11.743002 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c70b2a46ca14b57b55e8eaec58cd038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.759336 17390 tablet_bootstrap.cc:492] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:11.760398 17390 ts_tablet_manager.cc:1403] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.188s	user 0.014s	sys 0.007s
I20260709 12:30:11.763089 17399 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.775378 17399 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:11.757287 17428 consensus_queue.cc:1048] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:30:11.777184 17178 tablet_service.cc:1511] Processing CreateTablet for tablet 6c70b2a46ca14b57b55e8eaec58cd038 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:30:11.779577 17178 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c70b2a46ca14b57b55e8eaec58cd038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.784258 17389 tablet_bootstrap.cc:654] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:11.783356 17017 catalog_manager.cc:5697] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:11.779485 17086 consensus_peers.cc:597] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 -> Peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Couldn't send request to peer 7c0f44cba62f491392d41ca89376139a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:11.796808 17399 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:11.799811 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.799851 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:11.802460 17307 leader_election.cc:343] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.803323 17308 leader_election.cc:343] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:11.803920 17308 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:11.805519 17418 consensus_queue.cc:1048] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:11.810699 17399 raft_consensus.cc:2749] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:11.814684 17390 raft_consensus.cc:359] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.818873 17390 raft_consensus.cc:385] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:11.819402 17159 consensus_peers.cc:597] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:11.820945 17260 tablet_service.cc:1511] Processing CreateTablet for tablet 6c70b2a46ca14b57b55e8eaec58cd038 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:30:11.823001 17260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c70b2a46ca14b57b55e8eaec58cd038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.823516 17326 tablet_service.cc:1511] Processing CreateTablet for tablet 90bb36eac77e43ad8fc0568e85913a36 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:30:11.824795 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90bb36eac77e43ad8fc0568e85913a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.828305 17104 tablet_service.cc:1511] Processing CreateTablet for tablet 90bb36eac77e43ad8fc0568e85913a36 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:30:11.828680 17185 tablet_service.cc:1511] Processing CreateTablet for tablet 90bb36eac77e43ad8fc0568e85913a36 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:30:11.829887 17104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90bb36eac77e43ad8fc0568e85913a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.830250 17185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 90bb36eac77e43ad8fc0568e85913a36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.833132 17257 tablet_service.cc:1511] Processing CreateTablet for tablet d14098588e6e4421bbf71e16dea54da0 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:30:11.836007 17399 consensus_queue.cc:1048] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:11.839342 17418 consensus_queue.cc:1048] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:11.845018 17418 raft_consensus.cc:493] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.845520 17418 raft_consensus.cc:515] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:11.861017 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:11.862601 17257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d14098588e6e4421bbf71e16dea54da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:11.847988 17418 leader_election.cc:290] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:11.819350 17390 raft_consensus.cc:740] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.862799 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:11.900627 17393 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:11.901563 17393 ts_tablet_manager.cc:1403] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.303s	user 0.007s	sys 0.012s
W20260709 12:30:11.901651 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:11.903123 17390 consensus_queue.cc:260] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.920894 17390 ts_tablet_manager.cc:1434] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.160s	user 0.009s	sys 0.002s
I20260709 12:30:11.923647 17269 raft_consensus.cc:2468] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:11.937822 17425 raft_consensus.cc:493] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:11.939756 17390 tablet_bootstrap.cc:492] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
W20260709 12:30:11.892056 17159 leader_election.cc:343] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:11.940675 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 55763.8us
I20260709 12:30:11.942250 17159 leader_election.cc:304] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:11.943527 17393 raft_consensus.cc:359] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.945003 17393 raft_consensus.cc:385] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:11.945394 17393 raft_consensus.cc:740] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.950642 17425 raft_consensus.cc:515] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.955850 17396 raft_consensus.cc:2804] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:11.956586 17396 raft_consensus.cc:493] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:11.954159 17425 leader_election.cc:290] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:11.989060 17391 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:11.989642 17389 tablet_bootstrap.cc:492] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:11.989662 17391 ts_tablet_manager.cc:1403] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.294s	user 0.035s	sys 0.035s
I20260709 12:30:11.990177 17389 ts_tablet_manager.cc:1403] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.351s	user 0.060s	sys 0.033s
I20260709 12:30:11.973016 17396 raft_consensus.cc:3060] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:11.998610 17389 raft_consensus.cc:359] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:11.999964 17389 raft_consensus.cc:385] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.000432 17389 raft_consensus.cc:740] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:11.993522 17391 raft_consensus.cc:359] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.001766 17391 raft_consensus.cc:385] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.002239 17391 raft_consensus.cc:740] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
W20260709 12:30:11.975428 17005 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 57920.2us
I20260709 12:30:11.955896 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.010344 17344 raft_consensus.cc:2468] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:11.975368 17393 consensus_queue.cc:260] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:11.986299 17425 consensus_queue.cc:1048] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.029s
I20260709 12:30:12.007207 17389 consensus_queue.cc:260] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:12.038547 17084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 82177.1us
W20260709 12:30:12.029165 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 73374.5us
I20260709 12:30:12.047470 17326 tablet_service.cc:1511] Processing CreateTablet for tablet d14098588e6e4421bbf71e16dea54da0 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:30:12.048408 17326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d14098588e6e4421bbf71e16dea54da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.048772 17440 raft_consensus.cc:493] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.049401 17440 raft_consensus.cc:515] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.043284 17086 leader_election.cc:304] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:12.053457 17393 ts_tablet_manager.cc:1434] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.151s	user 0.010s	sys 0.001s
I20260709 12:30:12.053891 17455 raft_consensus.cc:2804] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.054419 17455 raft_consensus.cc:493] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.054819 17455 raft_consensus.cc:3060] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.055747 17106 tablet_service.cc:1511] Processing CreateTablet for tablet d14098588e6e4421bbf71e16dea54da0 (DEFAULT_TABLE table=test-workload [id=25d08fe26a3244b7a175a225cd49ff33]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:30:12.057185 17106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d14098588e6e4421bbf71e16dea54da0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:12.055042 17393 tablet_bootstrap.cc:492] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.069576 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:12.070385 17159 consensus_peers.cc:597] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:12.070469 17269 raft_consensus.cc:2468] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.041851 17391 consensus_queue.cc:260] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.107451 17440 leader_election.cc:290] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.108816 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af900dfc2784f8781502931a17eaa79" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.109727 17344 raft_consensus.cc:2468] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:12.113070 17235 leader_election.cc:304] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:12.115041 17440 raft_consensus.cc:2804] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.115551 17440 raft_consensus.cc:493] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.115877 17440 raft_consensus.cc:3060] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.123519 17390 tablet_bootstrap.cc:654] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.146085 17463 raft_consensus.cc:493] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.146449 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af900dfc2784f8781502931a17eaa79" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.146782 17463 raft_consensus.cc:515] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.147426 17195 raft_consensus.cc:2468] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:12.155238 17463 leader_election.cc:290] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:12.164351 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.165167 17268 raft_consensus.cc:2468] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.166630 17084 leader_election.cc:304] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:12.167498 17463 raft_consensus.cc:2804] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.167963 17463 raft_consensus.cc:493] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.168309 17463 raft_consensus.cc:3060] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.180088 17389 ts_tablet_manager.cc:1434] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.190s	user 0.021s	sys 0.005s
I20260709 12:30:12.181622 17389 tablet_bootstrap.cc:492] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.153370 17396 raft_consensus.cc:515] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.191479 17391 ts_tablet_manager.cc:1434] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.201s	user 0.006s	sys 0.011s
I20260709 12:30:12.192828 17391 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.200126 17393 tablet_bootstrap.cc:654] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.173339 17455 raft_consensus.cc:515] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.206024 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:12.190164 17440 raft_consensus.cc:515] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.212869 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af900dfc2784f8781502931a17eaa79" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:12.213653 17195 raft_consensus.cc:3060] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.213909 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0af900dfc2784f8781502931a17eaa79" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.214213 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:12.214697 17344 raft_consensus.cc:3060] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.214773 17270 raft_consensus.cc:3060] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.217953 17389 tablet_bootstrap.cc:654] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.222043 17463 raft_consensus.cc:515] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.223923 17344 raft_consensus.cc:2468] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:12.224576 17440 leader_election.cc:290] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.224973 17195 raft_consensus.cc:2468] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:12.228040 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:12.229072 17119 raft_consensus.cc:2393] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:12.229642 17396 leader_election.cc:290] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:12.230229 17463 leader_election.cc:290] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:12.236485 17234 leader_election.cc:304] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.238176 17440 raft_consensus.cc:2804] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.241353 17440 raft_consensus.cc:697] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:12.242112 17393 tablet_bootstrap.cc:492] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.242759 17393 ts_tablet_manager.cc:1403] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.188s	user 0.008s	sys 0.011s
I20260709 12:30:12.245376 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:12.245985 17393 raft_consensus.cc:359] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.246414 17268 raft_consensus.cc:3060] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.246904 17393 raft_consensus.cc:385] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.247653 17393 raft_consensus.cc:740] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.248509 17393 consensus_queue.cc:260] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.250926 17393 ts_tablet_manager.cc:1434] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:12.252214 17393 tablet_bootstrap.cc:492] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.241456 17270 raft_consensus.cc:2468] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:12.254876 17159 leader_election.cc:304] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:12.255070 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.255653 17396 raft_consensus.cc:2804] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.256203 17396 raft_consensus.cc:697] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:12.256305 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:12.257259 17396 consensus_queue.cc:237] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.258663 17085 leader_election.cc:304] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:12.260022 17463 raft_consensus.cc:2749] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.262768 17393 tablet_bootstrap.cc:654] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.267129 17391 tablet_bootstrap.cc:654] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.271114 17440 consensus_queue.cc:237] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.274533 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.281786 17344 raft_consensus.cc:3060] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.273900 17018 catalog_manager.cc:5697] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.275028 17455 leader_election.cc:290] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.277988 17393 tablet_bootstrap.cc:492] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.287536 17393 ts_tablet_manager.cc:1403] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.036s	user 0.004s	sys 0.013s
I20260709 12:30:12.290616 17390 tablet_bootstrap.cc:492] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.291278 17390 ts_tablet_manager.cc:1403] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.369s	user 0.025s	sys 0.043s
I20260709 12:30:12.291203 17393 raft_consensus.cc:359] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.292173 17393 raft_consensus.cc:385] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.292464 17393 raft_consensus.cc:740] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.286710 17268 raft_consensus.cc:2468] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:12.295225 17084 leader_election.cc:304] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:12.296195 17455 raft_consensus.cc:2804] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.296761 17455 raft_consensus.cc:697] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:12.298615 17455 consensus_queue.cc:237] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.298334 17390 raft_consensus.cc:359] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.299808 17390 raft_consensus.cc:385] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.300132 17390 raft_consensus.cc:740] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.301348 17390 consensus_queue.cc:260] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.302534 17344 raft_consensus.cc:2468] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:12.293244 17393 consensus_queue.cc:260] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.305719 17393 ts_tablet_manager.cc:1434] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:30:12.307126 17393 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.307521 17391 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:12.308161 17391 ts_tablet_manager.cc:1403] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.003s
I20260709 12:30:12.311434 17391 raft_consensus.cc:359] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.312139 17390 ts_tablet_manager.cc:1434] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:30:12.312352 17391 raft_consensus.cc:385] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.312999 17391 raft_consensus.cc:740] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.311714 17018 catalog_manager.cc:5697] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.314252 17389 tablet_bootstrap.cc:492] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.314038 17391 consensus_queue.cc:260] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.314886 17389 ts_tablet_manager.cc:1403] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.134s	user 0.003s	sys 0.016s
I20260709 12:30:12.316948 17391 ts_tablet_manager.cc:1434] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:12.317320 17390 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:12.318354 17391 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.321264 17389 raft_consensus.cc:359] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.327646 17391 tablet_bootstrap.cc:654] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.328111 17389 raft_consensus.cc:385] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.328583 17389 raft_consensus.cc:740] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.329553 17389 consensus_queue.cc:260] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.332290 17389 ts_tablet_manager.cc:1434] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.004s
I20260709 12:30:12.334851 17390 tablet_bootstrap.cc:654] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.329983 17019 catalog_manager.cc:5697] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:12.336952 17393 tablet_bootstrap.cc:654] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.342159 17389 tablet_bootstrap.cc:492] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.350545 17393 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.351076 17390 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.351228 17393 ts_tablet_manager.cc:1403] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.044s	user 0.006s	sys 0.007s
I20260709 12:30:12.351619 17390 ts_tablet_manager.cc:1403] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.035s	user 0.020s	sys 0.000s
I20260709 12:30:12.355065 17390 raft_consensus.cc:359] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.355997 17390 raft_consensus.cc:385] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.355914 17393 raft_consensus.cc:359] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.356598 17390 raft_consensus.cc:740] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.356829 17393 raft_consensus.cc:385] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.357195 17393 raft_consensus.cc:740] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.357779 17390 consensus_queue.cc:260] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.359767 17389 tablet_bootstrap.cc:654] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.365186 17390 ts_tablet_manager.cc:1434] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:30:12.366859 17390 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:12.362857 17393 consensus_queue.cc:260] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.374549 17390 tablet_bootstrap.cc:654] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.381898 17393 ts_tablet_manager.cc:1434] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.009s
I20260709 12:30:12.383992 17393 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.388958 17389 tablet_bootstrap.cc:492] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.389619 17389 ts_tablet_manager.cc:1403] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.006s
I20260709 12:30:12.391683 17440 raft_consensus.cc:493] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.392279 17440 raft_consensus.cc:515] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.392887 17389 raft_consensus.cc:359] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.393575 17389 raft_consensus.cc:385] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.393816 17389 raft_consensus.cc:740] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.394631 17440 leader_election.cc:290] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.398015 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26acf9f75043088e62624ad9f61c2e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.398051 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26acf9f75043088e62624ad9f61c2e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.398461 17389 consensus_queue.cc:260] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.399103 17344 raft_consensus.cc:2468] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:12.399721 17195 raft_consensus.cc:2468] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:12.401064 17389 ts_tablet_manager.cc:1434] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:30:12.401301 17235 leader_election.cc:304] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:12.403573 17393 tablet_bootstrap.cc:654] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.403510 17440 raft_consensus.cc:2804] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.404125 17440 raft_consensus.cc:493] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.404549 17440 raft_consensus.cc:3060] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.406792 17389 tablet_bootstrap.cc:492] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.413584 17440 raft_consensus.cc:515] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.391659 17390 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.413712 17389 tablet_bootstrap.cc:654] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.415095 17390 ts_tablet_manager.cc:1403] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.003s
I20260709 12:30:12.419023 17390 raft_consensus.cc:359] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.422256 17390 raft_consensus.cc:385] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.422869 17390 raft_consensus.cc:740] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.423365 17391 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:12.424425 17390 consensus_queue.cc:260] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.425103 17389 tablet_bootstrap.cc:492] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.426129 17389 ts_tablet_manager.cc:1403] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.008s
I20260709 12:30:12.426477 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26acf9f75043088e62624ad9f61c2e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:12.424464 17391 ts_tablet_manager.cc:1403] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.106s	user 0.031s	sys 0.058s
I20260709 12:30:12.427372 17195 raft_consensus.cc:3060] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.429605 17389 raft_consensus.cc:359] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.430478 17389 raft_consensus.cc:385] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.430900 17389 raft_consensus.cc:740] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.431166 17393 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.430743 17391 raft_consensus.cc:359] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.431721 17391 raft_consensus.cc:385] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.431809 17393 ts_tablet_manager.cc:1403] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.003s
I20260709 12:30:12.426985 17390 ts_tablet_manager.cc:1434] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:30:12.432003 17389 consensus_queue.cc:260] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.433502 17390 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:12.438824 17393 raft_consensus.cc:359] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.439592 17393 raft_consensus.cc:385] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.439719 17389 ts_tablet_manager.cc:1434] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:30:12.439950 17393 raft_consensus.cc:740] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.440872 17390 tablet_bootstrap.cc:654] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.441521 17389 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.441094 17393 consensus_queue.cc:260] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.432192 17391 raft_consensus.cc:740] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.443501 17391 consensus_queue.cc:260] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.445986 17391 ts_tablet_manager.cc:1434] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.001s
I20260709 12:30:12.447395 17391 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.453586 17195 raft_consensus.cc:2468] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:12.454248 17389 tablet_bootstrap.cc:654] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.454653 17391 tablet_bootstrap.cc:654] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.454825 17234 leader_election.cc:304] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.456104 17474 raft_consensus.cc:2804] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.456594 17474 raft_consensus.cc:697] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:12.457468 17474 consensus_queue.cc:237] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.464674 17393 ts_tablet_manager.cc:1434] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 12:30:12.467180 17390 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.467988 17390 ts_tablet_manager.cc:1403] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.035s	user 0.008s	sys 0.008s
I20260709 12:30:12.468171 17440 leader_election.cc:290] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.471552 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd26acf9f75043088e62624ad9f61c2e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.472235 17344 raft_consensus.cc:3060] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.472358 17390 raft_consensus.cc:359] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.473599 17390 raft_consensus.cc:385] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.473984 17390 raft_consensus.cc:740] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.477236 17391 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:12.477809 17391 ts_tablet_manager.cc:1403] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.031s	user 0.011s	sys 0.003s
I20260709 12:30:12.478094 17389 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.478739 17389 ts_tablet_manager.cc:1403] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.017s
I20260709 12:30:12.481050 17391 raft_consensus.cc:359] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.481786 17391 raft_consensus.cc:385] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.481593 17389 raft_consensus.cc:359] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.482110 17391 raft_consensus.cc:740] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.478104 17393 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.482949 17391 consensus_queue.cc:260] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.475350 17390 consensus_queue.cc:260] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.485141 17016 catalog_manager.cc:5697] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:12.486573 17390 ts_tablet_manager.cc:1434] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 12:30:12.489681 17391 ts_tablet_manager.cc:1434] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.005s
I20260709 12:30:12.482301 17389 raft_consensus.cc:385] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.491165 17389 raft_consensus.cc:740] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.491688 17344 raft_consensus.cc:2468] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:12.492231 17389 consensus_queue.cc:260] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.492580 17390 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:12.500104 17391 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.500965 17389 ts_tablet_manager.cc:1434] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:30:12.498744 17393 tablet_bootstrap.cc:654] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.505025 17389 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.509274 17391 tablet_bootstrap.cc:654] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.512789 17390 tablet_bootstrap.cc:654] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.516919 17393 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.517407 17393 ts_tablet_manager.cc:1403] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.003s
I20260709 12:30:12.517993 17391 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:12.518160 17389 tablet_bootstrap.cc:654] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.518671 17391 ts_tablet_manager.cc:1403] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:30:12.520255 17393 raft_consensus.cc:359] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.521026 17390 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.521023 17393 raft_consensus.cc:385] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.521909 17393 raft_consensus.cc:740] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.521997 17390 ts_tablet_manager.cc:1403] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.004s
I20260709 12:30:12.522168 17391 raft_consensus.cc:359] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.523533 17391 raft_consensus.cc:385] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.523867 17391 raft_consensus.cc:740] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.523192 17393 consensus_queue.cc:260] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.524839 17391 consensus_queue.cc:260] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.525751 17390 raft_consensus.cc:359] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.526336 17393 ts_tablet_manager.cc:1434] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:30:12.526577 17390 raft_consensus.cc:385] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.526985 17390 raft_consensus.cc:740] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.528051 17393 tablet_bootstrap.cc:492] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.533198 17390 consensus_queue.cc:260] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.535912 17393 tablet_bootstrap.cc:654] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.536975 17391 ts_tablet_manager.cc:1434] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:30:12.541723 17390 ts_tablet_manager.cc:1434] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:30:12.544922 17390 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:12.547055 17389 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.547694 17389 ts_tablet_manager.cc:1403] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.012s
I20260709 12:30:12.546473 17393 tablet_bootstrap.cc:492] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.550969 17393 ts_tablet_manager.cc:1403] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.000s
I20260709 12:30:12.551306 17389 raft_consensus.cc:359] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.551971 17390 tablet_bootstrap.cc:654] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.552243 17389 raft_consensus.cc:385] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.552650 17389 raft_consensus.cc:740] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.553534 17389 consensus_queue.cc:260] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.556087 17389 ts_tablet_manager.cc:1434] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:30:12.557358 17389 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.557372 17391 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.566393 17389 tablet_bootstrap.cc:654] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.570634 17391 tablet_bootstrap.cc:654] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.579819 17389 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.580667 17389 ts_tablet_manager.cc:1403] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.009s
I20260709 12:30:12.585183 17389 raft_consensus.cc:359] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.587559 17474 raft_consensus.cc:493] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.586301 17389 raft_consensus.cc:385] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.588472 17389 raft_consensus.cc:740] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.588224 17474 raft_consensus.cc:515] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.589834 17389 consensus_queue.cc:260] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.591828 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:12.592768 17389 ts_tablet_manager.cc:1434] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
W20260709 12:30:12.593727 17233 leader_election.cc:343] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.594947 17474 leader_election.cc:290] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:12.595403 17389 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.596217 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.596719 17391 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:12.597141 17195 raft_consensus.cc:2468] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:12.597345 17391 ts_tablet_manager.cc:1403] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.011s
I20260709 12:30:12.598035 17393 raft_consensus.cc:359] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.598673 17234 leader_election.cc:304] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:12.601202 17391 raft_consensus.cc:359] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.601966 17393 raft_consensus.cc:385] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.602177 17391 raft_consensus.cc:385] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.602710 17391 raft_consensus.cc:740] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.602824 17474 raft_consensus.cc:2804] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.603346 17474 raft_consensus.cc:493] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.603737 17474 raft_consensus.cc:3060] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.603744 17391 consensus_queue.cc:260] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.604635 17389 tablet_bootstrap.cc:654] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.612483 17474 raft_consensus.cc:515] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.602413 17393 raft_consensus.cc:740] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.615367 17393 consensus_queue.cc:260] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.615756 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:12.617213 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
W20260709 12:30:12.617486 17233 leader_election.cc:343] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.617748 17389 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.617753 17393 ts_tablet_manager.cc:1434] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.000s
I20260709 12:30:12.618786 17389 ts_tablet_manager.cc:1403] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.008s
I20260709 12:30:12.619735 17393 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.617930 17195 raft_consensus.cc:3060] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.621878 17455 raft_consensus.cc:493] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.623627 17455 raft_consensus.cc:515] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.623747 17389 raft_consensus.cc:359] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.624445 17389 raft_consensus.cc:385] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.624732 17389 raft_consensus.cc:740] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.626403 17428 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.626978 17428 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.627266 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.628019 17196 raft_consensus.cc:2468] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.629292 17428 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.629474 17390 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.630107 17390 ts_tablet_manager.cc:1403] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.086s	user 0.033s	sys 0.018s
I20260709 12:30:12.629931 17085 leader_election.cc:304] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.630524 17455 leader_election.cc:290] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:12.625577 17389 consensus_queue.cc:260] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.632472 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.630659 17195 raft_consensus.cc:2468] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:12.631531 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.639894 17391 ts_tablet_manager.cc:1434] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.005s
I20260709 12:30:12.669719 17234 leader_election.cc:304] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:12.627302 17393 tablet_bootstrap.cc:654] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:12.669741 17307 leader_election.cc:343] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.641750 17474 leader_election.cc:290] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:12.671090 17396 raft_consensus.cc:493] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.672045 17474 raft_consensus.cc:2804] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.672550 17396 raft_consensus.cc:515] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.672890 17474 raft_consensus.cc:697] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:12.655498 17466 raft_consensus.cc:493] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.673768 17466 raft_consensus.cc:515] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.675961 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.676518 17389 ts_tablet_manager.cc:1434] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.057s	user 0.013s	sys 0.006s
I20260709 12:30:12.676760 17343 raft_consensus.cc:2468] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:12.677771 17389 tablet_bootstrap.cc:492] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.678419 17390 raft_consensus.cc:359] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.668751 17344 raft_consensus.cc:2468] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.679348 17390 raft_consensus.cc:385] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.630961 17463 raft_consensus.cc:2804] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.641132 17440 raft_consensus.cc:493] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.675087 17396 leader_election.cc:290] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.632071 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:12.674072 17474 consensus_queue.cc:237] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.680287 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "795b495c58894c1eb861e953546a6e6d" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.680703 17463 raft_consensus.cc:493] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.681185 17463 raft_consensus.cc:3060] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.679147 17161 leader_election.cc:304] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.676786 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:12.683035 17396 raft_consensus.cc:2804] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.683454 17396 raft_consensus.cc:493] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.683800 17396 raft_consensus.cc:3060] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.679765 17390 raft_consensus.cc:740] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
W20260709 12:30:12.684100 17159 leader_election.cc:343] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.670338 17391 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.685285 17390 consensus_queue.cc:260] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:12.682449 17307 leader_election.cc:343] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.686715 17307 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:12.689275 17428 raft_consensus.cc:2749] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.691236 17390 ts_tablet_manager.cc:1434] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.024s	user 0.001s	sys 0.006s
I20260709 12:30:12.692420 17390 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:12.693111 17391 tablet_bootstrap.cc:654] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.695137 17389 tablet_bootstrap.cc:654] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.680367 17440 raft_consensus.cc:515] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.699347 17455 raft_consensus.cc:493] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.703027 17455 raft_consensus.cc:515] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:12.706094 17235 leader_election.cc:343] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.710942 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.711747 17389 tablet_bootstrap.cc:492] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:12.711711 17195 raft_consensus.cc:2468] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.712409 17389 ts_tablet_manager.cc:1403] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.035s	user 0.012s	sys 0.003s
I20260709 12:30:12.712013 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "795b495c58894c1eb861e953546a6e6d" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.713040 17196 raft_consensus.cc:2468] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:12.713562 17085 leader_election.cc:304] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.714552 17425 raft_consensus.cc:2804] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.714582 17234 leader_election.cc:304] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:12.715171 17425 raft_consensus.cc:493] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.715644 17425 raft_consensus.cc:3060] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.715591 17389 raft_consensus.cc:359] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.716473 17389 raft_consensus.cc:385] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.716800 17389 raft_consensus.cc:740] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.717489 17501 raft_consensus.cc:493] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.718086 17455 leader_election.cc:290] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:12.707451 17466 leader_election.cc:290] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.723407 17466 raft_consensus.cc:2804] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.723816 17466 raft_consensus.cc:493] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.723062 17501 raft_consensus.cc:515] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.724530 17393 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:12.720968 17396 raft_consensus.cc:515] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.721589 17391 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:12.725093 17393 ts_tablet_manager.cc:1403] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.106s	user 0.018s	sys 0.011s
I20260709 12:30:12.725366 17391 ts_tablet_manager.cc:1403] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.055s	user 0.009s	sys 0.008s
I20260709 12:30:12.728267 17393 raft_consensus.cc:359] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.717592 17389 consensus_queue.cc:260] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.724193 17466 raft_consensus.cc:3060] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.721503 17440 leader_election.cc:290] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:12.729008 17393 raft_consensus.cc:385] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.732474 17393 raft_consensus.cc:740] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.728979 17391 raft_consensus.cc:359] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.732978 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:12.733196 17391 raft_consensus.cc:385] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.733295 17393 consensus_queue.cc:260] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.735919 17393 ts_tablet_manager.cc:1434] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:30:12.736089 17390 tablet_bootstrap.cc:654] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.733790 17391 raft_consensus.cc:740] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.733668 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.736943 17458 raft_consensus.cc:493] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.737224 17268 raft_consensus.cc:2468] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.737522 17458 raft_consensus.cc:515] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.737672 17391 consensus_queue.cc:260] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.737215 17393 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:12.739822 17458 leader_election.cc:290] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.740185 17391 ts_tablet_manager.cc:1434] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:30:12.742012 17391 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:12.742621 17396 leader_election.cc:290] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.742777 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.743588 17343 raft_consensus.cc:3060] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.746110 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.746774 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.747002 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.752444 17471 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.752812 17268 raft_consensus.cc:2468] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:12.752840 17342 raft_consensus.cc:2468] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:12.753031 17471 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.754539 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:12.754596 17159 leader_election.cc:304] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.756783 17471 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:12.757321 17159 leader_election.cc:343] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.758322 17458 raft_consensus.cc:2804] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.758751 17458 raft_consensus.cc:493] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.759136 17458 raft_consensus.cc:3060] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:12.758764 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 44301.2us
I20260709 12:30:12.760309 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.761821 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.759104 17471 consensus_queue.cc:1048] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:12.764992 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.765967 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:12.761981 17086 leader_election.cc:343] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.766449 17234 leader_election.cc:304] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:12.766956 17471 raft_consensus.cc:493] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.767779 17471 raft_consensus.cc:515] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.768230 17440 raft_consensus.cc:2749] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.770052 17396 raft_consensus.cc:493] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.770301 17471 leader_election.cc:290] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.770948 17396 raft_consensus.cc:515] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.771198 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.772119 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.772090 17270 raft_consensus.cc:2468] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
W20260709 12:30:12.774115 17086 leader_election.cc:343] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.753788 17393 tablet_bootstrap.cc:654] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.775372 17501 leader_election.cc:290] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.775842 17084 leader_election.cc:304] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:12.730031 17425 raft_consensus.cc:515] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.783739 17343 raft_consensus.cc:2468] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:12.783420 17463 raft_consensus.cc:515] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.783828 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:12.777915 17471 raft_consensus.cc:2804] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.767442 17268 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:12.784672 17195 raft_consensus.cc:3060] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.785705 17396 leader_election.cc:290] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.787508 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:12.788213 17121 raft_consensus.cc:2468] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:12.789085 17391 tablet_bootstrap.cc:654] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:12.786255 17463 leader_election.cc:290] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:12.791764 17159 leader_election.cc:304] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.793606 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:12.794353 17194 raft_consensus.cc:3060] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.794910 17396 raft_consensus.cc:2804] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.795334 17396 raft_consensus.cc:493] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.795689 17396 raft_consensus.cc:3060] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.784700 17471 raft_consensus.cc:493] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.801291 17084 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:12.808037 17018 catalog_manager.cc:5697] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:12.784437 17458 raft_consensus.cc:515] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.801892 17471 raft_consensus.cc:3060] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.811576 17455 raft_consensus.cc:2804] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:12.812917 17455 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:12.813180 17474 consensus_queue.cc:1048] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:12.814376 17428 raft_consensus.cc:493] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.814950 17428 raft_consensus.cc:515] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.811990 17161 leader_election.cc:304] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:12.816865 17466 raft_consensus.cc:515] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:12.819144 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:12.819852 17121 raft_consensus.cc:2468] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:12.825534 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "795b495c58894c1eb861e953546a6e6d" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:12.825831 17428 leader_election.cc:290] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.826264 17193 raft_consensus.cc:3060] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.827240 17463 consensus_queue.cc:1048] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:12.827741 17307 leader_election.cc:304] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:12.832021 17195 raft_consensus.cc:2468] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:12.832214 17428 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.832746 17428 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.835427 17344 raft_consensus.cc:3060] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.813334 17455 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.838688 17425 leader_election.cc:290] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:12.841827 17194 raft_consensus.cc:2468] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:12.846714 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:12.852293 17389 ts_tablet_manager.cc:1434] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.140s	user 0.020s	sys 0.023s
I20260709 12:30:12.847488 17268 raft_consensus.cc:3060] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.855253 17196 raft_consensus.cc:2393] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:12.858080 17234 leader_election.cc:304] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:12.859921 17474 raft_consensus.cc:3060] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.864238 17390 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:12.864804 17390 ts_tablet_manager.cc:1403] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.173s	user 0.011s	sys 0.012s
I20260709 12:30:12.883319 17396 raft_consensus.cc:515] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.895561 17396 leader_election.cc:290] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.897531 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:12.898413 17463 raft_consensus.cc:493] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:12.880280 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.899121 17463 raft_consensus.cc:515] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:12.899874 17389 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:12.900080 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "795b495c58894c1eb861e953546a6e6d" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.901417 17463 leader_election.cc:290] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
W20260709 12:30:12.902168 17235 leader_election.cc:343] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:12.903451 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.885871 17501 consensus_queue.cc:1048] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:30:12.904017 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5746c5442b934f38910321caaa1db9a8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.904178 17339 raft_consensus.cc:3060] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.906119 17390 raft_consensus.cc:359] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.911805 17390 raft_consensus.cc:385] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.912202 17390 raft_consensus.cc:740] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.903959 17085 leader_election.cc:304] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
W20260709 12:30:12.915576 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:12.886978 17466 leader_election.cc:290] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.917326 17393 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
W20260709 12:30:12.879594 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 51951.4us
I20260709 12:30:12.897951 17455 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.918855 17268 raft_consensus.cc:2468] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:12.929590 17510 raft_consensus.cc:2804] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.919632 17390 consensus_queue.cc:260] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.925840 17463 raft_consensus.cc:2804] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:12.925619 17393 ts_tablet_manager.cc:1403] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.189s	user 0.019s	sys 0.018s
I20260709 12:30:12.921262 17343 raft_consensus.cc:3060] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.937770 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:12.942840 17085 leader_election.cc:304] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.943140 17393 raft_consensus.cc:359] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.943962 17393 raft_consensus.cc:385] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:12.930733 17474 raft_consensus.cc:2749] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:12.944787 17393 raft_consensus.cc:740] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:12.931354 17428 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:12.930310 17510 raft_consensus.cc:697] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:12.956511 17393 consensus_queue.cc:260] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:12.964583 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5746c5442b934f38910321caaa1db9a8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.966970 17458 leader_election.cc:290] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.968855 17510 consensus_queue.cc:237] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:12.969749 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:12.952903 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:12.970399 17342 raft_consensus.cc:3060] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.979168 17463 raft_consensus.cc:697] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:12.986644 17339 raft_consensus.cc:2468] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:12.988345 17161 leader_election.cc:304] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:12.992161 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:12.993044 17268 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:12.996100 17514 raft_consensus.cc:2764] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:30:12.978231 17455 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:12.999748 17393 ts_tablet_manager.cc:1434] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.000s
I20260709 12:30:13.000531 17390 ts_tablet_manager.cc:1434] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.135s	user 0.010s	sys 0.012s
I20260709 12:30:13.003592 17193 raft_consensus.cc:2468] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:12.993134 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0148e46ec1b2484e902b01edfda2976c" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:13.007400 17522 raft_consensus.cc:2804] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.007987 17522 raft_consensus.cc:697] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:13.008688 17518 consensus_queue.cc:1048] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.008963 17522 consensus_queue.cc:237] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.006628 17234 leader_election.cc:304] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:12.997787 17342 raft_consensus.cc:2468] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:13.013589 17466 raft_consensus.cc:2804] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.014204 17466 raft_consensus.cc:697] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:13.013556 17390 tablet_bootstrap.cc:492] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:13.003021 17194 raft_consensus.cc:2393] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:13.015277 17161 leader_election.cc:304] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:13.015102 17466 consensus_queue.cc:237] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:13.030817 17393 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:13.037859 17512 raft_consensus.cc:2804] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.038761 17512 raft_consensus.cc:697] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:13.039640 17512 consensus_queue.cc:237] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:13.040657 17267 raft_consensus.cc:2468] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:13.020439 17458 raft_consensus.cc:2804] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.012332 17343 raft_consensus.cc:2468] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:13.025599 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:13.020361 17501 consensus_queue.cc:1048] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:30:13.045722 17463 consensus_queue.cc:237] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.059664 17393 tablet_bootstrap.cc:654] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.049819 17458 raft_consensus.cc:697] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:13.064038 17458 consensus_queue.cc:237] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.061909 17339 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.049659 17471 raft_consensus.cc:515] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.109524 17518 raft_consensus.cc:493] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:13.134537 17519 log.cc:927] Time spent T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:30:13.142668 17266 raft_consensus.cc:3060] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.157634 17391 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:13.132149 17518 raft_consensus.cc:515] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:13.163787 17389 tablet_bootstrap.cc:654] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.163712 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f10e5fb23f4dc08b18d7e805d0a0f4" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:13.170002 17474 consensus_queue.cc:1048] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.175357 17505 consensus_queue.cc:1048] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.187252 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:13.189917 17471 leader_election.cc:290] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:13.190500 17391 ts_tablet_manager.cc:1403] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.449s	user 0.016s	sys 0.017s
I20260709 12:30:13.190258 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:13.227205 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f10e5fb23f4dc08b18d7e805d0a0f4" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:13.229295 17085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 286548us
W20260709 12:30:13.229650 17086 leader_election.cc:343] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.227515 17518 leader_election.cc:290] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:13.230578 17161 leader_election.cc:343] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.230674 17391 raft_consensus.cc:359] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.231598 17391 raft_consensus.cc:385] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.231941 17391 raft_consensus.cc:740] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.232837 17391 consensus_queue.cc:260] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:13.235131 17159 leader_election.cc:343] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.235639 17159 leader_election.cc:304] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:13.236289 17390 tablet_bootstrap.cc:654] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.236963 17396 raft_consensus.cc:2749] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.222527 17265 raft_consensus.cc:3060] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.238125 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:13.238874 17121 raft_consensus.cc:3060] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.240187 17391 ts_tablet_manager.cc:1434] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:30:13.241366 17391 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:13.238896 17523 log.cc:927] Time spent T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.156s	user 0.002s	sys 0.000s
I20260709 12:30:13.250555 17268 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:13.285086 17393 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:13.285706 17393 ts_tablet_manager.cc:1403] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.261s	user 0.011s	sys 0.005s
I20260709 12:30:13.293124 17391 tablet_bootstrap.cc:654] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:13.293534 17531 log.cc:927] Time spent T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:30:13.297569 17085 leader_election.cc:304] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:13.300516 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:13.320572 17233 consensus_peers.cc:597] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.318079 17084 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:13.332290 17307 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:13.333559 17514 raft_consensus.cc:2749] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.309813 17425 raft_consensus.cc:2749] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.324815 17532 raft_consensus.cc:2804] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.343878 17120 raft_consensus.cc:3060] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.347915 17393 raft_consensus.cc:359] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.348718 17393 raft_consensus.cc:385] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.349011 17393 raft_consensus.cc:740] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.349707 17532 raft_consensus.cc:697] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:13.350651 17532 consensus_queue.cc:237] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.352603 17016 catalog_manager.cc:5697] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:13.354530 17339 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:13.356403 17016 catalog_manager.cc:5697] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:13.357818 17265 raft_consensus.cc:2468] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:13.356196 17393 consensus_queue.cc:260] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.363339 17084 leader_election.cc:304] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:13.371665 17390 tablet_bootstrap.cc:492] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:13.372205 17390 ts_tablet_manager.cc:1403] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.359s	user 0.011s	sys 0.012s
I20260709 12:30:13.364240 17121 raft_consensus.cc:2468] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:13.343039 17512 consensus_queue.cc:1048] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260709 12:30:13.377637 17530 raft_consensus.cc:2804] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.378329 17530 raft_consensus.cc:697] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:13.402237 17389 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:13.402958 17389 ts_tablet_manager.cc:1403] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.520s	user 0.010s	sys 0.019s
I20260709 12:30:13.379331 17530 consensus_queue.cc:237] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.414687 17393 ts_tablet_manager.cc:1434] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.129s	user 0.006s	sys 0.005s
I20260709 12:30:13.419456 17391 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:13.420097 17391 ts_tablet_manager.cc:1403] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.179s	user 0.009s	sys 0.011s
I20260709 12:30:13.414179 17501 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.424129 17501 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.429795 17501 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:13.431250 17393 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:13.431355 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:13.432256 17340 raft_consensus.cc:2468] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:13.433662 17235 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:13.434774 17501 raft_consensus.cc:2804] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.435201 17501 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.435612 17501 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.441387 17393 tablet_bootstrap.cc:654] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.442811 17512 consensus_queue.cc:1048] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.445580 17391 raft_consensus.cc:359] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.446403 17391 raft_consensus.cc:385] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.446723 17391 raft_consensus.cc:740] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.448603 17536 raft_consensus.cc:493] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.449482 17536 raft_consensus.cc:515] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.451169 17390 raft_consensus.cc:359] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:13.452030 17536 leader_election.cc:290] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:13.452675 17389 raft_consensus.cc:359] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.452970 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e7b8f0ed341e0b42e5d3872c70010" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:13.457527 17547 log.cc:927] Time spent T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:30:13.459614 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e7b8f0ed341e0b42e5d3872c70010" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:13.461215 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:13.466897 17391 consensus_queue.cc:260] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.465471 17389 raft_consensus.cc:385] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.468714 17389 raft_consensus.cc:740] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.452190 17390 raft_consensus.cc:385] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.469812 17390 raft_consensus.cc:740] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.471325 17390 consensus_queue.cc:260] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:13.466889 17086 leader_election.cc:343] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.477588 17391 ts_tablet_manager.cc:1434] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.002s
I20260709 12:30:13.475560 17389 consensus_queue.cc:260] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:13.483922 17233 leader_election.cc:343] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.486022 17391 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:13.490664 17390 ts_tablet_manager.cc:1434] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.118s	user 0.008s	sys 0.002s
I20260709 12:30:13.493963 17389 ts_tablet_manager.cc:1434] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.091s	user 0.011s	sys 0.002s
I20260709 12:30:13.498548 17390 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
W20260709 12:30:13.489077 17085 leader_election.cc:343] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.503715 17085 leader_election.cc:304] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:13.505658 17536 raft_consensus.cc:2749] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.507890 17393 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:13.512549 17393 ts_tablet_manager.cc:1403] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.082s	user 0.018s	sys 0.013s
I20260709 12:30:13.516958 17510 consensus_queue.cc:1048] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.520481 17019 catalog_manager.cc:5697] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.524976 17015 catalog_manager.cc:5697] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:13.526028 17159 consensus_peers.cc:597] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.526654 17393 raft_consensus.cc:359] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.530100 17393 raft_consensus.cc:385] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.530371 17393 raft_consensus.cc:740] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.531119 17389 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:13.532719 17393 consensus_queue.cc:260] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.535598 17510 consensus_queue.cc:1048] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.540004 17393 ts_tablet_manager.cc:1434] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:30:13.540205 17391 tablet_bootstrap.cc:654] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.541366 17393 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:13.544389 17551 consensus_queue.cc:1048] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.544880 17501 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.572630 17551 raft_consensus.cc:493] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.573181 17551 raft_consensus.cc:515] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.580199 17458 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.581941 17458 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.578825 17017 catalog_manager.cc:5697] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.585536 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:13.587777 17234 leader_election.cc:343] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.594347 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:13.600576 17233 leader_election.cc:343] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.602177 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:13.603621 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:13.578516 17551 leader_election.cc:290] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:13.601022 17536 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.605227 17536 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.605526 17512 consensus_queue.cc:1048] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:13.605695 17017 catalog_manager.cc:5697] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:13.602366 17233 leader_election.cc:304] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:13.607633 17536 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
W20260709 12:30:13.607439 17159 leader_election.cc:343] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.610462 17505 raft_consensus.cc:2749] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.610888 17121 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:13.611781 17425 consensus_queue.cc:1048] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.615849 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:13.616562 17194 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
W20260709 12:30:13.617699 17235 consensus_peers.cc:597] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 -> Peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Couldn't send request to peer 7c0f44cba62f491392d41ca89376139a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.617725 17085 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:13.618932 17532 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.619335 17532 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.619733 17532 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.622263 17389 tablet_bootstrap.cc:654] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.600077 17458 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:13.626230 17514 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.627584 17514 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.631503 17501 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:13.633273 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:13.636693 17340 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.638401 17425 consensus_queue.cc:1048] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.639331 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:13.636093 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:13.639842 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:13.641750 17233 leader_election.cc:343] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.642031 17425 consensus_queue.cc:1048] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.643513 17159 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310
I20260709 12:30:13.634325 17514 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:13.646209 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:13.653784 17308 leader_election.cc:343] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.659545 17512 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.672573 17390 tablet_bootstrap.cc:654] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.674266 17391 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
W20260709 12:30:13.670624 17307 leader_election.cc:343] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.677073 17514 raft_consensus.cc:493] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.677724 17514 raft_consensus.cc:515] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:13.673445 17084 leader_election.cc:343] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.679924 17514 leader_election.cc:290] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:13.679952 17307 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:13.681167 17514 raft_consensus.cc:2749] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.682580 17458 consensus_queue.cc:1048] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:13.683348 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f10e5fb23f4dc08b18d7e805d0a0f4" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:13.684160 17195 raft_consensus.cc:2468] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:13.669917 17512 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.685672 17512 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.688090 17536 consensus_queue.cc:1048] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:13.674887 17391 ts_tablet_manager.cc:1403] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.196s	user 0.015s	sys 0.002s
I20260709 12:30:13.693123 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f10e5fb23f4dc08b18d7e805d0a0f4" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:13.698767 17393 tablet_bootstrap.cc:654] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:13.709286 17307 leader_election.cc:343] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.723826 17308 leader_election.cc:304] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310
I20260709 12:30:13.725090 17514 raft_consensus.cc:2804] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.725597 17514 raft_consensus.cc:493] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.726046 17514 raft_consensus.cc:3060] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.706461 17391 raft_consensus.cc:359] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:13.715169 17562 log.cc:927] Time spent T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
W20260709 12:30:13.730314 17564 log.cc:927] Time spent T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:30:13.734289 17391 raft_consensus.cc:385] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.742976 17391 raft_consensus.cc:740] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.752223 17391 consensus_queue.cc:260] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.719223 17532 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.761509 17340 raft_consensus.cc:2468] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:13.778177 17235 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:13.780959 17501 raft_consensus.cc:2804] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.781817 17501 raft_consensus.cc:697] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:13.767908 17514 raft_consensus.cc:515] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:13.783368 17501 consensus_queue.cc:237] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.786530 17017 catalog_manager.cc:5697] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } }
I20260709 12:30:13.787686 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f10e5fb23f4dc08b18d7e805d0a0f4" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:13.791468 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60f10e5fb23f4dc08b18d7e805d0a0f4" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:13.792511 17195 raft_consensus.cc:3060] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.794582 17514 leader_election.cc:290] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:13.795605 17391 ts_tablet_manager.cc:1434] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.093s	user 0.007s	sys 0.003s
W20260709 12:30:13.795693 17307 leader_election.cc:343] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.796764 17391 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:13.781303 17512 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.807138 17389 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:13.807689 17389 ts_tablet_manager.cc:1403] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.303s	user 0.017s	sys 0.013s
W20260709 12:30:13.810400 17570 log.cc:927] Time spent T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.003s
I20260709 12:30:13.810290 17389 raft_consensus.cc:359] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.811069 17389 raft_consensus.cc:385] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.811367 17389 raft_consensus.cc:740] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.812125 17389 consensus_queue.cc:260] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.814267 17389 ts_tablet_manager.cc:1434] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:30:13.821338 17532 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:13.822268 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:13.823717 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
W20260709 12:30:13.825407 17084 leader_election.cc:343] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.830607 17391 tablet_bootstrap.cc:654] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.833086 17389 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:13.835062 17390 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:13.835529 17390 ts_tablet_manager.cc:1403] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.337s	user 0.008s	sys 0.009s
I20260709 12:30:13.838479 17390 raft_consensus.cc:359] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.839449 17390 raft_consensus.cc:385] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.839808 17390 raft_consensus.cc:740] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.840662 17390 consensus_queue.cc:260] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.841374 17193 raft_consensus.cc:2393] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:13.849076 17389 tablet_bootstrap.cc:654] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.846750 17512 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:13.856958 17393 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:13.857609 17393 ts_tablet_manager.cc:1403] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.317s	user 0.012s	sys 0.007s
I20260709 12:30:13.858069 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:13.862048 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:13.862951 17120 raft_consensus.cc:2393] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:13.866009 17390 ts_tablet_manager.cc:1434] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:30:13.867429 17390 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:13.868963 17019 catalog_manager.cc:5697] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:13.871727 17159 leader_election.cc:343] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.873214 17393 raft_consensus.cc:359] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.874087 17393 raft_consensus.cc:385] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.832249 17195 raft_consensus.cc:2468] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:13.878142 17393 raft_consensus.cc:740] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.881392 17085 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:13.884397 17159 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:13.884660 17530 raft_consensus.cc:2749] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.885721 17512 raft_consensus.cc:2749] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.888119 17308 leader_election.cc:304] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310
I20260709 12:30:13.890566 17393 consensus_queue.cc:260] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:13.893895 17389 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:13.894354 17389 ts_tablet_manager.cc:1403] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.001s
I20260709 12:30:13.893548 17514 raft_consensus.cc:2804] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:13.897920 17393 ts_tablet_manager.cc:1434] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 12:30:13.898317 17440 raft_consensus.cc:493] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.901008 17440 raft_consensus.cc:515] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:13.901136 17514 raft_consensus.cc:697] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:13.904412 17391 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:13.905063 17391 ts_tablet_manager.cc:1403] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.109s	user 0.014s	sys 0.006s
I20260709 12:30:13.904937 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:13.903918 17514 consensus_queue.cc:237] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:13.908355 17235 leader_election.cc:343] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.909760 17390 tablet_bootstrap.cc:654] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.915185 17391 raft_consensus.cc:359] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.916004 17391 raft_consensus.cc:385] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.916419 17440 leader_election.cc:290] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:13.917037 17389 raft_consensus.cc:359] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.917824 17389 raft_consensus.cc:385] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:13.918100 17389 raft_consensus.cc:740] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.916296 17391 raft_consensus.cc:740] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:13.919643 17391 consensus_queue.cc:260] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.921998 17391 ts_tablet_manager.cc:1434] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:30:13.923007 17389 consensus_queue.cc:260] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.925907 17389 ts_tablet_manager.cc:1434] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.002s
I20260709 12:30:13.927189 17389 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:13.938351 17393 tablet_bootstrap.cc:492] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:13.942781 17576 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.943602 17576 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.947736 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:13.948597 17342 raft_consensus.cc:2468] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:13.950045 17086 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:13.950965 17578 raft_consensus.cc:2804] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:13.951432 17578 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:13.951778 17578 raft_consensus.cc:3060] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.955518 17576 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:13.960059 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:13.961627 17233 leader_election.cc:343] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:13.962188 17233 leader_election.cc:304] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:13.960009 17578 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.964846 17501 raft_consensus.cc:2749] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:13.965021 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:13.965866 17193 raft_consensus.cc:2468] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:13.966005 17578 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:13.972597 17576 consensus_queue.cc:1048] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:13.973078 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:13.975855 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:13.987546 17193 raft_consensus.cc:3060] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.982824 17393 tablet_bootstrap.cc:654] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:13.983980 17342 raft_consensus.cc:3060] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:13.991458 17458 raft_consensus.cc:493] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.978447 17391 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:13.992125 17458 raft_consensus.cc:515] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:13.995591 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e7b8f0ed341e0b42e5d3872c70010" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:13.996345 17339 raft_consensus.cc:2468] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:13.997943 17576 raft_consensus.cc:493] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:13.999233 17161 leader_election.cc:304] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:14.002018 17510 raft_consensus.cc:2804] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.002591 17510 raft_consensus.cc:493] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.003021 17510 raft_consensus.cc:3060] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.004351 17458 leader_election.cc:290] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:13.998809 17576 raft_consensus.cc:515] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.014779 17390 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:14.015625 17390 ts_tablet_manager.cc:1403] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.148s	user 0.014s	sys 0.035s
W20260709 12:30:14.022332 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:14.022219 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e7b8f0ed341e0b42e5d3872c70010" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.023201 17578 consensus_queue.cc:1048] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.027642 17391 tablet_bootstrap.cc:654] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.030838 17121 raft_consensus.cc:2468] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:14.032853 17576 leader_election.cc:290] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:14.033545 17390 raft_consensus.cc:359] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.034698 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.036034 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:14.042953 17389 tablet_bootstrap.cc:654] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.044693 17085 leader_election.cc:304] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:14.044780 17390 raft_consensus.cc:385] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.010376 17342 raft_consensus.cc:2468] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:14.046764 17390 raft_consensus.cc:740] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.047998 17390 consensus_queue.cc:260] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.048276 17086 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:14.046150 17015 catalog_manager.cc:5697] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:14.046962 17578 raft_consensus.cc:2749] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.049471 17576 raft_consensus.cc:2804] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.050093 17576 raft_consensus.cc:697] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:14.051260 17576 consensus_queue.cc:237] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:14.052668 17086 consensus_peers.cc:597] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 -> Peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Couldn't send request to peer 7c0f44cba62f491392d41ca89376139a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:14.055331 17390 ts_tablet_manager.cc:1434] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.001s
I20260709 12:30:14.056643 17390 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:14.048273 17017 catalog_manager.cc:5697] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.071991 17193 raft_consensus.cc:2468] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:14.072306 17393 tablet_bootstrap.cc:492] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:14.073948 17393 ts_tablet_manager.cc:1403] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.166s	user 0.027s	sys 0.022s
I20260709 12:30:14.073829 17510 raft_consensus.cc:515] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.076294 17510 leader_election.cc:290] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.077148 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e7b8f0ed341e0b42e5d3872c70010" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:14.077790 17342 raft_consensus.cc:3060] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.080287 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4e7b8f0ed341e0b42e5d3872c70010" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:14.080976 17121 raft_consensus.cc:3060] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.081326 17393 raft_consensus.cc:359] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.082013 17393 raft_consensus.cc:385] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.082267 17393 raft_consensus.cc:740] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.082966 17393 consensus_queue.cc:260] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.085072 17393 ts_tablet_manager.cc:1434] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.004s
I20260709 12:30:14.086156 17393 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:14.086951 17390 tablet_bootstrap.cc:654] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.095573 17121 raft_consensus.cc:2468] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:14.096482 17514 raft_consensus.cc:493] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.096990 17514 raft_consensus.cc:515] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.099282 17342 raft_consensus.cc:2468] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:14.106618 17389 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:14.117916 17389 ts_tablet_manager.cc:1403] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.191s	user 0.020s	sys 0.003s
I20260709 12:30:14.119431 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.120383 17266 raft_consensus.cc:2393] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:14.107146 17159 leader_election.cc:304] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:14.108445 17514 leader_election.cc:290] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.109153 17578 consensus_queue.cc:1048] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.122872 17510 raft_consensus.cc:2804] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.123399 17510 raft_consensus.cc:697] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:14.114724 17393 tablet_bootstrap.cc:654] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.124253 17510 consensus_queue.cc:237] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.127348 17389 raft_consensus.cc:359] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.128222 17389 raft_consensus.cc:385] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.128544 17389 raft_consensus.cc:740] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.129621 17389 consensus_queue.cc:260] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.133234 17389 ts_tablet_manager.cc:1434] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:30:14.134570 17389 tablet_bootstrap.cc:492] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:14.139066 17501 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.139760 17501 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.141651 17389 tablet_bootstrap.cc:654] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.142735 17501 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:14.144538 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:14.145958 17193 raft_consensus.cc:2393] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:14.150663 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.151563 17117 raft_consensus.cc:2393] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:14.156098 17233 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:14.157128 17440 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.157756 17017 catalog_manager.cc:5697] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:14.161700 17389 tablet_bootstrap.cc:492] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:14.162323 17389 ts_tablet_manager.cc:1403] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.007s
I20260709 12:30:14.165043 17117 raft_consensus.cc:3060] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.168706 17440 raft_consensus.cc:2749] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.172612 17393 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:14.173204 17393 ts_tablet_manager.cc:1403] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.087s	user 0.007s	sys 0.011s
I20260709 12:30:14.175290 17391 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:14.199877 17391 ts_tablet_manager.cc:1403] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.222s	user 0.014s	sys 0.018s
I20260709 12:30:14.177865 17393 raft_consensus.cc:359] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.202454 17393 raft_consensus.cc:385] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.202904 17393 raft_consensus.cc:740] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.203781 17393 consensus_queue.cc:260] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.206179 17393 ts_tablet_manager.cc:1434] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.003s
I20260709 12:30:14.207027 17391 raft_consensus.cc:359] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.191023 17389 raft_consensus.cc:359] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.213491 17389 raft_consensus.cc:385] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.213518 17391 raft_consensus.cc:385] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.213912 17389 raft_consensus.cc:740] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.214059 17391 raft_consensus.cc:740] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.172343 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.207295 17393 tablet_bootstrap.cc:492] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:14.214846 17389 consensus_queue.cc:260] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.216200 17307 leader_election.cc:304] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:14.217549 17514 raft_consensus.cc:3060] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.217968 17389 ts_tablet_manager.cc:1434] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 12:30:14.219331 17389 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:14.211895 17390 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:14.212565 17501 consensus_queue.cc:1048] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.209187 17015 catalog_manager.cc:5697] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.215036 17391 consensus_queue.cc:260] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.222476 17393 tablet_bootstrap.cc:654] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.237022 17391 ts_tablet_manager.cc:1434] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:30:14.238410 17391 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:14.240526 17233 consensus_peers.cc:597] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:14.228761 17390 ts_tablet_manager.cc:1403] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.172s	user 0.023s	sys 0.021s
I20260709 12:30:14.266885 17390 raft_consensus.cc:359] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.267994 17390 raft_consensus.cc:385] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.268308 17390 raft_consensus.cc:740] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.229553 17514 raft_consensus.cc:2749] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.226903 17389 tablet_bootstrap.cc:654] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.270395 17390 consensus_queue.cc:260] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.272904 17390 ts_tablet_manager.cc:1434] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:14.274230 17390 tablet_bootstrap.cc:492] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:14.282169 17393 tablet_bootstrap.cc:492] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:14.282828 17393 ts_tablet_manager.cc:1403] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.010s
I20260709 12:30:14.285987 17393 raft_consensus.cc:359] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.286679 17393 raft_consensus.cc:385] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.286942 17393 raft_consensus.cc:740] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.289964 17390 tablet_bootstrap.cc:654] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.292761 17393 consensus_queue.cc:260] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.295362 17393 ts_tablet_manager.cc:1434] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:30:14.296453 17393 tablet_bootstrap.cc:492] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:14.304665 17390 tablet_bootstrap.cc:492] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:14.308193 17390 ts_tablet_manager.cc:1403] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.034s	user 0.010s	sys 0.001s
I20260709 12:30:14.312302 17390 raft_consensus.cc:359] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.313046 17390 raft_consensus.cc:385] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.313318 17390 raft_consensus.cc:740] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.305691 17393 tablet_bootstrap.cc:654] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.317422 17390 consensus_queue.cc:260] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.301177 17391 tablet_bootstrap.cc:654] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.330370 17390 ts_tablet_manager.cc:1434] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:30:14.337870 17390 tablet_bootstrap.cc:492] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:14.347395 17510 raft_consensus.cc:493] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.348100 17510 raft_consensus.cc:515] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.354172 17391 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:14.355007 17391 ts_tablet_manager.cc:1403] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.117s	user 0.015s	sys 0.004s
I20260709 12:30:14.363746 17389 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:14.364483 17389 ts_tablet_manager.cc:1403] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.145s	user 0.033s	sys 0.023s
I20260709 12:30:14.365914 17391 raft_consensus.cc:359] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.366762 17391 raft_consensus.cc:385] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.367019 17391 raft_consensus.cc:740] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.367719 17391 consensus_queue.cc:260] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.368018 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d81d3402a7848f886f20c683e7874f3" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.368825 17514 consensus_queue.cc:1048] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:14.369483 17159 leader_election.cc:343] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.370115 17391 ts_tablet_manager.cc:1434] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:30:14.370996 17391 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:14.372732 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d81d3402a7848f886f20c683e7874f3" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:14.376125 17161 leader_election.cc:343] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.370993 17510 leader_election.cc:290] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:14.376950 17161 leader_election.cc:304] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:14.384476 17391 tablet_bootstrap.cc:654] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.386142 17393 tablet_bootstrap.cc:492] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:14.386830 17393 ts_tablet_manager.cc:1403] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.091s	user 0.011s	sys 0.003s
I20260709 12:30:14.387176 17389 raft_consensus.cc:359] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.387948 17389 raft_consensus.cc:385] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.388231 17389 raft_consensus.cc:740] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.391580 17389 consensus_queue.cc:260] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.394172 17389 ts_tablet_manager.cc:1434] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:30:14.382714 17510 raft_consensus.cc:2749] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.396986 17389 tablet_bootstrap.cc:492] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:14.408212 17393 raft_consensus.cc:359] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.409621 17393 raft_consensus.cc:385] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.409953 17393 raft_consensus.cc:740] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.416532 17342 raft_consensus.cc:3060] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.418982 17393 consensus_queue.cc:260] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.421216 17393 ts_tablet_manager.cc:1434] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.000s
I20260709 12:30:14.422345 17393 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:14.433229 17389 tablet_bootstrap.cc:654] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.436985 17266 raft_consensus.cc:3060] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.416462 17390 tablet_bootstrap.cc:654] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.449393 17576 consensus_queue.cc:1048] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.469125 17389 tablet_bootstrap.cc:492] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:14.469914 17389 ts_tablet_manager.cc:1403] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.075s	user 0.011s	sys 0.007s
I20260709 12:30:14.475749 17604 consensus_queue.cc:1048] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:14.477203 17510 raft_consensus.cc:493] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.477730 17510 raft_consensus.cc:515] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.479771 17510 leader_election.cc:290] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:14.474205 17389 raft_consensus.cc:359] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.482990 17389 raft_consensus.cc:385] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.483436 17389 raft_consensus.cc:740] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.482226 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.484719 17269 raft_consensus.cc:2468] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:14.484486 17389 consensus_queue.cc:260] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.486003 17159 leader_election.cc:304] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:14.487470 17458 raft_consensus.cc:2804] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.487948 17458 raft_consensus.cc:493] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.488022 17389 ts_tablet_manager.cc:1434] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.004s
I20260709 12:30:14.488404 17458 raft_consensus.cc:3060] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.489626 17389 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:14.490653 17574 consensus_queue.cc:1048] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.474239 17393 tablet_bootstrap.cc:654] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.512920 17466 raft_consensus.cc:493] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.511523 17576 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.524011 17466 raft_consensus.cc:515] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.532433 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.534716 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35817ffe9ce34abd82d949dc55a8ae78" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:14.535825 17466 leader_election.cc:290] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:14.548323 17235 leader_election.cc:343] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.550721 17390 tablet_bootstrap.cc:492] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:14.551403 17390 ts_tablet_manager.cc:1403] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.214s	user 0.014s	sys 0.019s
I20260709 12:30:14.554819 17390 raft_consensus.cc:359] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.555608 17390 raft_consensus.cc:385] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.555939 17390 raft_consensus.cc:740] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.556699 17390 consensus_queue.cc:260] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.568295 17389 tablet_bootstrap.cc:654] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.527240 17576 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.570214 17391 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
W20260709 12:30:14.568290 17159 leader_election.cc:343] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.561460 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35817ffe9ce34abd82d949dc55a8ae78" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:14.570173 17390 ts_tablet_manager.cc:1434] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.001s
I20260709 12:30:14.577409 17391 ts_tablet_manager.cc:1403] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.207s	user 0.035s	sys 0.052s
I20260709 12:30:14.572376 17193 raft_consensus.cc:2468] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:14.598073 17576 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:14.598064 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.615121 17234 leader_election.cc:304] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:14.617961 17507 raft_consensus.cc:2804] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.618361 17507 raft_consensus.cc:493] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.615361 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:14.618757 17458 raft_consensus.cc:515] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.618834 17507 raft_consensus.cc:3060] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.621260 17458 leader_election.cc:290] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:14.623337 17086 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:14.623356 17391 raft_consensus.cc:359] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.625186 17536 raft_consensus.cc:2749] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.630282 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:14.636384 17390 tablet_bootstrap.cc:492] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:14.641742 17510 consensus_queue.cc:1048] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:14.643848 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:14.644565 17266 raft_consensus.cc:3060] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:14.644702 17159 leader_election.cc:343] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.646533 17391 raft_consensus.cc:385] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.647454 17391 raft_consensus.cc:740] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.654017 17393 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:14.654631 17393 ts_tablet_manager.cc:1403] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.233s	user 0.011s	sys 0.030s
I20260709 12:30:14.657734 17393 raft_consensus.cc:359] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.658408 17393 raft_consensus.cc:385] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.659469 17393 raft_consensus.cc:740] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.663192 17389 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:14.649439 17391 consensus_queue.cc:260] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.663781 17389 ts_tablet_manager.cc:1403] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.174s	user 0.022s	sys 0.002s
I20260709 12:30:14.662824 17393 consensus_queue.cc:260] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.670222 17391 ts_tablet_manager.cc:1434] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.092s	user 0.009s	sys 0.001s
I20260709 12:30:14.674084 17393 ts_tablet_manager.cc:1434] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.001s
I20260709 12:30:14.685653 17389 raft_consensus.cc:359] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.686808 17389 raft_consensus.cc:385] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.688037 17389 raft_consensus.cc:740] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.687239 17618 consensus_queue.cc:1048] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:30:14.696264 17505 raft_consensus.cc:493] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.696836 17505 raft_consensus.cc:515] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.700472 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd19f1dc408f4801b7e7d672696228cc" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.701125 17121 raft_consensus.cc:2468] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:14.701972 17390 tablet_bootstrap.cc:654] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.702426 17233 leader_election.cc:304] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:14.703316 17466 raft_consensus.cc:2804] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.703681 17466 raft_consensus.cc:493] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.703998 17466 raft_consensus.cc:3060] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.708781 17391 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:14.710768 17389 consensus_queue.cc:260] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.711191 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd19f1dc408f4801b7e7d672696228cc" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:14.713095 17234 leader_election.cc:343] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.713703 17389 ts_tablet_manager.cc:1434] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.001s
I20260709 12:30:14.713900 17505 leader_election.cc:290] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:14.715184 17389 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:14.720958 17266 raft_consensus.cc:2468] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:14.722910 17159 leader_election.cc:304] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:14.728696 17393 tablet_bootstrap.cc:492] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:14.733731 17613 raft_consensus.cc:2804] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.741029 17391 tablet_bootstrap.cc:654] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.737577 17613 raft_consensus.cc:697] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:14.747538 17440 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.761052 17507 raft_consensus.cc:515] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.762161 17501 raft_consensus.cc:493] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.763187 17613 consensus_queue.cc:237] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.764946 17604 raft_consensus.cc:493] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.780915 17604 raft_consensus.cc:515] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.783497 17604 leader_election.cc:290] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.785271 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.768286 17466 raft_consensus.cc:515] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.792726 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.801808 17604 raft_consensus.cc:493] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.768637 17440 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.811784 17389 tablet_bootstrap.cc:654] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.813130 17266 raft_consensus.cc:2468] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
W20260709 12:30:14.805579 17307 leader_election.cc:343] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.815531 17440 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.815951 17307 leader_election.cc:304] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:14.818346 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:14.819262 17565 raft_consensus.cc:2804] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.819972 17565 raft_consensus.cc:493] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.820396 17565 raft_consensus.cc:3060] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.821357 17466 leader_election.cc:290] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
W20260709 12:30:14.822438 17235 leader_election.cc:343] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.776755 17501 raft_consensus.cc:515] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.808225 17576 raft_consensus.cc:493] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.824523 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd19f1dc408f4801b7e7d672696228cc" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:14.825522 17501 leader_election.cc:290] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:14.826853 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d81d3402a7848f886f20c683e7874f3" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:14.826907 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d81d3402a7848f886f20c683e7874f3" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:14.827761 17194 raft_consensus.cc:2468] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:14.828235 17604 raft_consensus.cc:515] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:14.828742 17235 leader_election.cc:343] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:14.828939 17234 leader_election.cc:343] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.824349 17576 raft_consensus.cc:515] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.832018 17234 leader_election.cc:304] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:14.836702 17501 raft_consensus.cc:2804] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:14.837255 17501 raft_consensus.cc:493] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:14.837707 17501 raft_consensus.cc:3060] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.840531 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:14.841022 17576 leader_election.cc:290] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:14.843758 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:14.846257 17085 leader_election.cc:304] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:14.847908 17536 raft_consensus.cc:2749] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.851357 17510 raft_consensus.cc:493] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.852038 17510 raft_consensus.cc:515] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.852483 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35817ffe9ce34abd82d949dc55a8ae78" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:14.853511 17194 raft_consensus.cc:3060] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.855243 17507 leader_election.cc:290] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:14.854399 17510 leader_election.cc:290] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.862353 17604 leader_election.cc:290] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:14.863062 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35817ffe9ce34abd82d949dc55a8ae78" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:14.863837 17342 raft_consensus.cc:3060] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.864176 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35817ffe9ce34abd82d949dc55a8ae78" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:14.865098 17266 raft_consensus.cc:2393] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate 6c1eb70ad94940b89118a32a370a4310 in this term.
I20260709 12:30:14.869427 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35817ffe9ce34abd82d949dc55a8ae78" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:14.866926 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47941e6ecfe34dfeb4aeb7f68e020eb5" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:14.874444 17161 leader_election.cc:343] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.874964 17308 leader_election.cc:304] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:14.873772 17393 tablet_bootstrap.cc:654] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.869496 17565 raft_consensus.cc:515] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:14.906634 17117 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.906876 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.908090 17390 tablet_bootstrap.cc:492] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:14.908962 17390 ts_tablet_manager.cc:1403] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.273s	user 0.024s	sys 0.018s
I20260709 12:30:14.911513 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd19f1dc408f4801b7e7d672696228cc" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:14.917090 17119 raft_consensus.cc:3060] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:14.917143 17233 leader_election.cc:343] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.910763 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47941e6ecfe34dfeb4aeb7f68e020eb5" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.917884 17233 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:14.910315 17613 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:14.921479 17501 raft_consensus.cc:515] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.922655 17623 raft_consensus.cc:2749] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:14.923877 17159 leader_election.cc:343] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:14.924441 17159 leader_election.cc:304] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:14.925387 17390 raft_consensus.cc:359] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.925746 17510 raft_consensus.cc:2749] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:14.926250 17390 raft_consensus.cc:385] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.926645 17390 raft_consensus.cc:740] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.928597 17565 leader_election.cc:290] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.928851 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:14.929656 17266 raft_consensus.cc:3060] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.931527 17390 consensus_queue.cc:260] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:14.940451 17119 raft_consensus.cc:2468] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:14.951136 17391 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:14.952071 17391 ts_tablet_manager.cc:1403] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.244s	user 0.021s	sys 0.005s
I20260709 12:30:14.952867 17389 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:14.953817 17389 ts_tablet_manager.cc:1403] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.239s	user 0.022s	sys 0.012s
I20260709 12:30:14.935950 17613 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:14.959056 17613 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:14.965044 17391 raft_consensus.cc:359] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.968282 17391 raft_consensus.cc:385] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.968685 17391 raft_consensus.cc:740] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.974180 17389 raft_consensus.cc:359] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.975319 17389 raft_consensus.cc:385] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.975703 17389 raft_consensus.cc:740] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.976605 17389 consensus_queue.cc:260] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:14.979372 17389 ts_tablet_manager.cc:1434] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:30:14.980542 17389 tablet_bootstrap.cc:492] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:14.980324 17017 catalog_manager.cc:5697] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:14.987444 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d81d3402a7848f886f20c683e7874f3" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:14.988454 17195 raft_consensus.cc:3060] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.990299 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2d81d3402a7848f886f20c683e7874f3" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:14.989159 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:14.991017 17341 raft_consensus.cc:3060] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.987509 17501 leader_election.cc:290] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:14.994251 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
W20260709 12:30:14.997354 17161 leader_election.cc:343] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.000442 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:14.971822 17194 raft_consensus.cc:2468] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:15.003298 17234 leader_election.cc:304] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:15.006317 17389 tablet_bootstrap.cc:654] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.007681 17233 leader_election.cc:304] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:15.009171 17623 raft_consensus.cc:2804] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.009941 17623 raft_consensus.cc:697] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:15.011118 17623 consensus_queue.cc:237] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.012224 17501 raft_consensus.cc:2804] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.026343 17342 raft_consensus.cc:2468] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:15.026394 17501 raft_consensus.cc:697] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:15.027030 17604 raft_consensus.cc:2749] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.027786 17501 consensus_queue.cc:237] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:15.010208 17307 leader_election.cc:343] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.034088 17393 tablet_bootstrap.cc:492] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:15.034668 17393 ts_tablet_manager.cc:1403] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.306s	user 0.031s	sys 0.039s
W20260709 12:30:15.036985 17159 leader_election.cc:343] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.037592 17159 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.037626 17393 raft_consensus.cc:359] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.058516 17393 raft_consensus.cc:385] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.058954 17393 raft_consensus.cc:740] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.039458 17390 ts_tablet_manager.cc:1434] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.130s	user 0.011s	sys 0.000s
I20260709 12:30:15.059741 17393 consensus_queue.cc:260] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.061043 17390 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:15.062166 17393 ts_tablet_manager.cc:1434] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.027s	user 0.002s	sys 0.003s
I20260709 12:30:15.044040 17266 raft_consensus.cc:2468] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:15.044968 17620 raft_consensus.cc:2749] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.063303 17393 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:15.066740 17341 raft_consensus.cc:2468] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:15.017073 17391 consensus_queue.cc:260] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.071161 17390 tablet_bootstrap.cc:654] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.073163 17391 ts_tablet_manager.cc:1434] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.121s	user 0.009s	sys 0.001s
I20260709 12:30:15.021514 17195 raft_consensus.cc:2468] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:15.076251 17234 leader_election.cc:304] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:15.067356 17307 leader_election.cc:304] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:15.077526 17604 raft_consensus.cc:2804] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.070155 17389 tablet_bootstrap.cc:492] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:15.078058 17604 raft_consensus.cc:697] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:15.078100 17623 raft_consensus.cc:2804] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.046838 17019 catalog_manager.cc:5697] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:15.078466 17389 ts_tablet_manager.cc:1403] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.098s	user 0.010s	sys 0.004s
I20260709 12:30:15.078794 17623 raft_consensus.cc:697] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:15.078975 17604 consensus_queue.cc:237] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.079722 17623 consensus_queue.cc:237] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.082875 17389 raft_consensus.cc:359] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.083593 17389 raft_consensus.cc:385] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.083892 17389 raft_consensus.cc:740] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.084828 17389 consensus_queue.cc:260] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.087311 17389 ts_tablet_manager.cc:1434] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.004s
I20260709 12:30:15.089104 17389 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:15.090313 17391 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.095320 17389 tablet_bootstrap.cc:654] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.097180 17391 tablet_bootstrap.cc:654] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.112928 17393 tablet_bootstrap.cc:654] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.120368 17389 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:15.121007 17389 ts_tablet_manager.cc:1403] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 12:30:15.125344 17393 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:15.125943 17393 ts_tablet_manager.cc:1403] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.001s
I20260709 12:30:15.125715 17389 raft_consensus.cc:359] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.126590 17389 raft_consensus.cc:385] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.126912 17389 raft_consensus.cc:740] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.127632 17389 consensus_queue.cc:260] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.129199 17393 raft_consensus.cc:359] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.130004 17393 raft_consensus.cc:385] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.130154 17389 ts_tablet_manager.cc:1434] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:30:15.130367 17393 raft_consensus.cc:740] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.131510 17389 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:15.131819 17393 consensus_queue.cc:260] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.134325 17393 ts_tablet_manager.cc:1434] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 12:30:15.135326 17393 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:15.138639 17389 tablet_bootstrap.cc:654] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.140825 17390 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:15.141399 17390 ts_tablet_manager.cc:1403] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.081s	user 0.019s	sys 0.013s
I20260709 12:30:15.141746 17393 tablet_bootstrap.cc:654] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.141244 17016 catalog_manager.cc:5697] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.147897 17117 raft_consensus.cc:3060] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.152503 17536 raft_consensus.cc:493] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.144798 17390 raft_consensus.cc:359] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.164875 17613 consensus_queue.cc:1048] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:30:15.181016 17536 raft_consensus.cc:515] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.157249 17017 catalog_manager.cc:5697] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.186823 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:15.187444 17017 catalog_manager.cc:5697] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.183578 17536 leader_election.cc:290] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.185220 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.193475 17084 leader_election.cc:304] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:15.194445 17574 raft_consensus.cc:2749] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.164036 17620 raft_consensus.cc:493] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.181011 17390 raft_consensus.cc:385] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.154191 17393 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:15.196112 17393 ts_tablet_manager.cc:1403] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.061s	user 0.007s	sys 0.012s
I20260709 12:30:15.196309 17620 raft_consensus.cc:515] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.195793 17390 raft_consensus.cc:740] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
W20260709 12:30:15.203145 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:15.203929 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1711901c61714bc1bbb4f2bec64dadb3" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.203825 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1711901c61714bc1bbb4f2bec64dadb3" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.204880 17266 raft_consensus.cc:2468] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
W20260709 12:30:15.206177 17161 leader_election.cc:343] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.205505 17620 leader_election.cc:290] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:15.212181 17390 consensus_queue.cc:260] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.211385 17159 leader_election.cc:304] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.213850 17620 raft_consensus.cc:2804] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.214210 17620 raft_consensus.cc:493] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.214569 17620 raft_consensus.cc:3060] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.218470 17393 raft_consensus.cc:359] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.219154 17393 raft_consensus.cc:385] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.219426 17393 raft_consensus.cc:740] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.219594 17391 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:15.220197 17391 ts_tablet_manager.cc:1403] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.146s	user 0.014s	sys 0.000s
I20260709 12:30:15.221491 17390 ts_tablet_manager.cc:1434] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.080s	user 0.003s	sys 0.006s
I20260709 12:30:15.223273 17391 raft_consensus.cc:359] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.224120 17391 raft_consensus.cc:385] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.224416 17391 raft_consensus.cc:740] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.224546 17390 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:15.226888 17389 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:15.228078 17389 ts_tablet_manager.cc:1403] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.097s	user 0.014s	sys 0.015s
I20260709 12:30:15.231273 17389 raft_consensus.cc:359] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.231946 17389 raft_consensus.cc:385] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.232210 17389 raft_consensus.cc:740] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.232460 17117 raft_consensus.cc:3060] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.232874 17389 consensus_queue.cc:260] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.235312 17389 ts_tablet_manager.cc:1434] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:30:15.236447 17389 tablet_bootstrap.cc:492] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:15.225193 17391 consensus_queue.cc:260] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.240250 17391 ts_tablet_manager.cc:1434] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:30:15.241330 17391 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.226732 17393 consensus_queue.cc:260] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.243693 17393 ts_tablet_manager.cc:1434] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.028s	user 0.005s	sys 0.000s
I20260709 12:30:15.244865 17393 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:15.247601 17620 raft_consensus.cc:515] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.249512 17389 tablet_bootstrap.cc:654] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.251572 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1711901c61714bc1bbb4f2bec64dadb3" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
W20260709 12:30:15.253307 17161 leader_election.cc:343] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.254171 17620 leader_election.cc:290] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:15.254515 17390 tablet_bootstrap.cc:654] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.256392 17391 tablet_bootstrap.cc:654] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.256609 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1711901c61714bc1bbb4f2bec64dadb3" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:15.257332 17266 raft_consensus.cc:3060] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.268175 17393 tablet_bootstrap.cc:654] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.271859 17510 consensus_queue.cc:1048] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.272989 17266 raft_consensus.cc:2468] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:15.275311 17159 leader_election.cc:304] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.276424 17510 raft_consensus.cc:493] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.276624 17618 raft_consensus.cc:2804] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.276919 17510 raft_consensus.cc:515] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.277801 17618 raft_consensus.cc:697] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:15.278987 17510 leader_election.cc:290] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.278724 17618 consensus_queue.cc:237] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.283175 17614 raft_consensus.cc:493] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.283653 17614 raft_consensus.cc:515] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.285516 17614 leader_election.cc:290] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.294307 17390 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:15.294917 17390 ts_tablet_manager.cc:1403] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.071s	user 0.006s	sys 0.007s
I20260709 12:30:15.297657 17390 raft_consensus.cc:359] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.298626 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e37e6c15d9f4c37b87d6f494bdc66bf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.299382 17390 raft_consensus.cc:385] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.299784 17390 raft_consensus.cc:740] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.300464 17390 consensus_queue.cc:260] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.302198 17390 ts_tablet_manager.cc:1434] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.001s
I20260709 12:30:15.303066 17390 tablet_bootstrap.cc:492] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:15.311010 17393 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:15.311504 17393 ts_tablet_manager.cc:1403] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.000s
I20260709 12:30:15.313628 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e37e6c15d9f4c37b87d6f494bdc66bf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.314895 17604 raft_consensus.cc:493] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.315256 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd19f1dc408f4801b7e7d672696228cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.315382 17604 raft_consensus.cc:515] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.318117 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd19f1dc408f4801b7e7d672696228cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.319275 17117 raft_consensus.cc:2393] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 6c1eb70ad94940b89118a32a370a4310 in this term.
I20260709 12:30:15.314432 17269 raft_consensus.cc:2468] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:15.322104 17393 raft_consensus.cc:359] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:15.322458 17159 leader_election.cc:343] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.323040 17393 raft_consensus.cc:385] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.325805 17393 raft_consensus.cc:740] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.323372 17604 leader_election.cc:290] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:15.324904 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47941e6ecfe34dfeb4aeb7f68e020eb5" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:15.328500 17195 raft_consensus.cc:2468] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:15.329228 17393 consensus_queue.cc:260] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.331004 17308 leader_election.cc:304] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:15.331610 17393 ts_tablet_manager.cc:1434] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 12:30:15.331893 17604 raft_consensus.cc:2804] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.332513 17604 raft_consensus.cc:493] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.332659 17393 tablet_bootstrap.cc:492] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:15.333006 17604 raft_consensus.cc:3060] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.336509 17390 tablet_bootstrap.cc:654] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.338701 17501 raft_consensus.cc:493] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.339630 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47941e6ecfe34dfeb4aeb7f68e020eb5" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.339694 17501 raft_consensus.cc:515] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:15.342134 17307 leader_election.cc:343] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.343441 17501 leader_election.cc:290] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:15.345088 17159 leader_election.cc:304] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:15.346114 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a9d0d27df74b748881bc3ef8245a62" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.345302 17017 catalog_manager.cc:5697] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.346915 17342 raft_consensus.cc:2468] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:15.348512 17235 leader_election.cc:304] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:15.352003 17501 raft_consensus.cc:2804] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.352504 17501 raft_consensus.cc:493] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.352952 17501 raft_consensus.cc:3060] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.355509 17159 leader_election.cc:304] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:15.356319 17618 raft_consensus.cc:3060] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.356559 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a9d0d27df74b748881bc3ef8245a62" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:15.359126 17233 leader_election.cc:343] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.359959 17391 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:15.360574 17391 ts_tablet_manager.cc:1403] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.119s	user 0.010s	sys 0.035s
I20260709 12:30:15.363883 17391 raft_consensus.cc:359] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.364606 17391 raft_consensus.cc:385] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.364886 17391 raft_consensus.cc:740] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.365674 17391 consensus_queue.cc:260] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.368077 17391 ts_tablet_manager.cc:1434] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:15.369084 17391 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.371327 17618 raft_consensus.cc:2749] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.374334 17618 raft_consensus.cc:2804] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.385653 17393 tablet_bootstrap.cc:654] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.386863 17389 tablet_bootstrap.cc:492] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:15.387458 17389 ts_tablet_manager.cc:1403] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.151s	user 0.025s	sys 0.036s
I20260709 12:30:15.382268 17618 raft_consensus.cc:493] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.392458 17389 raft_consensus.cc:359] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.390334 17618 raft_consensus.cc:3060] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.393415 17391 tablet_bootstrap.cc:654] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.393760 17393 tablet_bootstrap.cc:492] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:15.394381 17393 ts_tablet_manager.cc:1403] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.062s	user 0.013s	sys 0.000s
I20260709 12:30:15.395588 17390 tablet_bootstrap.cc:492] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:15.396086 17390 ts_tablet_manager.cc:1403] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.001s
I20260709 12:30:15.397516 17393 raft_consensus.cc:359] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.398265 17393 raft_consensus.cc:385] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.398561 17393 raft_consensus.cc:740] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.399305 17393 consensus_queue.cc:260] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.400487 17390 raft_consensus.cc:359] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.401593 17393 ts_tablet_manager.cc:1434] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:15.401564 17390 raft_consensus.cc:385] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.406677 17390 raft_consensus.cc:740] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.407595 17390 consensus_queue.cc:260] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.405172 17501 raft_consensus.cc:515] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.410648 17501 leader_election.cc:290] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:15.411417 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a9d0d27df74b748881bc3ef8245a62" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:15.411783 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a9d0d27df74b748881bc3ef8245a62" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:15.412204 17341 raft_consensus.cc:3060] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.412424 17390 ts_tablet_manager.cc:1434] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.002s
I20260709 12:30:15.414157 17390 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
W20260709 12:30:15.415081 17233 leader_election.cc:343] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.421653 17390 tablet_bootstrap.cc:654] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.393244 17389 raft_consensus.cc:385] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.406781 17393 tablet_bootstrap.cc:492] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:15.404829 17391 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:15.427547 17391 ts_tablet_manager.cc:1403] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.006s
I20260709 12:30:15.431241 17391 raft_consensus.cc:359] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.431933 17391 raft_consensus.cc:385] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.432189 17391 raft_consensus.cc:740] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.433039 17391 consensus_queue.cc:260] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.434028 17389 raft_consensus.cc:740] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.435122 17389 consensus_queue.cc:260] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.437657 17389 ts_tablet_manager.cc:1434] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 12:30:15.437729 17341 raft_consensus.cc:2468] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:15.439041 17389 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:15.440434 17391 ts_tablet_manager.cc:1434] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.013s	user 0.002s	sys 0.005s
I20260709 12:30:15.441517 17391 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.446563 17389 tablet_bootstrap.cc:654] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.393239 17604 raft_consensus.cc:515] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.451426 17618 raft_consensus.cc:515] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.456167 17235 leader_election.cc:304] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:15.457587 17623 raft_consensus.cc:2804] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.458153 17623 raft_consensus.cc:697] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:15.459301 17393 tablet_bootstrap.cc:654] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.459098 17623 consensus_queue.cc:237] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.463919 17604 leader_election.cc:290] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:15.466751 17391 tablet_bootstrap.cc:654] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.467237 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47941e6ecfe34dfeb4aeb7f68e020eb5" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:15.468012 17507 consensus_queue.cc:1048] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.468248 17194 raft_consensus.cc:3060] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.469904 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47941e6ecfe34dfeb4aeb7f68e020eb5" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
W20260709 12:30:15.473627 17307 leader_election.cc:343] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.474063 17116 raft_consensus.cc:3060] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.474910 17389 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:15.475409 17389 ts_tablet_manager.cc:1403] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.005s
I20260709 12:30:15.480785 17389 raft_consensus.cc:359] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.481452 17389 raft_consensus.cc:385] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.481735 17389 raft_consensus.cc:740] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.482378 17389 consensus_queue.cc:260] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.484488 17389 ts_tablet_manager.cc:1434] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:30:15.485446 17389 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:15.493243 17604 consensus_queue.cc:1048] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.495134 17390 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:15.495760 17390 ts_tablet_manager.cc:1403] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.082s	user 0.031s	sys 0.014s
I20260709 12:30:15.499536 17194 raft_consensus.cc:2468] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:15.501919 17308 leader_election.cc:304] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:15.502828 17574 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.503171 17604 raft_consensus.cc:2804] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.503499 17574 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.504630 17604 raft_consensus.cc:697] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:15.505275 17390 raft_consensus.cc:359] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.505975 17390 raft_consensus.cc:385] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.506347 17390 raft_consensus.cc:740] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.507354 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.508074 17269 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:15.509545 17084 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:15.509744 17390 consensus_queue.cc:260] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.510504 17536 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:15.510922 17536 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.511265 17536 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:15.513159 17618 leader_election.cc:290] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.513998 17390 ts_tablet_manager.cc:1434] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:30:15.513859 17604 consensus_queue.cc:237] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.515138 17390 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:15.516165 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e37e6c15d9f4c37b87d6f494bdc66bf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:15.516908 17269 raft_consensus.cc:3060] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.521135 17389 tablet_bootstrap.cc:654] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.505900 17574 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.535436 17390 tablet_bootstrap.cc:654] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.543383 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:15.544263 17194 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:15.547554 17614 raft_consensus.cc:493] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.547931 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e37e6c15d9f4c37b87d6f494bdc66bf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:15.548139 17614 raft_consensus.cc:515] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:15.549713 17159 leader_election.cc:343] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.551147 17614 leader_election.cc:290] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:15.552213 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec669aa1bf9434f9b198f6d8e768348" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.558420 17017 catalog_manager.cc:5697] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.560122 17391 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:15.560702 17391 ts_tablet_manager.cc:1403] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.017s
I20260709 12:30:15.566138 17391 raft_consensus.cc:359] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.566924 17391 raft_consensus.cc:385] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.567196 17391 raft_consensus.cc:740] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.567905 17391 consensus_queue.cc:260] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.572559 17501 consensus_queue.cc:1048] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.573771 17391 ts_tablet_manager.cc:1434] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:30:15.575037 17391 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.583330 17604 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.584069 17507 consensus_queue.cc:1048] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.584134 17604 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.588617 17604 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:15.590360 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.591185 17266 raft_consensus.cc:2468] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:15.592950 17307 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:15.594173 17604 raft_consensus.cc:2804] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.594807 17604 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.595275 17604 raft_consensus.cc:3060] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.598948 17507 raft_consensus.cc:493] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.599529 17507 raft_consensus.cc:515] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.600582 17501 consensus_queue.cc:1048] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.602576 17507 leader_election.cc:290] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:15.595329 17391 tablet_bootstrap.cc:654] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.606582 17604 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.609915 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9614ba0ca1894975b6696b3724157ee6" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:15.612224 17235 leader_election.cc:343] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.613178 17604 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:15.614030 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:15.614836 17266 raft_consensus.cc:3060] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.616982 17391 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:15.617684 17391 ts_tablet_manager.cc:1403] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.002s
I20260709 12:30:15.621158 17507 consensus_queue.cc:1048] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.623260 17391 raft_consensus.cc:359] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.624589 17391 raft_consensus.cc:385] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.625183 17391 raft_consensus.cc:740] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.626772 17391 consensus_queue.cc:260] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.629690 17391 ts_tablet_manager.cc:1434] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:30:15.631067 17391 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.635578 17501 consensus_queue.cc:1048] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.638872 17391 tablet_bootstrap.cc:654] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.639178 17618 raft_consensus.cc:493] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.639909 17618 raft_consensus.cc:515] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.641652 17019 catalog_manager.cc:5697] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.653367 17266 raft_consensus.cc:2468] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:15.660317 17576 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.660923 17576 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.663327 17576 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:15.555157 17536 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:15.667634 17161 leader_election.cc:343] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.676154 17393 tablet_bootstrap.cc:492] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:15.676676 17618 leader_election.cc:290] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.678251 17393 ts_tablet_manager.cc:1403] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.272s	user 0.015s	sys 0.018s
W20260709 12:30:15.679206 17661 log.cc:927] Time spent T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
W20260709 12:30:15.704638 17665 log.cc:927] Time spent T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:30:15.680722 17389 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:15.683265 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9614ba0ca1894975b6696b3724157ee6" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.709599 17389 ts_tablet_manager.cc:1403] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.224s	user 0.012s	sys 0.004s
I20260709 12:30:15.685340 17118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.710414 17118 raft_consensus.cc:2468] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:15.686936 17114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:15.711669 17114 raft_consensus.cc:3060] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.713310 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.691380 17307 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:15.714664 17604 raft_consensus.cc:2804] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.716773 17618 consensus_queue.cc:1048] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.720855 17269 raft_consensus.cc:2468] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:15.721575 17114 raft_consensus.cc:2468] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
W20260709 12:30:15.693903 17666 log.cc:927] Time spent T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.088s	user 0.004s	sys 0.000s
I20260709 12:30:15.696504 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec669aa1bf9434f9b198f6d8e768348" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.698988 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.704128 17536 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
W20260709 12:30:15.726105 17159 leader_election.cc:343] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.726999 17159 leader_election.cc:304] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
W20260709 12:30:15.728416 17233 leader_election.cc:343] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.728760 17620 raft_consensus.cc:2804] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.729508 17620 raft_consensus.cc:697] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:15.730553 17620 consensus_queue.cc:237] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.733737 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:15.734803 17389 raft_consensus.cc:359] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.735590 17389 raft_consensus.cc:385] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.735859 17389 raft_consensus.cc:740] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
W20260709 12:30:15.736296 17159 leader_election.cc:343] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.736745 17159 leader_election.cc:304] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.737453 17510 raft_consensus.cc:2749] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.737506 17393 raft_consensus.cc:359] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.738607 17393 raft_consensus.cc:385] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.739050 17393 raft_consensus.cc:740] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.739886 17393 consensus_queue.cc:260] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.729003 17233 leader_election.cc:304] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.746001 17440 raft_consensus.cc:2749] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:15.757011 17159 leader_election.cc:343] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.758591 17159 leader_election.cc:304] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:15.759284 17510 raft_consensus.cc:2749] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.761977 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:15.742340 17393 ts_tablet_manager.cc:1434] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 12:30:15.763145 17389 consensus_queue.cc:260] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.762761 17194 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:15.778857 17390 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:15.779392 17390 ts_tablet_manager.cc:1403] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.264s	user 0.033s	sys 0.050s
I20260709 12:30:15.783023 17339 raft_consensus.cc:3060] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.795887 17390 raft_consensus.cc:359] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.796859 17390 raft_consensus.cc:385] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.797644 17393 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:15.797196 17390 raft_consensus.cc:740] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.802369 17389 ts_tablet_manager.cc:1434] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.092s	user 0.008s	sys 0.000s
I20260709 12:30:15.791725 17604 raft_consensus.cc:697] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:15.798338 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.802716 17390 consensus_queue.cc:260] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:15.794945 17086 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 82502.3us
I20260709 12:30:15.817060 17604 consensus_queue.cc:237] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.822737 17390 ts_tablet_manager.cc:1434] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.043s	user 0.009s	sys 0.000s
W20260709 12:30:15.805305 17084 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 105554us
I20260709 12:30:15.843500 17574 raft_consensus.cc:493] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.844015 17574 raft_consensus.cc:515] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:15.846105 17574 leader_election.cc:290] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:15.848990 17507 raft_consensus.cc:493] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.849562 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:15.850359 17265 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:15.849651 17507 raft_consensus.cc:515] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.851984 17390 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:15.853519 17389 tablet_bootstrap.cc:492] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:15.857429 17507 leader_election.cc:290] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:15.859826 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce19f767e060412c8bc15f391d74b87f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:15.860488 17193 raft_consensus.cc:2468] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:15.861910 17234 leader_election.cc:304] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:15.863749 17440 raft_consensus.cc:2804] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.864166 17440 raft_consensus.cc:493] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.864539 17440 raft_consensus.cc:3060] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.864835 17391 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:15.864665 17390 tablet_bootstrap.cc:654] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.866703 17391 ts_tablet_manager.cc:1403] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.236s	user 0.024s	sys 0.005s
I20260709 12:30:15.868398 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a9d0d27df74b748881bc3ef8245a62" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.865482 17266 raft_consensus.cc:2393] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:15.873320 17340 raft_consensus.cc:2393] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 6c1eb70ad94940b89118a32a370a4310 in this term.
I20260709 12:30:15.868398 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce19f767e060412c8bc15f391d74b87f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.876245 17391 raft_consensus.cc:359] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.876817 17391 raft_consensus.cc:385] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:15.879277 17235 leader_election.cc:343] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.877005 17391 raft_consensus.cc:740] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.881289 17391 consensus_queue.cc:260] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.896605 17391 ts_tablet_manager.cc:1434] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.002s
I20260709 12:30:15.891993 17507 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.897922 17507 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.891716 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b6a9d0d27df74b748881bc3ef8245a62" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.881705 17514 consensus_queue.cc:1048] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:30:15.912369 17086 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.913393 17574 raft_consensus.cc:2749] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.914311 17084 leader_election.cc:304] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:15.915874 17265 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:15.906782 17674 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.916972 17674 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.917810 17507 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:15.918690 17536 raft_consensus.cc:3060] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.919775 17683 raft_consensus.cc:493] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.920596 17618 consensus_queue.cc:1048] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:15.921497 17674 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:15.920312 17683 raft_consensus.cc:515] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.926090 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:15.926857 17193 raft_consensus.cc:2468] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:15.922735 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.929448 17085 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:15.916934 17440 raft_consensus.cc:515] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:15.936806 17536 raft_consensus.cc:2749] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.937628 17574 raft_consensus.cc:2804] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.938153 17574 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.938621 17574 raft_consensus.cc:3060] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.941059 17391 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:15.947052 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:15.947129 17086 leader_election.cc:343] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:15.950019 17234 leader_election.cc:343] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.951303 17683 leader_election.cc:290] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:15.933980 17084 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:15.952664 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1711901c61714bc1bbb4f2bec64dadb3" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:15.953858 17673 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:15.953353 17018 catalog_manager.cc:5697] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.954680 17673 raft_consensus.cc:697] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:15.955276 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:15.956007 17673 consensus_queue.cc:237] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:15.958608 17393 tablet_bootstrap.cc:654] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.959731 17507 consensus_queue.cc:1048] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:15.960207 17235 leader_election.cc:343] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.962975 17235 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:15.963871 17507 raft_consensus.cc:2749] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.963565 17574 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.966328 17389 tablet_bootstrap.cc:654] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.966656 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:15.967674 17391 tablet_bootstrap.cc:654] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:15.967908 17086 leader_election.cc:343] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.969950 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1711901c61714bc1bbb4f2bec64dadb3" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:15.936208 17016 catalog_manager.cc:5697] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.979208 17307 leader_election.cc:304] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:15.981026 17114 raft_consensus.cc:3060] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.982579 17618 raft_consensus.cc:493] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.983263 17618 raft_consensus.cc:515] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:15.926074 17194 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:15.981833 17574 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:15.988121 17618 leader_election.cc:290] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:16.008621 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:16.013849 17574 raft_consensus.cc:493] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.026646 17514 raft_consensus.cc:2749] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:16.014087 17161 leader_election.cc:343] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.017211 17393 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.029742 17393 ts_tablet_manager.cc:1403] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.257s	user 0.020s	sys 0.009s
I20260709 12:30:16.017364 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce19f767e060412c8bc15f391d74b87f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:16.018178 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:16.017364 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:16.021656 17390 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.022049 17604 consensus_queue.cc:1048] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.024144 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ce19f767e060412c8bc15f391d74b87f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:16.026273 17440 leader_election.cc:290] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:16.033105 17194 raft_consensus.cc:3060] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.033401 17196 raft_consensus.cc:3060] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.032855 17390 ts_tablet_manager.cc:1403] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.191s	user 0.022s	sys 0.021s
W20260709 12:30:16.038277 17159 leader_election.cc:343] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.038833 17159 leader_election.cc:304] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.040298 17620 raft_consensus.cc:2749] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:16.043241 17235 leader_election.cc:343] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.052867 17393 raft_consensus.cc:359] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.053777 17393 raft_consensus.cc:385] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.054133 17393 raft_consensus.cc:740] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.055025 17393 consensus_queue.cc:260] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.055243 17675 consensus_queue.cc:1048] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.030164 17574 raft_consensus.cc:515] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.048234 17194 raft_consensus.cc:2468] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:16.065063 17393 ts_tablet_manager.cc:1434] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 12:30:16.069826 17390 raft_consensus.cc:359] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.096989 17390 raft_consensus.cc:385] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.097417 17390 raft_consensus.cc:740] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.098464 17390 consensus_queue.cc:260] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.101588 17390 ts_tablet_manager.cc:1434] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.000s
I20260709 12:30:16.073920 17196 raft_consensus.cc:2468] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:16.075169 17019 catalog_manager.cc:5697] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 2, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.076067 17234 leader_election.cc:304] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.103658 17085 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.104131 17440 raft_consensus.cc:2804] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.104391 17393 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.104619 17674 raft_consensus.cc:2804] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.104765 17440 raft_consensus.cc:697] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:16.105278 17674 raft_consensus.cc:697] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:16.105906 17440 consensus_queue.cc:237] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.106241 17674 consensus_queue.cc:237] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.081506 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e37e6c15d9f4c37b87d6f494bdc66bf" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:16.110672 17390 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.082095 17574 leader_election.cc:290] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:16.087128 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e37e6c15d9f4c37b87d6f494bdc66bf" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:16.112528 17265 raft_consensus.cc:2393] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:16.113672 17084 leader_election.cc:304] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:16.088994 17391 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:16.114935 17391 ts_tablet_manager.cc:1403] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.174s	user 0.013s	sys 0.001s
I20260709 12:30:16.117616 17390 tablet_bootstrap.cc:654] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.095073 17389 tablet_bootstrap.cc:492] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.118673 17389 ts_tablet_manager.cc:1403] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.287s	user 0.010s	sys 0.028s
I20260709 12:30:16.121322 17389 raft_consensus.cc:359] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.121904 17389 raft_consensus.cc:385] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.122169 17389 raft_consensus.cc:740] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.117712 17391 raft_consensus.cc:359] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.123801 17391 raft_consensus.cc:385] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.123288 17674 raft_consensus.cc:3060] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.125488 17391 raft_consensus.cc:740] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.124629 17389 consensus_queue.cc:260] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.128172 17393 tablet_bootstrap.cc:654] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.129253 17389 ts_tablet_manager.cc:1434] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 12:30:16.126380 17391 consensus_queue.cc:260] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.133145 17391 ts_tablet_manager.cc:1434] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.001s
I20260709 12:30:16.139169 17674 raft_consensus.cc:2749] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.130427 17389 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.152452 17019 catalog_manager.cc:5697] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.153833 17391 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:16.154453 17507 consensus_queue.cc:1048] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:30:16.164469 17016 catalog_manager.cc:5697] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.185238 17389 tablet_bootstrap.cc:654] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.190537 17391 tablet_bootstrap.cc:654] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.218165 17393 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.218962 17393 ts_tablet_manager.cc:1403] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.115s	user 0.019s	sys 0.000s
I20260709 12:30:16.225272 17604 consensus_queue.cc:1048] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.222034 17393 raft_consensus.cc:359] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.235590 17393 raft_consensus.cc:385] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.235966 17393 raft_consensus.cc:740] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.236726 17393 consensus_queue.cc:260] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.239777 17393 ts_tablet_manager.cc:1434] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:30:16.240886 17393 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.244859 17604 consensus_queue.cc:1048] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.249039 17393 tablet_bootstrap.cc:654] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.259913 17620 consensus_queue.cc:1048] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.261911 17393 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.262603 17393 ts_tablet_manager.cc:1403] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.001s
I20260709 12:30:16.265843 17393 raft_consensus.cc:359] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.265753 17604 raft_consensus.cc:493] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.271005 17604 raft_consensus.cc:515] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.273972 17604 leader_election.cc:290] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:16.281107 17389 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.281718 17389 ts_tablet_manager.cc:1403] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.152s	user 0.036s	sys 0.032s
I20260709 12:30:16.284437 17389 raft_consensus.cc:359] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.285145 17389 raft_consensus.cc:385] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.285455 17389 raft_consensus.cc:740] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.286178 17389 consensus_queue.cc:260] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.288115 17389 ts_tablet_manager.cc:1434] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 12:30:16.289109 17389 tablet_bootstrap.cc:492] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.292543 17620 consensus_queue.cc:1048] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.294938 17390 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.296985 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9614ba0ca1894975b6696b3724157ee6" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:16.297705 17390 ts_tablet_manager.cc:1403] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.195s	user 0.026s	sys 0.020s
I20260709 12:30:16.297827 17265 raft_consensus.cc:2468] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:16.301174 17389 tablet_bootstrap.cc:654] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.303321 17307 leader_election.cc:304] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:16.304419 17604 raft_consensus.cc:2804] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.304924 17604 raft_consensus.cc:493] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.305375 17604 raft_consensus.cc:3060] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.301746 17393 raft_consensus.cc:385] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.307207 17393 raft_consensus.cc:740] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.308071 17393 consensus_queue.cc:260] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.310621 17393 ts_tablet_manager.cc:1434] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:30:16.311880 17393 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.312433 17389 tablet_bootstrap.cc:492] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.313284 17389 ts_tablet_manager.cc:1403] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.008s
I20260709 12:30:16.316299 17389 raft_consensus.cc:359] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.316980 17389 raft_consensus.cc:385] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.317299 17389 raft_consensus.cc:740] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.317924 17389 consensus_queue.cc:260] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.319188 17114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9614ba0ca1894975b6696b3724157ee6" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:16.320055 17389 ts_tablet_manager.cc:1434] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:30:16.321167 17389 tablet_bootstrap.cc:492] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.326699 17507 raft_consensus.cc:493] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.327257 17507 raft_consensus.cc:515] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.329582 17391 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:16.330201 17391 ts_tablet_manager.cc:1403] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.177s	user 0.013s	sys 0.001s
I20260709 12:30:16.330711 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:16.331275 17507 leader_election.cc:290] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:16.333478 17391 raft_consensus.cc:359] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.335845 17391 raft_consensus.cc:385] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.336226 17391 raft_consensus.cc:740] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
W20260709 12:30:16.336771 17307 leader_election.cc:343] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.333761 17389 tablet_bootstrap.cc:654] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:16.336853 17235 leader_election.cc:343] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.337014 17391 consensus_queue.cc:260] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.341395 17391 ts_tablet_manager.cc:1434] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:30:16.338896 17393 tablet_bootstrap.cc:654] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.342861 17391 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:16.348832 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:16.351539 17391 tablet_bootstrap.cc:654] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.352821 17390 raft_consensus.cc:359] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.353639 17390 raft_consensus.cc:385] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.353994 17390 raft_consensus.cc:740] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
W20260709 12:30:16.354417 17233 leader_election.cc:343] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.354933 17233 leader_election.cc:304] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.356407 17623 raft_consensus.cc:2749] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.358747 17390 consensus_queue.cc:260] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.361174 17390 ts_tablet_manager.cc:1434] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:30:16.362142 17390 tablet_bootstrap.cc:492] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.369313 17604 raft_consensus.cc:515] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.391036 17675 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.391552 17675 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.393035 17683 raft_consensus.cc:493] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.393592 17675 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:16.393968 17683 raft_consensus.cc:515] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.396885 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:16.397768 17194 raft_consensus.cc:2393] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:16.399353 17614 raft_consensus.cc:493] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.399816 17614 raft_consensus.cc:515] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.401918 17614 leader_election.cc:290] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:16.401218 17683 leader_election.cc:290] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:16.403621 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:16.403971 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec669aa1bf9434f9b198f6d8e768348" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:16.404341 17269 raft_consensus.cc:2468] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:16.404598 17194 raft_consensus.cc:2468] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:16.384918 17389 tablet_bootstrap.cc:492] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.385289 17510 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.406183 17389 ts_tablet_manager.cc:1403] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.007s
I20260709 12:30:16.405799 17308 leader_election.cc:304] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:16.406208 17510 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.407426 17683 raft_consensus.cc:2804] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.408010 17683 raft_consensus.cc:493] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.408511 17683 raft_consensus.cc:3060] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.408880 17510 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:16.409164 17389 raft_consensus.cc:359] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.409854 17389 raft_consensus.cc:385] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.410113 17389 raft_consensus.cc:740] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.410728 17389 consensus_queue.cc:260] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.410944 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:16.414999 17161 leader_election.cc:343] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.419476 17604 leader_election.cc:290] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:16.406615 17159 leader_election.cc:304] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:16.421576 17614 raft_consensus.cc:2804] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.421968 17614 raft_consensus.cc:493] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.422372 17614 raft_consensus.cc:3060] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.423921 17389 ts_tablet_manager.cc:1434] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:30:16.425064 17389 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.430136 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:16.430924 17265 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:16.432426 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9614ba0ca1894975b6696b3724157ee6" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:16.433108 17269 raft_consensus.cc:3060] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.435038 17159 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.436837 17620 raft_consensus.cc:2804] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.437247 17620 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.437610 17620 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.439150 17393 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.422271 17390 tablet_bootstrap.cc:654] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.440442 17393 ts_tablet_manager.cc:1403] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.129s	user 0.008s	sys 0.007s
I20260709 12:30:16.449651 17393 raft_consensus.cc:359] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.450760 17393 raft_consensus.cc:385] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.451134 17393 raft_consensus.cc:740] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.453084 17389 tablet_bootstrap.cc:654] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.460958 17393 consensus_queue.cc:260] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.466846 17620 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.469892 17620 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:16.470727 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
W20260709 12:30:16.473042 17161 leader_election.cc:343] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:16.486284 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:16.491516 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:16.492576 17266 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.493470 17265 raft_consensus.cc:1275] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:16.459821 17683 raft_consensus.cc:515] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:16.481586 17709 log.cc:927] Time spent T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:30:16.500099 17674 consensus_queue.cc:1048] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.500746 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:16.503952 17391 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:16.504578 17391 ts_tablet_manager.cc:1403] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.162s	user 0.062s	sys 0.063s
I20260709 12:30:16.505402 17614 raft_consensus.cc:515] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.507272 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec669aa1bf9434f9b198f6d8e768348" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:16.508011 17120 raft_consensus.cc:2468] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:16.507117 17393 ts_tablet_manager.cc:1434] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.061s	user 0.010s	sys 0.001s
I20260709 12:30:16.508257 17391 raft_consensus.cc:359] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.509298 17391 raft_consensus.cc:385] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.509711 17391 raft_consensus.cc:740] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.513891 17393 tablet_bootstrap.cc:492] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.514019 17390 tablet_bootstrap.cc:492] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.514883 17390 ts_tablet_manager.cc:1403] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.153s	user 0.026s	sys 0.009s
I20260709 12:30:16.511142 17391 consensus_queue.cc:260] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.518456 17390 raft_consensus.cc:359] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.519229 17390 raft_consensus.cc:385] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.519521 17390 raft_consensus.cc:740] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.520521 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9614ba0ca1894975b6696b3724157ee6" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:16.521345 17120 raft_consensus.cc:3060] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.523275 17683 leader_election.cc:290] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:16.523883 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec669aa1bf9434f9b198f6d8e768348" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:16.524042 17390 consensus_queue.cc:260] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.524498 17194 raft_consensus.cc:3060] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.526232 17390 ts_tablet_manager.cc:1434] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:30:16.515888 17269 raft_consensus.cc:2468] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:16.531742 17194 raft_consensus.cc:2468] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:16.527524 17390 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.532285 17340 raft_consensus.cc:3060] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.532431 17307 leader_election.cc:304] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:16.533448 17683 raft_consensus.cc:2804] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.534181 17683 raft_consensus.cc:697] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:16.534873 17308 leader_election.cc:304] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:16.535526 17266 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:16.536070 17604 raft_consensus.cc:2804] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.536746 17604 raft_consensus.cc:697] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:16.537317 17159 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.537688 17604 consensus_queue.cc:237] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.540500 17683 consensus_queue.cc:237] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.531945 17391 ts_tablet_manager.cc:1434] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.002s
I20260709 12:30:16.538533 17620 raft_consensus.cc:2804] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.538965 17507 raft_consensus.cc:493] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.545217 17507 raft_consensus.cc:515] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.545204 17620 raft_consensus.cc:697] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:16.547646 17507 leader_election.cc:290] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:16.527498 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:16.549108 17268 raft_consensus.cc:3060] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.551555 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:16.553889 17235 leader_election.cc:343] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.556854 17114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6ec669aa1bf9434f9b198f6d8e768348" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:16.557616 17114 raft_consensus.cc:3060] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.547647 17620 consensus_queue.cc:237] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.564502 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:16.567840 17574 raft_consensus.cc:493] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.568470 17574 raft_consensus.cc:515] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.570143 17501 consensus_queue.cc:1048] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.567723 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:16.522887 17614 leader_election.cc:290] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:16.576085 17307 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:16.578260 17574 leader_election.cc:290] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:16.578639 17684 raft_consensus.cc:3060] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.582798 17440 consensus_queue.cc:1048] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:30:16.586776 17114 raft_consensus.cc:2468] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:16.589359 17120 raft_consensus.cc:2468] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:16.579970 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:16.594587 17267 raft_consensus.cc:2468] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
W20260709 12:30:16.592883 17159 leader_election.cc:343] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.596115 17084 leader_election.cc:304] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:16.599005 17574 raft_consensus.cc:2804] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.599478 17574 raft_consensus.cc:493] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.599840 17574 raft_consensus.cc:3060] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.600117 17684 raft_consensus.cc:2749] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.593670 17114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:16.604607 17159 leader_election.cc:343] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.607890 17389 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.608599 17389 ts_tablet_manager.cc:1403] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.184s	user 0.021s	sys 0.017s
I20260709 12:30:16.612884 17391 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:16.617369 17268 raft_consensus.cc:2468] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:16.623993 17194 raft_consensus.cc:1275] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:16.626735 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:16.627586 17195 raft_consensus.cc:2468] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:16.629379 17390 tablet_bootstrap.cc:654] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.633059 17576 consensus_queue.cc:1048] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.635246 17576 consensus_queue.cc:1048] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.580974 17393 tablet_bootstrap.cc:654] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.643183 17574 raft_consensus.cc:515] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:16.645346 17233 leader_election.cc:343] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.645953 17233 leader_election.cc:304] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.646927 17501 raft_consensus.cc:2749] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.648211 17574 leader_election.cc:290] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:16.650135 17159 leader_election.cc:304] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:16.651331 17510 raft_consensus.cc:2804] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.652015 17510 raft_consensus.cc:697] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:16.653290 17510 consensus_queue.cc:237] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.654465 17019 catalog_manager.cc:5697] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.651326 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:16.661372 17268 raft_consensus.cc:3060] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.668735 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:16.670971 17195 raft_consensus.cc:3060] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.672068 17019 catalog_manager.cc:5697] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.676929 17019 catalog_manager.cc:5697] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.674095 17389 raft_consensus.cc:359] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.678103 17389 raft_consensus.cc:385] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.678413 17389 raft_consensus.cc:740] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.680653 17389 consensus_queue.cc:260] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.681584 17390 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.682086 17390 ts_tablet_manager.cc:1403] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.155s	user 0.013s	sys 0.000s
I20260709 12:30:16.683418 17389 ts_tablet_manager.cc:1434] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.074s	user 0.009s	sys 0.000s
I20260709 12:30:16.684403 17574 consensus_queue.cc:1048] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.685479 17390 raft_consensus.cc:359] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.686154 17390 raft_consensus.cc:385] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.686386 17390 raft_consensus.cc:740] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.687183 17390 consensus_queue.cc:260] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.688203 17391 tablet_bootstrap.cc:654] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.689213 17389 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.689473 17390 ts_tablet_manager.cc:1434] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:30:16.695268 17390 tablet_bootstrap.cc:492] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.696995 17389 tablet_bootstrap.cc:654] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.708738 17017 catalog_manager.cc:5697] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.713673 17390 tablet_bootstrap.cc:654] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.717347 17510 raft_consensus.cc:493] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.718040 17510 raft_consensus.cc:515] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.722045 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:16.722633 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:16.724226 17161 leader_election.cc:343] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:16.725929 17159 leader_election.cc:343] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.726390 17159 leader_election.cc:304] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.727335 17620 raft_consensus.cc:2749] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:16.728305 17510 leader_election.cc:290] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:16.768047 17393 tablet_bootstrap.cc:492] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.768684 17393 ts_tablet_manager.cc:1403] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.256s	user 0.006s	sys 0.010s
I20260709 12:30:16.801024 17393 raft_consensus.cc:359] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.801916 17268 raft_consensus.cc:2468] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:16.803645 17393 raft_consensus.cc:385] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.803998 17393 raft_consensus.cc:740] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.804708 17393 consensus_queue.cc:260] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.804998 17510 raft_consensus.cc:493] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:16.805666 17510 raft_consensus.cc:515] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.806093 17390 tablet_bootstrap.cc:492] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.806711 17390 ts_tablet_manager.cc:1403] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.116s	user 0.020s	sys 0.028s
I20260709 12:30:16.806880 17393 ts_tablet_manager.cc:1434] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 12:30:16.808784 17195 raft_consensus.cc:2468] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 1.
I20260709 12:30:16.809540 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440c7aeff35f4f3988c3eebfef1c6acf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:16.810185 17085 leader_election.cc:304] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:16.810956 17510 leader_election.cc:290] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:16.811503 17574 raft_consensus.cc:2804] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:30:16.811597 17161 leader_election.cc:343] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.813100 17574 raft_consensus.cc:697] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:16.813508 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440c7aeff35f4f3988c3eebfef1c6acf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:16.814271 17268 raft_consensus.cc:2468] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:16.814477 17574 consensus_queue.cc:237] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.815550 17159 leader_election.cc:304] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.816988 17389 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.817243 17510 raft_consensus.cc:2804] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:16.817687 17510 raft_consensus.cc:493] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:16.817693 17389 ts_tablet_manager.cc:1403] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.133s	user 0.031s	sys 0.027s
I20260709 12:30:16.818679 17510 raft_consensus.cc:3060] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.822525 17390 raft_consensus.cc:359] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.823280 17390 raft_consensus.cc:385] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.823408 17391 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:16.823632 17390 raft_consensus.cc:740] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.824039 17391 ts_tablet_manager.cc:1403] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.214s	user 0.024s	sys 0.002s
I20260709 12:30:16.824465 17390 consensus_queue.cc:260] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.827283 17390 ts_tablet_manager.cc:1434] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.001s
I20260709 12:30:16.827298 17391 raft_consensus.cc:359] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.828555 17389 raft_consensus.cc:359] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.830268 17393 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.830776 17389 raft_consensus.cc:385] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.831483 17389 raft_consensus.cc:740] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.832261 17389 consensus_queue.cc:260] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.834444 17390 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.836004 17389 ts_tablet_manager.cc:1434] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:30:16.830211 17391 raft_consensus.cc:385] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.836495 17391 raft_consensus.cc:740] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.837378 17391 consensus_queue.cc:260] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.840014 17391 ts_tablet_manager.cc:1434] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:30:16.840284 17510 raft_consensus.cc:515] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.842984 17393 tablet_bootstrap.cc:654] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.843276 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440c7aeff35f4f3988c3eebfef1c6acf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:16.843978 17268 raft_consensus.cc:3060] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:16.843887 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440c7aeff35f4f3988c3eebfef1c6acf" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:16.844570 17016 catalog_manager.cc:5697] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:16.845793 17161 leader_election.cc:343] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:16.846451 17510 leader_election.cc:290] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:16.852417 17268 raft_consensus.cc:2468] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:16.855887 17390 tablet_bootstrap.cc:654] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.857213 17159 leader_election.cc:304] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:16.859217 17389 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.862295 17391 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:16.862607 17510 raft_consensus.cc:2804] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:16.863442 17510 raft_consensus.cc:697] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:16.864298 17510 consensus_queue.cc:237] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:16.868459 17390 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.868976 17390 ts_tablet_manager.cc:1403] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.003s
I20260709 12:30:16.871613 17390 raft_consensus.cc:359] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.872265 17390 raft_consensus.cc:385] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.872506 17390 raft_consensus.cc:740] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.874569 17389 tablet_bootstrap.cc:654] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.879386 17391 tablet_bootstrap.cc:654] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.879055 17390 consensus_queue.cc:260] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.885682 17390 ts_tablet_manager.cc:1434] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 12:30:16.886850 17390 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.896590 17389 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.897192 17389 ts_tablet_manager.cc:1403] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.004s
I20260709 12:30:16.895367 17017 catalog_manager.cc:5697] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.901041 17389 raft_consensus.cc:359] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.901803 17389 raft_consensus.cc:385] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.902074 17389 raft_consensus.cc:740] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.898306 17390 tablet_bootstrap.cc:654] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.899803 17391 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:16.906001 17391 ts_tablet_manager.cc:1403] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.065s	user 0.015s	sys 0.002s
I20260709 12:30:16.906814 17389 consensus_queue.cc:260] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.909772 17391 raft_consensus.cc:359] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.910622 17391 raft_consensus.cc:385] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.910878 17391 raft_consensus.cc:740] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.913569 17393 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.914127 17393 ts_tablet_manager.cc:1403] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.100s	user 0.009s	sys 0.008s
I20260709 12:30:16.914134 17389 ts_tablet_manager.cc:1434] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 12:30:16.911499 17391 consensus_queue.cc:260] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.919287 17390 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:16.919858 17390 ts_tablet_manager.cc:1403] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.004s
I20260709 12:30:16.921372 17391 ts_tablet_manager.cc:1434] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 12:30:16.923424 17389 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:16.923084 17393 raft_consensus.cc:359] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.924225 17393 raft_consensus.cc:385] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.924593 17393 raft_consensus.cc:740] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.925382 17393 consensus_queue.cc:260] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:16.926048 17390 raft_consensus.cc:359] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.926793 17390 raft_consensus.cc:385] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.927096 17390 raft_consensus.cc:740] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.927382 17393 ts_tablet_manager.cc:1434] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:30:16.927843 17390 consensus_queue.cc:260] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:16.928696 17393 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.929978 17390 ts_tablet_manager.cc:1434] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:30:16.936254 17389 tablet_bootstrap.cc:654] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.936396 17393 tablet_bootstrap.cc:654] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.938426 17391 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:16.943607 17604 consensus_queue.cc:1048] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.947501 17390 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:16.956802 17391 tablet_bootstrap.cc:654] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.962129 17390 tablet_bootstrap.cc:654] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.971822 17389 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:16.972630 17389 ts_tablet_manager.cc:1403] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.049s	user 0.008s	sys 0.007s
I20260709 12:30:16.974255 17393 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:16.974848 17393 ts_tablet_manager.cc:1403] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.014s
I20260709 12:30:16.976703 17389 raft_consensus.cc:359] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.977495 17393 raft_consensus.cc:359] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.978322 17393 raft_consensus.cc:385] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.978634 17393 raft_consensus.cc:740] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.979281 17393 consensus_queue.cc:260] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.977888 17389 raft_consensus.cc:385] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:16.980435 17389 raft_consensus.cc:740] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:16.981801 17393 ts_tablet_manager.cc:1434] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:16.982971 17393 tablet_bootstrap.cc:492] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:16.982748 17389 consensus_queue.cc:260] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:16.989375 17393 tablet_bootstrap.cc:654] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:16.998265 17389 ts_tablet_manager.cc:1434] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.000s
I20260709 12:30:17.013875 17675 raft_consensus.cc:493] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.014429 17675 raft_consensus.cc:515] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.014654 17604 consensus_queue.cc:1048] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:17.016578 17675 leader_election.cc:290] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:17.018741 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.019423 17268 raft_consensus.cc:2468] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:17.020756 17576 raft_consensus.cc:493] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.021211 17576 raft_consensus.cc:515] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.022464 17604 raft_consensus.cc:493] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.023120 17604 raft_consensus.cc:515] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.025302 17604 leader_election.cc:290] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.026780 17576 leader_election.cc:290] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:17.028064 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.028785 17120 raft_consensus.cc:2468] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:17.031186 17307 leader_election.cc:304] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:17.035387 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.035838 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:17.036181 17265 raft_consensus.cc:2468] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:17.036558 17340 raft_consensus.cc:2468] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:17.038012 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.038414 17084 leader_election.cc:304] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:17.039175 17194 raft_consensus.cc:2468] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:17.041163 17308 leader_election.cc:304] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:17.041651 17673 raft_consensus.cc:2804] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.042038 17673 raft_consensus.cc:493] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.043041 17684 raft_consensus.cc:2804] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.043444 17340 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.043452 17684 raft_consensus.cc:493] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.045198 17684 raft_consensus.cc:3060] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.045284 17675 raft_consensus.cc:2804] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.045815 17675 raft_consensus.cc:493] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.054888 17675 raft_consensus.cc:3060] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.053229 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:17.054246 17159 consensus_peers.cc:597] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:17.064812 17307 leader_election.cc:343] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.042361 17673 raft_consensus.cc:3060] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.058903 17683 raft_consensus.cc:493] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.070881 17683 raft_consensus.cc:515] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.072814 17683 leader_election.cc:290] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.073532 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.074326 17120 raft_consensus.cc:2468] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:17.075879 17307 leader_election.cc:304] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:17.081488 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.082661 17193 raft_consensus.cc:2468] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:17.083236 17675 raft_consensus.cc:515] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.086644 17683 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.087360 17683 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.089584 17683 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:17.091951 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.055634 17684 raft_consensus.cc:515] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.097856 17684 leader_election.cc:290] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:17.093283 17501 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.099632 17391 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:17.115688 17393 tablet_bootstrap.cc:492] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:17.102035 17675 leader_election.cc:290] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.102291 17389 tablet_bootstrap.cc:492] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:17.103484 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:17.118923 17194 raft_consensus.cc:3060] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.108006 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.109431 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:17.121263 17265 raft_consensus.cc:3060] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.122828 17390 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:17.123431 17390 ts_tablet_manager.cc:1403] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.176s	user 0.048s	sys 0.100s
I20260709 12:30:17.111003 17114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.111511 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.114524 17744 consensus_queue.cc:1048] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:17.100832 17514 raft_consensus.cc:2804] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.126230 17514 raft_consensus.cc:493] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.126324 17389 tablet_bootstrap.cc:654] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.126691 17514 raft_consensus.cc:3060] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.126664 17390 raft_consensus.cc:359] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.127506 17390 raft_consensus.cc:385] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.127832 17390 raft_consensus.cc:740] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.128584 17390 consensus_queue.cc:260] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.129643 17265 raft_consensus.cc:2468] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:17.130558 17390 ts_tablet_manager.cc:1434] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:30:17.115749 17501 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.118244 17393 ts_tablet_manager.cc:1403] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.135s	user 0.016s	sys 0.008s
I20260709 12:30:17.137724 17393 raft_consensus.cc:359] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.123733 17268 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:17.116117 17391 ts_tablet_manager.cc:1403] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.178s	user 0.008s	sys 0.016s
I20260709 12:30:17.139775 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.140743 17510 consensus_queue.cc:1048] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:17.141160 17233 leader_election.cc:343] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.141992 17389 tablet_bootstrap.cc:492] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:17.142565 17389 ts_tablet_manager.cc:1403] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.041s	user 0.019s	sys 0.000s
I20260709 12:30:17.142408 17514 raft_consensus.cc:515] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.142962 17391 raft_consensus.cc:359] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.143738 17391 raft_consensus.cc:385] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.144026 17391 raft_consensus.cc:740] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.144716 17391 consensus_queue.cc:260] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.138361 17390 tablet_bootstrap.cc:492] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:17.138451 17501 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:17.145463 17389 raft_consensus.cc:359] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.138447 17393 raft_consensus.cc:385] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.146209 17389 raft_consensus.cc:385] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.146474 17393 raft_consensus.cc:740] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.147796 17393 consensus_queue.cc:260] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:17.150206 17307 leader_election.cc:343] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.151098 17307 leader_election.cc:304] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:17.152168 17307 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:17.153477 17390 tablet_bootstrap.cc:654] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.155853 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.156597 17115 raft_consensus.cc:3060] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.168303 17390 tablet_bootstrap.cc:492] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:17.168536 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:17.168970 17390 ts_tablet_manager.cc:1403] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.003s
I20260709 12:30:17.169420 17195 raft_consensus.cc:3060] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.162439 17391 ts_tablet_manager.cc:1434] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 12:30:17.174729 17391 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:17.178807 17195 raft_consensus.cc:2468] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:17.199555 17194 raft_consensus.cc:2468] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:17.186370 17673 raft_consensus.cc:515] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.201203 17114 raft_consensus.cc:2393] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:17.146543 17389 raft_consensus.cc:740] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.202412 17389 consensus_queue.cc:260] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:17.202836 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 50748.8us
I20260709 12:30:17.204169 17389 ts_tablet_manager.cc:1434] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.061s	user 0.005s	sys 0.000s
I20260709 12:30:17.208287 17673 leader_election.cc:290] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:17.208711 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:17.209785 17265 raft_consensus.cc:2393] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:17.184309 17393 ts_tablet_manager.cc:1434] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.000s
I20260709 12:30:17.181669 17391 tablet_bootstrap.cc:654] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.188161 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:17.193816 17514 leader_election.cc:290] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.194972 17618 consensus_queue.cc:1048] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:17.211506 17514 raft_consensus.cc:2804] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:17.212037 17514 raft_consensus.cc:697] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:17.212973 17514 consensus_queue.cc:237] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.214218 17390 raft_consensus.cc:359] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.215054 17390 raft_consensus.cc:385] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.215139 17393 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:17.215417 17390 raft_consensus.cc:740] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.217289 17675 raft_consensus.cc:2804] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:17.217789 17675 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.218189 17675 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:17.216212 17390 consensus_queue.cc:260] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.180779 17115 raft_consensus.cc:2468] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:17.181491 17510 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:17.225059 17235 leader_election.cc:343] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.225499 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56a32f727d4a47368863c010ce8508c8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:17.232295 17510 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.232710 17235 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:17.235766 17389 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:17.235620 17308 leader_election.cc:304] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:17.236608 17618 consensus_queue.cc:1048] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.236933 17744 raft_consensus.cc:2804] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:17.237005 17308 leader_election.cc:304] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:17.237668 17744 raft_consensus.cc:697] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:17.238633 17086 leader_election.cc:304] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:17.236006 17501 raft_consensus.cc:2749] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.238655 17744 consensus_queue.cc:237] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.243480 17510 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:17.246508 17673 raft_consensus.cc:2749] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.235196 17684 consensus_queue.cc:1048] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:30:17.262416 17391 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:17.263262 17391 ts_tablet_manager.cc:1403] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.089s	user 0.018s	sys 0.008s
I20260709 12:30:17.277611 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:17.285310 17159 leader_election.cc:343] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.287428 17391 raft_consensus.cc:359] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.288142 17391 raft_consensus.cc:385] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:17.258540 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 64483.5us
I20260709 12:30:17.288388 17391 raft_consensus.cc:740] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.294992 17391 consensus_queue.cc:260] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.310885 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:17.314913 17620 raft_consensus.cc:493] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.315601 17393 tablet_bootstrap.cc:654] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.315515 17620 raft_consensus.cc:515] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.296061 17683 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.316746 17673 consensus_queue.cc:1048] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:17.318528 17683 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.319656 17340 raft_consensus.cc:2468] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:17.322378 17683 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.323246 17620 leader_election.cc:290] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:17.323657 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.330363 17391 ts_tablet_manager.cc:1434] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.067s	user 0.003s	sys 0.004s
I20260709 12:30:17.330837 17194 raft_consensus.cc:2468] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:17.326256 17389 tablet_bootstrap.cc:654] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.295687 17514 raft_consensus.cc:2804] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:17.328785 17390 ts_tablet_manager.cc:1434] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.159s	user 0.019s	sys 0.003s
I20260709 12:30:17.331645 17391 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:17.324909 17675 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.327430 17618 raft_consensus.cc:493] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.345348 17390 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:17.350450 17618 raft_consensus.cc:515] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.339012 17514 raft_consensus.cc:697] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:17.359611 17339 raft_consensus.cc:1240] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Rejecting Update request from peer c75b62c746bd4d90a28483bc9d5e3c84 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:30:17.359887 17618 leader_election.cc:290] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:17.360430 17514 consensus_queue.cc:237] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.360785 17576 raft_consensus.cc:493] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.366075 17675 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
W20260709 12:30:17.377130 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 52358.9us
I20260709 12:30:17.364408 17507 raft_consensus.cc:493] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.349666 17161 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:17.370751 17576 raft_consensus.cc:515] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:17.387763 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 167222us
I20260709 12:30:17.394088 17393 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:17.429355 17393 ts_tablet_manager.cc:1403] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.215s	user 0.013s	sys 0.008s
I20260709 12:30:17.427356 17391 tablet_bootstrap.cc:654] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.417600 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.422962 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.391665 17507 raft_consensus.cc:515] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.400980 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.391225 17673 raft_consensus.cc:493] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.404291 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.449930 17507 leader_election.cc:290] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.450613 17507 raft_consensus.cc:493] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.451057 17507 raft_consensus.cc:515] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.451776 17576 leader_election.cc:290] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:17.453168 17507 leader_election.cc:290] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:17.454035 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.454818 17194 raft_consensus.cc:2468] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:17.455977 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:17.444774 17760 raft_consensus.cc:2804] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.458479 17760 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.458316 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.458869 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.459828 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.460373 17389 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:17.458918 17760 raft_consensus.cc:3060] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.460943 17389 ts_tablet_manager.cc:1403] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.226s	user 0.042s	sys 0.020s
I20260709 12:30:17.463811 17265 raft_consensus.cc:2468] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:17.464051 17389 raft_consensus.cc:359] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.470904 17389 raft_consensus.cc:385] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.471235 17389 raft_consensus.cc:740] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
W20260709 12:30:17.447269 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 108689us
I20260709 12:30:17.472467 17390 tablet_bootstrap.cc:654] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.473585 17084 leader_election.cc:304] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:17.468953 17673 raft_consensus.cc:515] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.469250 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.475215 17193 raft_consensus.cc:2468] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:17.475962 17574 raft_consensus.cc:2749] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.478214 17234 leader_election.cc:304] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:17.467047 17308 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:17.503844 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.506966 17684 raft_consensus.cc:2804] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.507543 17684 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.507892 17684 raft_consensus.cc:3060] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.481123 17018 catalog_manager.cc:5697] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:17.491581 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:17.498955 17389 consensus_queue.cc:260] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.500092 17761 consensus_queue.cc:1059] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:17.477248 17159 leader_election.cc:343] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:17.517684 17159 leader_election.cc:343] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.468951 17760 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.521306 17620 raft_consensus.cc:493] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.521867 17620 raft_consensus.cc:515] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.522269 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:17.523949 17620 leader_election.cc:290] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:17.524065 17235 leader_election.cc:343] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.525724 17761 raft_consensus.cc:3055] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:17.526120 17761 raft_consensus.cc:740] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:17.517146 17510 raft_consensus.cc:493] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.527163 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:17.527105 17510 raft_consensus.cc:515] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.527570 17761 consensus_queue.cc:260] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.528997 17761 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:17.530966 17507 raft_consensus.cc:2804] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:17.531129 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 53015.7us
I20260709 12:30:17.531426 17507 raft_consensus.cc:493] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.531874 17507 raft_consensus.cc:3060] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.532083 17234 leader_election.cc:304] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:17.538805 17750 consensus_queue.cc:1048] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:17.539865 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58121712c0045238ff325078dcf0653" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:17.540482 17623 raft_consensus.cc:2804] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.540930 17623 raft_consensus.cc:493] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.541262 17623 raft_consensus.cc:3060] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.544299 17389 ts_tablet_manager.cc:1434] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.083s	user 0.010s	sys 0.000s
I20260709 12:30:17.537712 17159 leader_election.cc:304] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:17.545707 17389 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:17.484064 17514 raft_consensus.cc:493] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.552175 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58121712c0045238ff325078dcf0653" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.547154 17574 consensus_queue.cc:1048] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:30:17.555858 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:17.556437 17161 leader_election.cc:343] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.560020 17684 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.561285 17341 raft_consensus.cc:2393] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:17.564757 17684 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.532546 17760 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:17.560463 17389 tablet_bootstrap.cc:654] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:17.553591 17159 leader_election.cc:343] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.588796 17159 leader_election.cc:304] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:17.574395 17393 raft_consensus.cc:359] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.589689 17393 raft_consensus.cc:385] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.590076 17393 raft_consensus.cc:740] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.590835 17393 consensus_queue.cc:260] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.599295 17510 leader_election.cc:290] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:17.600669 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.604494 17673 leader_election.cc:290] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.531941 17307 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
W20260709 12:30:17.606760 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 74930.5us
I20260709 12:30:17.608106 17684 raft_consensus.cc:2749] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:30:17.608906 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:17.609867 17195 raft_consensus.cc:2393] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
W20260709 12:30:17.598294 17161 leader_election.cc:343] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.612658 17340 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:17.613417 17440 raft_consensus.cc:493] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.613986 17440 raft_consensus.cc:515] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.616219 17440 leader_election.cc:290] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.618248 17507 raft_consensus.cc:515] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.619314 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.620128 17194 raft_consensus.cc:2393] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:17.620510 17507 leader_election.cc:290] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:17.616297 17307 leader_election.cc:343] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:17.623806 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 70002.3us
I20260709 12:30:17.624106 17085 leader_election.cc:304] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:17.625144 17574 raft_consensus.cc:3060] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.625397 17393 ts_tablet_manager.cc:1434] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.196s	user 0.011s	sys 0.000s
I20260709 12:30:17.626298 17118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:17.626961 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:17.628438 17233 leader_election.cc:343] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.551373 17750 raft_consensus.cc:2804] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:30:17.627875 17159 leader_election.cc:343] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.631179 17159 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:17.623754 17760 raft_consensus.cc:2749] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.635484 17121 raft_consensus.cc:3060] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.635679 17618 raft_consensus.cc:2749] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:30:17.638829 17159 leader_election.cc:343] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.639418 17159 leader_election.cc:304] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:17.640362 17618 raft_consensus.cc:2749] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.645376 17390 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:17.645938 17390 ts_tablet_manager.cc:1403] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.301s	user 0.017s	sys 0.014s
I20260709 12:30:17.569772 17514 raft_consensus.cc:515] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.614039 17623 raft_consensus.cc:515] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.631542 17750 raft_consensus.cc:493] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:30:17.663913 17159 leader_election.cc:343] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.664515 17159 leader_election.cc:304] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:17.665319 17761 raft_consensus.cc:2749] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.668792 17391 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:17.669588 17391 ts_tablet_manager.cc:1403] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.338s	user 0.012s	sys 0.024s
I20260709 12:30:17.672163 17393 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:17.653050 17114 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.674268 17390 raft_consensus.cc:359] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.676090 17390 raft_consensus.cc:385] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.676532 17390 raft_consensus.cc:740] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.671257 17750 raft_consensus.cc:3060] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.675928 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:17.679644 17514 leader_election.cc:290] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:17.682204 17342 raft_consensus.cc:3060] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.683854 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:17.686596 17390 consensus_queue.cc:260] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.688457 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:17.702939 17235 leader_election.cc:343] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.703254 17194 raft_consensus.cc:2468] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:17.720041 17574 raft_consensus.cc:2749] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.724174 17391 raft_consensus.cc:359] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.728770 17391 raft_consensus.cc:385] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.729732 17391 raft_consensus.cc:740] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.726063 17195 raft_consensus.cc:3060] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.739357 17390 ts_tablet_manager.cc:1434] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.093s	user 0.011s	sys 0.000s
W20260709 12:30:17.743428 17307 leader_election.cc:343] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:17.747869 17307 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:17.749723 17744 raft_consensus.cc:2749] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.739859 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440c7aeff35f4f3988c3eebfef1c6acf" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:17.756675 17265 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "440c7aeff35f4f3988c3eebfef1c6acf" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:17.761475 17307 leader_election.cc:304] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:17.756975 17393 tablet_bootstrap.cc:654] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.762283 17623 leader_election.cc:290] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.745227 17391 consensus_queue.cc:260] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.767009 17391 ts_tablet_manager.cc:1434] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.005s
I20260709 12:30:17.769418 17391 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:17.771533 17234 leader_election.cc:304] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:17.775833 17196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:17.776366 17770 raft_consensus.cc:2804] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:17.776855 17770 raft_consensus.cc:493] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:17.777160 17770 raft_consensus.cc:3060] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.777354 17514 consensus_queue.cc:1048] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:17.784884 17390 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:17.795513 17391 tablet_bootstrap.cc:654] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:17.798734 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:17.837093 17195 raft_consensus.cc:2468] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:17.840548 17770 raft_consensus.cc:515] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.804210 17750 raft_consensus.cc:515] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:17.844758 17750 leader_election.cc:290] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:17.850843 17196 raft_consensus.cc:2393] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:17.858525 17389 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:17.859233 17389 ts_tablet_manager.cc:1403] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.314s	user 0.018s	sys 0.023s
I20260709 12:30:17.874572 17390 tablet_bootstrap.cc:654] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.890336 17604 raft_consensus.cc:2749] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:17.881644 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:17.878329 17234 leader_election.cc:304] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:17.915872 17393 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:17.916847 17393 ts_tablet_manager.cc:1403] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.264s	user 0.016s	sys 0.016s
I20260709 12:30:17.896000 17514 consensus_queue.cc:1048] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:30:17.929034 17391 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:17.937856 17391 ts_tablet_manager.cc:1403] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.170s	user 0.040s	sys 0.003s
I20260709 12:30:17.911346 17770 leader_election.cc:290] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:17.931268 17266 raft_consensus.cc:2393] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 6c1eb70ad94940b89118a32a370a4310 in this term.
I20260709 12:30:17.932844 17393 raft_consensus.cc:359] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.950372 17393 raft_consensus.cc:385] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.937309 17618 consensus_queue.cc:1048] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:30:17.937978 17623 raft_consensus.cc:2804] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:17.950911 17393 raft_consensus.cc:740] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
W20260709 12:30:17.960423 17085 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 63370.9us
I20260709 12:30:17.958153 17393 consensus_queue.cc:260] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.967957 17393 ts_tablet_manager.cc:1434] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:30:17.961349 17623 raft_consensus.cc:697] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:17.957648 17113 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.971498 17623 consensus_queue.cc:237] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:17.972851 17673 raft_consensus.cc:493] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:17.973706 17393 tablet_bootstrap.cc:492] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:17.973497 17673 raft_consensus.cc:515] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:17.975901 17673 leader_election.cc:290] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:17.977075 17389 raft_consensus.cc:359] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.977877 17389 raft_consensus.cc:385] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:17.978132 17389 raft_consensus.cc:740] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:17.978849 17389 consensus_queue.cc:260] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:17.980569 17393 tablet_bootstrap.cc:654] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:17.981313 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
W20260709 12:30:17.957278 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 87171.8us
I20260709 12:30:17.985988 17389 ts_tablet_manager.cc:1434] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.126s	user 0.007s	sys 0.002s
I20260709 12:30:17.987833 17390 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:17.988705 17390 ts_tablet_manager.cc:1403] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.204s	user 0.019s	sys 0.000s
I20260709 12:30:17.996438 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:17.999276 17194 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:18.000012 17194 raft_consensus.cc:3060] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:17.999867 17390 raft_consensus.cc:359] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.009156 17390 raft_consensus.cc:385] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.009620 17390 raft_consensus.cc:740] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.014175 17194 raft_consensus.cc:2468] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:18.019812 17390 consensus_queue.cc:260] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:18.028076 17233 leader_election.cc:343] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.031014 17390 ts_tablet_manager.cc:1434] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.042s	user 0.005s	sys 0.004s
I20260709 12:30:18.021661 17604 raft_consensus.cc:493] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.034219 17391 raft_consensus.cc:359] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.035091 17391 raft_consensus.cc:385] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.035414 17391 raft_consensus.cc:740] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.039338 17391 consensus_queue.cc:260] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.040630 17756 consensus_queue.cc:1048] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:18.040637 17234 leader_election.cc:304] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:18.041890 17391 ts_tablet_manager.cc:1434] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.104s	user 0.007s	sys 0.000s
I20260709 12:30:18.048596 17440 raft_consensus.cc:2804] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.044323 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:18.033207 17762 consensus_queue.cc:1048] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:30:18.072468 17390 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:18.053345 17440 raft_consensus.cc:697] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:18.077090 17440 consensus_queue.cc:237] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.044955 17604 raft_consensus.cc:515] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.109246 17390 tablet_bootstrap.cc:654] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:18.126461 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 82974us
I20260709 12:30:18.124411 17342 raft_consensus.cc:2468] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:18.150424 17117 raft_consensus.cc:3060] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.155562 17390 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:18.156430 17390 ts_tablet_manager.cc:1403] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.084s	user 0.010s	sys 0.005s
I20260709 12:30:18.160779 17604 leader_election.cc:290] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:18.168787 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:18.169927 17268 raft_consensus.cc:2468] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:18.172747 17307 leader_election.cc:304] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:18.173787 17796 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.174337 17796 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:18.176412 17788 log.cc:927] Time spent T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:30:18.176801 17604 raft_consensus.cc:2804] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.177309 17604 raft_consensus.cc:493] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.177650 17604 raft_consensus.cc:3060] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.178193 17796 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:18.179147 17389 tablet_bootstrap.cc:492] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:18.180919 17393 tablet_bootstrap.cc:492] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:18.181420 17393 ts_tablet_manager.cc:1403] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.212s	user 0.022s	sys 0.011s
I20260709 12:30:18.184078 17393 raft_consensus.cc:359] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.184778 17393 raft_consensus.cc:385] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.185024 17393 raft_consensus.cc:740] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.185657 17393 consensus_queue.cc:260] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.190817 17086 leader_election.cc:304] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:18.191896 17777 raft_consensus.cc:2804] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.197839 17393 ts_tablet_manager.cc:1434] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:30:18.200640 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:18.200686 17744 consensus_queue.cc:1048] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.201622 17266 raft_consensus.cc:2468] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 0.
I20260709 12:30:18.205628 17393 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:18.192337 17777 raft_consensus.cc:493] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.217325 17507 raft_consensus.cc:493] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.217221 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:18.224349 17266 raft_consensus.cc:2468] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:18.230155 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:18.220592 17507 raft_consensus.cc:515] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.235635 17507 leader_election.cc:290] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:18.239955 17307 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:18.240222 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:18.240901 17744 raft_consensus.cc:2804] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.241289 17744 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.241633 17744 raft_consensus.cc:3060] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.213332 17777 raft_consensus.cc:3060] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:18.211346 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 178831us
I20260709 12:30:18.251143 17391 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:18.252947 17744 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.255909 17744 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:18.260900 17117 raft_consensus.cc:2468] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:18.261003 17790 raft_consensus.cc:493] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.262036 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:18.264640 17267 raft_consensus.cc:3060] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.264514 17790 raft_consensus.cc:515] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.263789 17782 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.265663 17782 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.267812 17782 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:18.270035 17790 leader_election.cc:290] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:18.273768 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58121712c0045238ff325078dcf0653" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:18.274569 17195 raft_consensus.cc:2468] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:18.277458 17234 leader_election.cc:304] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:18.246366 17389 tablet_bootstrap.cc:654] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.278966 17507 raft_consensus.cc:2804] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.279825 17507 raft_consensus.cc:493] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.280264 17507 raft_consensus.cc:3060] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.281662 17391 tablet_bootstrap.cc:654] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.284682 17390 raft_consensus.cc:359] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.285665 17390 raft_consensus.cc:385] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.285969 17390 raft_consensus.cc:740] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.286973 17015 catalog_manager.cc:5697] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:18.287949 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58121712c0045238ff325078dcf0653" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:18.291431 17015 catalog_manager.cc:5697] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:18.287629 17390 consensus_queue.cc:260] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.288470 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:18.293071 17233 leader_election.cc:304] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:18.301164 17390 ts_tablet_manager.cc:1434] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.144s	user 0.007s	sys 0.002s
I20260709 12:30:18.302597 17159 leader_election.cc:304] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310
I20260709 12:30:18.307199 17614 raft_consensus.cc:2804] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.307706 17614 raft_consensus.cc:697] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:18.297951 17684 consensus_queue.cc:1048] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260709 12:30:18.300004 17307 leader_election.cc:343] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.308600 17614 consensus_queue.cc:237] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.304894 17393 tablet_bootstrap.cc:654] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.275143 17267 raft_consensus.cc:2468] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:18.316922 17390 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:18.299723 17440 raft_consensus.cc:2749] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.332267 17119 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:18.334153 17159 leader_election.cc:343] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:18.329694 17235 leader_election.cc:343] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.337679 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:18.342674 17307 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:18.336278 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:18.343731 17756 raft_consensus.cc:2804] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.344310 17756 raft_consensus.cc:697] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:18.345230 17756 consensus_queue.cc:237] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:18.347565 17233 leader_election.cc:343] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:18.348971 17307 leader_election.cc:343] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.356459 17604 raft_consensus.cc:515] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:18.363806 17161 leader_election.cc:343] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.421692 17389 tablet_bootstrap.cc:492] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:18.422984 17389 ts_tablet_manager.cc:1403] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.244s	user 0.012s	sys 0.020s
I20260709 12:30:18.382947 17161 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:18.428169 17507 raft_consensus.cc:515] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.440842 17620 raft_consensus.cc:2749] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:18.452615 17802 log.cc:927] Time spent T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
W20260709 12:30:18.448203 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 85227.4us
I20260709 12:30:18.456444 17604 leader_election.cc:290] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:18.457788 17390 tablet_bootstrap.cc:654] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.458151 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:18.459201 17270 raft_consensus.cc:3060] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.501248 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:18.510442 17389 raft_consensus.cc:359] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.511633 17389 raft_consensus.cc:385] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.516004 17389 raft_consensus.cc:740] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.518530 17391 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:18.519510 17391 ts_tablet_manager.cc:1403] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.269s	user 0.011s	sys 0.010s
I20260709 12:30:18.481938 17777 raft_consensus.cc:515] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:18.523101 17806 log.cc:927] Time spent T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:30:18.526952 17121 raft_consensus.cc:2393] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate 79e92a05527f4a0b98b8352cb68a1010 in this term.
I20260709 12:30:18.527654 17777 leader_election.cc:290] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:18.529021 17270 raft_consensus.cc:2468] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:18.533644 17391 raft_consensus.cc:359] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.534741 17391 raft_consensus.cc:385] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.535081 17391 raft_consensus.cc:740] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.536273 17389 consensus_queue.cc:260] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.543582 17507 leader_election.cc:290] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:18.560550 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:18.561551 17266 raft_consensus.cc:2393] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:18.575726 17307 leader_election.cc:304] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:18.581207 17391 consensus_queue.cc:260] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.587708 17389 ts_tablet_manager.cc:1434] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.164s	user 0.010s	sys 0.003s
I20260709 12:30:18.588658 17389 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:18.593705 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:18.582180 17744 raft_consensus.cc:2804] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.595851 17391 ts_tablet_manager.cc:1434] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.001s
I20260709 12:30:18.600260 17393 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:18.600925 17393 ts_tablet_manager.cc:1403] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.396s	user 0.013s	sys 0.023s
I20260709 12:30:18.601374 17390 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:18.602063 17390 ts_tablet_manager.cc:1403] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.285s	user 0.036s	sys 0.050s
I20260709 12:30:18.602200 17389 tablet_bootstrap.cc:654] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.603941 17393 raft_consensus.cc:359] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.604563 17393 raft_consensus.cc:385] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.604804 17393 raft_consensus.cc:740] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.606604 17744 raft_consensus.cc:697] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:18.608275 17744 consensus_queue.cc:237] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.613533 17393 consensus_queue.cc:260] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.621301 17390 raft_consensus.cc:359] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.631001 17390 raft_consensus.cc:385] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.631361 17390 raft_consensus.cc:740] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.632066 17390 consensus_queue.cc:260] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:18.645205 17390 ts_tablet_manager.cc:1434] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.043s	user 0.010s	sys 0.000s
I20260709 12:30:18.653153 17393 ts_tablet_manager.cc:1434] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.001s
I20260709 12:30:18.641824 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7ffe1eaf86b4aa8bfa40bb0defe1cfd" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:18.656749 17393 tablet_bootstrap.cc:492] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:18.661533 17391 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:18.664037 17390 tablet_bootstrap.cc:492] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:18.672901 17390 tablet_bootstrap.cc:654] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.676111 17818 raft_consensus.cc:493] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.676838 17818 raft_consensus.cc:515] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:18.682597 17233 consensus_peers.cc:597] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:18.686949 17818 leader_election.cc:290] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:18.708240 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58121712c0045238ff325078dcf0653" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:18.721628 17391 tablet_bootstrap.cc:654] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:18.724318 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:18.725159 17195 raft_consensus.cc:2468] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:18.726979 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e58121712c0045238ff325078dcf0653" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:18.727622 17195 raft_consensus.cc:3060] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:18.716235 17235 leader_election.cc:343] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.730021 17235 leader_election.cc:304] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:18.739192 17086 leader_election.cc:304] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:18.739706 17195 raft_consensus.cc:2468] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:18.740672 17775 raft_consensus.cc:2749] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.755276 17623 raft_consensus.cc:2749] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:18.757997 17235 consensus_peers.cc:597] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 -> Peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Couldn't send request to peer 7c0f44cba62f491392d41ca89376139a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:18.758931 17235 leader_election.cc:343] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.715376 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:18.769188 17234 leader_election.cc:304] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:18.771895 17811 raft_consensus.cc:2804] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:18.772437 17811 raft_consensus.cc:697] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:18.773353 17811 consensus_queue.cc:237] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.777048 17501 consensus_queue.cc:1048] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.769707 17623 consensus_queue.cc:1048] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:18.779193 17389 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:18.779948 17389 ts_tablet_manager.cc:1403] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.191s	user 0.030s	sys 0.024s
I20260709 12:30:18.784355 17389 raft_consensus.cc:359] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.785292 17389 raft_consensus.cc:385] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.786180 17389 raft_consensus.cc:740] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.787266 17389 consensus_queue.cc:260] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.802156 17389 ts_tablet_manager.cc:1434] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:30:18.804014 17389 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
W20260709 12:30:18.814396 17307 leader_election.cc:343] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.823781 17390 tablet_bootstrap.cc:492] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:18.824548 17390 ts_tablet_manager.cc:1403] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.165s	user 0.045s	sys 0.024s
I20260709 12:30:18.838603 17308 leader_election.cc:304] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:18.846562 17389 tablet_bootstrap.cc:654] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:18.850320 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:18.848201 17775 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.855307 17775 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.869733 17618 raft_consensus.cc:493] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.870512 17618 raft_consensus.cc:515] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.873449 17756 raft_consensus.cc:2804] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:18.873989 17756 raft_consensus.cc:493] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:18.874363 17756 raft_consensus.cc:3060] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.903808 17391 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:18.904095 17618 leader_election.cc:290] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:18.904449 17391 ts_tablet_manager.cc:1403] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.245s	user 0.016s	sys 0.018s
I20260709 12:30:18.905535 17337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62bfda640da40f9ab6e7b1da9479a5c" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:18.908236 17775 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:18.910255 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:18.911790 17344 raft_consensus.cc:2393] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:18.913585 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62bfda640da40f9ab6e7b1da9479a5c" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:18.915796 17390 raft_consensus.cc:359] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.918108 17390 raft_consensus.cc:385] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.918419 17390 raft_consensus.cc:740] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.919539 17797 raft_consensus.cc:493] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:18.919294 17390 consensus_queue.cc:260] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:18.920089 17797 raft_consensus.cc:515] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:18.921360 17390 ts_tablet_manager.cc:1434] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.096s	user 0.007s	sys 0.000s
I20260709 12:30:18.922329 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:18.923257 17193 raft_consensus.cc:2393] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:18.925567 17393 tablet_bootstrap.cc:654] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:18.927520 17161 leader_election.cc:343] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.929196 17343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90bb36eac77e43ad8fc0568e85913a36" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:18.929636 17159 leader_election.cc:343] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.930161 17159 leader_election.cc:304] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
W20260709 12:30:18.930836 17161 leader_election.cc:343] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:18.954974 17085 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:18.966420 17804 raft_consensus.cc:3060] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:18.959990 17614 consensus_queue.cc:1048] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:30:18.958636 17618 raft_consensus.cc:2749] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:18.978363 17391 raft_consensus.cc:359] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:18.979228 17391 raft_consensus.cc:385] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:18.979501 17391 raft_consensus.cc:740] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:18.981477 17760 consensus_queue.cc:1048] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:19.033196 17744 consensus_queue.cc:1048] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:19.044100 17390 tablet_bootstrap.cc:492] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:19.039321 17760 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.066228 17819 raft_consensus.cc:493] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.066880 17819 raft_consensus.cc:515] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.067827 17804 raft_consensus.cc:2749] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.069077 17797 leader_election.cc:290] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:19.070188 17118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90bb36eac77e43ad8fc0568e85913a36" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:19.062582 17760 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.082719 17811 raft_consensus.cc:493] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.083305 17811 raft_consensus.cc:515] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.083827 17760 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:19.074226 17307 consensus_peers.cc:597] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:19.074389 17507 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c75b62c746bd4d90a28483bc9d5e3c84)
I20260709 12:30:19.086642 17191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:19.078193 17785 raft_consensus.cc:493] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.089546 17785 raft_consensus.cc:515] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.069175 17819 leader_election.cc:290] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:19.086545 17507 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.091109 17391 consensus_queue.cc:260] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.098243 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:19.098802 17785 leader_election.cc:290] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:19.101927 17391 ts_tablet_manager.cc:1434] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.197s	user 0.015s	sys 0.001s
I20260709 12:30:19.103070 17391 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:19.091097 17393 tablet_bootstrap.cc:492] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:19.104787 17393 ts_tablet_manager.cc:1403] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.448s	user 0.016s	sys 0.009s
I20260709 12:30:19.106312 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b33f790d3fc4b0099ea91c45cc6fd11" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:19.107177 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:19.107720 17268 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc38664bb0494fe080bd97ce12c2481b" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:19.109027 17191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b33f790d3fc4b0099ea91c45cc6fd11" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:19.102473 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:19.097008 17811 leader_election.cc:290] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
W20260709 12:30:19.110649 17234 leader_election.cc:343] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.110649 17338 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 2.
W20260709 12:30:19.143242 17307 leader_election.cc:343] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.151036 17269 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:19.144454 17018 catalog_manager.cc:5697] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.158571 17391 tablet_bootstrap.cc:654] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.162060 17018 catalog_manager.cc:5697] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.157529 17393 raft_consensus.cc:359] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.167181 17393 raft_consensus.cc:385] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.167515 17393 raft_consensus.cc:740] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
W20260709 12:30:19.165438 17233 leader_election.cc:343] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:19.169188 17159 leader_election.cc:343] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.170094 17390 tablet_bootstrap.cc:654] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.169962 17159 leader_election.cc:304] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:19.168160 17233 leader_election.cc:304] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:19.169520 17393 consensus_queue.cc:260] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.173272 17159 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:19.173717 17307 leader_election.cc:304] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:19.178172 17501 raft_consensus.cc:2749] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.181485 17507 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:19.181715 17120 raft_consensus.cc:3060] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.182315 17793 raft_consensus.cc:2749] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.182250 17393 ts_tablet_manager.cc:1434] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.077s	user 0.007s	sys 0.001s
I20260709 12:30:19.177973 17797 raft_consensus.cc:2804] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:19.184257 17393 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:19.184182 17797 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.184768 17797 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:19.189584 17191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:19.176017 17796 raft_consensus.cc:2749] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.204413 17756 raft_consensus.cc:515] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:19.206737 17393 tablet_bootstrap.cc:654] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.206933 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:19.207912 17338 raft_consensus.cc:2393] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 in current term 2: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
W20260709 12:30:19.211007 17826 log.cc:927] Time spent T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: Append to log took a long time: real 0.064s	user 0.000s	sys 0.002s
I20260709 12:30:19.215610 17343 raft_consensus.cc:3060] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.221711 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:19.222651 17269 raft_consensus.cc:2468] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:19.229175 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:19.239187 17389 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:19.241395 17389 ts_tablet_manager.cc:1403] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.438s	user 0.028s	sys 0.028s
W20260709 12:30:19.243286 17307 leader_election.cc:343] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.245959 17307 leader_election.cc:304] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
W20260709 12:30:19.245014 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 45981.2us
I20260709 12:30:19.253206 17744 raft_consensus.cc:2804] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.261574 17744 raft_consensus.cc:493] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.261986 17744 raft_consensus.cc:3060] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.257535 17756 leader_election.cc:290] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:19.289791 17192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:19.297870 17192 raft_consensus.cc:3060] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.300155 17795 consensus_queue.cc:1048] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.056s
I20260709 12:30:19.303668 17514 consensus_queue.cc:1048] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:19.311710 17015 catalog_manager.cc:5697] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } }
I20260709 12:30:19.313468 17015 catalog_manager.cc:5697] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } }
I20260709 12:30:19.315480 17015 catalog_manager.cc:5697] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.285845 17797 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.319566 17797 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:19.320281 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:19.320915 17270 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:30:19.323175 17191 raft_consensus.cc:2368] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 for earlier term 2. Current term is 3.
I20260709 12:30:19.328481 17270 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 3.
I20260709 12:30:19.331111 17159 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:19.332536 17234 leader_election.cc:400] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Vote denied by peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307) with higher term. Message: Invalid argument: T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 for earlier term 2. Current term is 3.
I20260709 12:30:19.332705 17797 raft_consensus.cc:2804] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:19.333565 17797 raft_consensus.cc:697] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:19.334650 17797 consensus_queue.cc:237] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.344372 17234 leader_election.cc:403] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:30:19.377876 17501 raft_consensus.cc:2749] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307) with higher term. Message: Invalid argument: T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c1eb70ad94940b89118a32a370a4310 for earlier term 2. Current term is 3.
I20260709 12:30:19.370985 17818 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.373513 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:19.383073 17389 raft_consensus.cc:359] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.385986 17389 raft_consensus.cc:385] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.386349 17389 raft_consensus.cc:740] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.387707 17344 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:30:19.387262 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 53877.1us
I20260709 12:30:19.385043 17818 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:19.390918 17390 tablet_bootstrap.cc:492] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:19.411541 17390 ts_tablet_manager.cc:1403] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.368s	user 0.016s	sys 0.017s
I20260709 12:30:19.399838 17811 consensus_queue.cc:1048] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:30:19.388168 17389 consensus_queue.cc:260] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.416389 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:19.391623 17192 raft_consensus.cc:2468] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:19.406360 17018 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60932 (request call id 65) took 1365 ms. Trace:
I20260709 12:30:19.417711 17018 rpcz_store.cc:276] 0709 12:30:18.040612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:30:18.040769 (+   157us) service_pool.cc:224] Handling call
0709 12:30:19.406256 (+1365487us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:30:19.313819 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:30:19.314387 (+   568us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:30:19.314425 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:30:19.314442 (+    17us) tablet.cc:662] Decoding operations
0709 12:30:19.314854 (+   412us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:30:19.314931 (+    77us) write_op.cc:641] Partition lock acquired for write op
0709 12:30:19.314957 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:30:19.315273 (+   316us) tablet.cc:701] Row locks acquired
0709 12:30:19.315290 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:30:19.315433 (+   143us) write_op.cc:270] Start()
0709 12:30:19.315545 (+   112us) write_op.cc:276] Timestamp: P: 1783600219315392 usec, L: 0
0709 12:30:19.315570 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:30:19.316229 (+   659us) log.cc:844] Serialized 910 byte log entry
0709 12:30:19.320314 (+  4085us) op_driver.cc:464] REPLICATION: finished
0709 12:30:19.320729 (+   415us) write_op.cc:301] APPLY: starting
0709 12:30:19.320798 (+    69us) tablet.cc:1367] starting BulkCheckPresence
0709 12:30:19.320915 (+   117us) tablet.cc:1370] finished BulkCheckPresence
0709 12:30:19.321013 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:30:19.321307 (+   294us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:30:19.321457 (+   150us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:30:19.321511 (+    54us) write_op.cc:312] APPLY: finished
0709 12:30:19.321699 (+   188us) log.cc:844] Serialized 28 byte log entry
0709 12:30:19.321858 (+   159us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:30:19.321943 (+    85us) write_op.cc:454] Released schema lock
0709 12:30:19.321985 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:30:19.322286 (+   301us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:30:19.322310 (+    24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5441,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":228,"apply.run_cpu_time_us":1533,"apply.run_wall_time_us":1532,"num_ops":2,"prepare.queue_time_us":117111,"prepare.run_cpu_time_us":2781,"prepare.run_wall_time_us":21777,"replication_time_us":4615,"thread_start_us":73789,"threads_started":1}]]}
I20260709 12:30:19.404820 17818 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:19.409798 17744 raft_consensus.cc:515] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.426421 17390 raft_consensus.cc:359] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.427160 17390 raft_consensus.cc:385] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.427443 17390 raft_consensus.cc:740] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.427570 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:19.428229 17266 raft_consensus.cc:3060] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.428223 17390 consensus_queue.cc:260] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.428781 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:19.429876 17193 raft_consensus.cc:2468] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:19.432098 17389 ts_tablet_manager.cc:1434] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.190s	user 0.006s	sys 0.001s
I20260709 12:30:19.433221 17389 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84: Bootstrap starting.
I20260709 12:30:19.435073 17393 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:19.435822 17393 ts_tablet_manager.cc:1403] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.252s	user 0.019s	sys 0.047s
I20260709 12:30:19.417219 17112 raft_consensus.cc:2468] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:19.440511 17307 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:19.441723 17604 raft_consensus.cc:2804] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:19.442621 17604 raft_consensus.cc:493] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.443089 17604 raft_consensus.cc:3060] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:19.447391 17744 leader_election.cc:290] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:19.451985 17393 raft_consensus.cc:359] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.453084 17393 raft_consensus.cc:385] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.453402 17393 raft_consensus.cc:740] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.454778 17019 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60920 (request call id 75) took 1072 ms. Trace:
I20260709 12:30:19.455672 17308 leader_election.cc:304] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:19.459741 17341 raft_consensus.cc:3060] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.460840 17344 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 3.
I20260709 12:30:19.455308 17019 rpcz_store.cc:276] 0709 12:30:18.383317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:30:18.383531 (+   214us) service_pool.cc:224] Handling call
0709 12:30:19.454703 (+1071172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10310,"reported_tablets":5}
I20260709 12:30:19.467680 17393 consensus_queue.cc:260] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.467059 17744 raft_consensus.cc:2804] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.471549 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:19.484135 17770 raft_consensus.cc:493] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.491731 17770 raft_consensus.cc:515] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
W20260709 12:30:19.493216 17307 leader_election.cc:343] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.485446 17390 ts_tablet_manager.cc:1434] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.074s	user 0.007s	sys 0.000s
I20260709 12:30:19.496516 17192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62bfda640da40f9ab6e7b1da9479a5c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:19.497328 17192 raft_consensus.cc:2468] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:19.498519 17811 raft_consensus.cc:493] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.499115 17811 raft_consensus.cc:515] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.496673 17390 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:19.502720 17391 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:19.499845 17744 raft_consensus.cc:697] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:19.517025 17391 ts_tablet_manager.cc:1403] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.414s	user 0.020s	sys 0.015s
I20260709 12:30:19.517737 17744 consensus_queue.cc:237] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:19.505555 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:19.499467 17770 leader_election.cc:290] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:19.519999 17393 ts_tablet_manager.cc:1434] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.084s	user 0.010s	sys 0.000s
I20260709 12:30:19.542721 17393 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
I20260709 12:30:19.503949 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62bfda640da40f9ab6e7b1da9479a5c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:19.523458 17389 tablet_bootstrap.cc:654] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.504202 17811 leader_election.cc:290] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:19.504005 17234 leader_election.cc:304] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:19.529731 17604 raft_consensus.cc:515] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:19.533247 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:19.534312 17233 leader_election.cc:343] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.535109 17790 consensus_queue.cc:1048] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:19.553606 17393 tablet_bootstrap.cc:654] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.486469 17266 raft_consensus.cc:2468] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:19.561214 17790 raft_consensus.cc:2804] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.561681 17790 raft_consensus.cc:493] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.562023 17790 raft_consensus.cc:3060] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:19.559343 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 55416us
I20260709 12:30:19.544883 17391 raft_consensus.cc:359] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.577606 17391 raft_consensus.cc:385] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.578330 17307 leader_election.cc:304] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:19.580150 17391 raft_consensus.cc:740] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.576190 17019 catalog_manager.cc:5697] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.582894 17391 consensus_queue.cc:260] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.582572 17818 raft_consensus.cc:2804] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.593683 17391 ts_tablet_manager.cc:1434] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.076s	user 0.008s	sys 0.000s
I20260709 12:30:19.599787 17391 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:19.595080 17235 leader_election.cc:343] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.603235 17235 leader_election.cc:304] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
W20260709 12:30:19.604683 17235 leader_election.cc:343] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.606541 17389 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84: No bootstrap required, opened a new log
I20260709 12:30:19.607132 17389 ts_tablet_manager.cc:1403] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent bootstrapping tablet: real 0.174s	user 0.027s	sys 0.002s
I20260709 12:30:19.608044 17390 tablet_bootstrap.cc:654] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.598091 17818 raft_consensus.cc:697] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:19.610379 17389 raft_consensus.cc:359] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.612689 17389 raft_consensus.cc:385] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.612979 17389 raft_consensus.cc:740] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.597324 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:19.616014 17389 consensus_queue.cc:260] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.611253 17818 consensus_queue.cc:237] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.629309 17389 ts_tablet_manager.cc:1434] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.005s
I20260709 12:30:19.611243 17770 raft_consensus.cc:2749] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.619201 17116 raft_consensus.cc:3060] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:19.647173 17393 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:19.647852 17393 ts_tablet_manager.cc:1403] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.105s	user 0.014s	sys 0.015s
I20260709 12:30:19.651118 17574 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.651074 17393 raft_consensus.cc:359] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.651927 17393 raft_consensus.cc:385] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.651746 17574 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:19.652233 17393 raft_consensus.cc:740] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.653113 17393 consensus_queue.cc:260] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:19.654934 17604 leader_election.cc:290] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:19.663249 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:19.666390 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:19.671518 17084 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:19.672966 17673 raft_consensus.cc:2749] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:19.677516 17191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "458688757584439eb735f41c9b0e9eaa" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:19.678184 17191 raft_consensus.cc:3060] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:19.654083 17574 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:19.701066 17393 ts_tablet_manager.cc:1434] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.003s
I20260709 12:30:19.703020 17393 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310: Bootstrap starting.
W20260709 12:30:19.708844 17848 log.cc:927] Time spent T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:30:19.729018 17790 raft_consensus.cc:515] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.749187 17393 tablet_bootstrap.cc:654] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.761996 17192 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62bfda640da40f9ab6e7b1da9479a5c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:19.762805 17192 raft_consensus.cc:3060] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.778795 17790 leader_election.cc:290] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:19.783723 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d62bfda640da40f9ab6e7b1da9479a5c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:19.783669 17192 raft_consensus.cc:2468] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:19.789472 17234 leader_election.cc:304] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:19.790395 17790 raft_consensus.cc:2804] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.791471 17790 raft_consensus.cc:697] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
W20260709 12:30:19.791615 17235 leader_election.cc:343] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.792555 17790 consensus_queue.cc:237] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:19.797207 17116 raft_consensus.cc:2468] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 2.
I20260709 12:30:19.814210 17307 leader_election.cc:304] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:19.828631 17819 raft_consensus.cc:2804] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:19.836946 17819 raft_consensus.cc:697] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:19.928238 17390 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:19.928954 17390 ts_tablet_manager.cc:1403] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.433s	user 0.020s	sys 0.019s
I20260709 12:30:19.899012 17191 raft_consensus.cc:2468] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 2.
I20260709 12:30:19.933480 17019 catalog_manager.cc:5697] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.986865 17342 raft_consensus.cc:1275] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Refusing update from remote peer c75b62c746bd4d90a28483bc9d5e3c84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:30:19.990985 17852 log.cc:927] Time spent T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:30:20.001623 17859 raft_consensus.cc:493] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.002331 17859 raft_consensus.cc:515] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:20.003818 17853 log.cc:927] Time spent T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: Append to log took a long time: real 0.122s	user 0.000s	sys 0.005s
I20260709 12:30:20.004841 17859 leader_election.cc:290] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:19.978649 17794 raft_consensus.cc:493] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.008847 17393 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310: No bootstrap required, opened a new log
I20260709 12:30:19.971377 17819 consensus_queue.cc:237] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.014572 17390 raft_consensus.cc:359] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.017459 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:20.019399 17782 consensus_queue.cc:1048] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.024449 17391 tablet_bootstrap.cc:654] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:20.041141 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:20.041308 17393 ts_tablet_manager.cc:1403] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310: Time spent bootstrapping tablet: real 0.339s	user 0.102s	sys 0.088s
I20260709 12:30:20.042708 17390 raft_consensus.cc:385] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.047160 17390 raft_consensus.cc:740] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.039230 17760 raft_consensus.cc:493] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.048764 17390 consensus_queue.cc:260] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.057339 17390 ts_tablet_manager.cc:1434] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.128s	user 0.006s	sys 0.002s
I20260709 12:30:20.053128 17838 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.066998 17390 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:20.044173 17794 raft_consensus.cc:515] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:20.070148 17393 raft_consensus.cc:359] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.070901 17393 raft_consensus.cc:385] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.071161 17393 raft_consensus.cc:740] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.071532 17267 raft_consensus.cc:1275] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Refusing update from remote peer c75b62c746bd4d90a28483bc9d5e3c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:30:20.053310 17760 raft_consensus.cc:515] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:20.100150 17190 raft_consensus.cc:2468] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 0.
I20260709 12:30:20.076160 17838 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.130920 17862 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.135842 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:20.135840 17862 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.133527 17121 raft_consensus.cc:3060] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.140867 17390 tablet_bootstrap.cc:654] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.141228 17393 consensus_queue.cc:260] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.143761 17393 ts_tablet_manager.cc:1434] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.002s
I20260709 12:30:20.150841 17017 catalog_manager.cc:5697] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.154091 17838 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:20.196506 17234 leader_election.cc:304] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:20.202040 17811 raft_consensus.cc:2804] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.215552 17811 raft_consensus.cc:493] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.215937 17811 raft_consensus.cc:3060] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.216463 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:20.217262 17339 raft_consensus.cc:2468] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:20.219735 17862 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:20.227203 17233 leader_election.cc:343] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.209890 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:20.227125 17791 consensus_queue.cc:1048] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:30:20.249017 17391 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:20.250927 17391 ts_tablet_manager.cc:1403] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.654s	user 0.022s	sys 0.007s
I20260709 12:30:20.271590 17760 leader_election.cc:290] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:20.272353 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:20.273195 17794 leader_election.cc:290] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:20.273180 17190 raft_consensus.cc:2468] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:20.279649 17871 consensus_queue.cc:1048] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:30:20.279642 17307 leader_election.cc:343] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.287643 17811 raft_consensus.cc:515] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:20.295169 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b33f790d3fc4b0099ea91c45cc6fd11" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:20.295895 17269 raft_consensus.cc:2468] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:20.298221 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:20.299083 17269 raft_consensus.cc:2393] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 6c1eb70ad94940b89118a32a370a4310 in this term.
W20260709 12:30:20.299566 17235 consensus_peers.cc:597] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 -> Peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Couldn't send request to peer 7c0f44cba62f491392d41ca89376139a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:20.291319 17391 raft_consensus.cc:359] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.292574 17161 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
W20260709 12:30:20.301614 17307 consensus_peers.cc:597] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:20.302019 17794 raft_consensus.cc:2804] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.302470 17794 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.302876 17794 raft_consensus.cc:3060] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.310863 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:20.311556 17118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:20.312767 17120 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b33f790d3fc4b0099ea91c45cc6fd11" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:20.313570 17307 consensus_peers.cc:597] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:20.298772 17811 leader_election.cc:290] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:20.318447 17308 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:20.319806 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:20.321254 17193 raft_consensus.cc:3060] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.319280 17391 raft_consensus.cc:385] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.325357 17391 raft_consensus.cc:740] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.326099 17391 consensus_queue.cc:260] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.329213 17794 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.331971 17391 ts_tablet_manager.cc:1434] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.000s
I20260709 12:30:20.333094 17391 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:20.341856 17390 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:20.342587 17390 ts_tablet_manager.cc:1403] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.276s	user 0.011s	sys 0.008s
I20260709 12:30:20.355671 17159 leader_election.cc:304] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
W20260709 12:30:20.360471 17159 leader_election.cc:343] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:20.365540 17159 leader_election.cc:343] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.353616 17744 raft_consensus.cc:2804] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.365939 17159 leader_election.cc:304] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:20.366096 17744 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.366514 17744 raft_consensus.cc:3060] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.371745 17868 raft_consensus.cc:2804] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.372238 17868 raft_consensus.cc:493] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.374589 17118 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:20.372555 17868 raft_consensus.cc:3060] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.378188 17507 consensus_queue.cc:1048] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.388019 17015 catalog_manager.cc:5697] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } }
I20260709 12:30:20.389834 17390 raft_consensus.cc:359] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.390722 17390 raft_consensus.cc:385] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:30:20.390841 17233 leader_election.cc:343] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.381313 17744 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.393443 17744 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:20.393501 17390 raft_consensus.cc:740] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.388722 17794 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:20.394816 17159 leader_election.cc:343] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.413160 17337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:20.414412 17337 raft_consensus.cc:2393] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c75b62c746bd4d90a28483bc9d5e3c84 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:20.419337 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:20.421072 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
W20260709 12:30:20.422300 17159 leader_election.cc:343] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.432638 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:20.433439 17195 raft_consensus.cc:2393] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7c0f44cba62f491392d41ca89376139a in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:20.438624 17193 raft_consensus.cc:2468] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:20.439317 17871 raft_consensus.cc:2749] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.450753 17017 catalog_manager.cc:5697] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.394322 17390 consensus_queue.cc:260] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.453699 17017 catalog_manager.cc:5697] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 2, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 2 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.463434 17234 leader_election.cc:304] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
W20260709 12:30:20.465010 17307 leader_election.cc:343] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:20.465013 17308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 52703.5us
I20260709 12:30:20.475437 17390 ts_tablet_manager.cc:1434] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.133s	user 0.007s	sys 0.000s
I20260709 12:30:20.476317 17390 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:20.481986 17623 raft_consensus.cc:2804] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.504594 17623 raft_consensus.cc:697] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:20.485388 17795 consensus_queue.cc:1048] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 12:30:20.497633 17790 raft_consensus.cc:493] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.532675 17790 raft_consensus.cc:515] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.542960 17868 raft_consensus.cc:515] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:20.546537 17868 leader_election.cc:290] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:20.505911 17623 consensus_queue.cc:237] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:20.580193 17866 raft_consensus.cc:493] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.587145 17866 raft_consensus.cc:515] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.589763 17866 leader_election.cc:290] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:20.548097 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b33f790d3fc4b0099ea91c45cc6fd11" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:20.610628 17266 raft_consensus.cc:3060] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:20.612859 17308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 135278us
I20260709 12:30:20.614562 17308 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:20.615291 17604 raft_consensus.cc:2749] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.619524 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:20.620164 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:20.627575 17235 leader_election.cc:343] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.636202 17194 raft_consensus.cc:1275] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 7c0f44cba62f491392d41ca89376139a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:20.642174 17604 consensus_queue.cc:1048] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:20.650625 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 51708.8us
I20260709 12:30:20.657579 17115 raft_consensus.cc:3060] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.657891 17120 raft_consensus.cc:1275] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Refusing update from remote peer 7c0f44cba62f491392d41ca89376139a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:20.664395 17266 raft_consensus.cc:2468] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:20.561915 17390 tablet_bootstrap.cc:654] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.548027 17790 leader_election.cc:290] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:20.596422 17391 tablet_bootstrap.cc:654] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.604929 17121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:20.605727 17116 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b33f790d3fc4b0099ea91c45cc6fd11" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:20.607429 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ba13b6455354f64a81d0be356cf1982" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:20.680747 17085 leader_election.cc:304] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
W20260709 12:30:20.681170 17233 leader_election.cc:343] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.681806 17233 leader_election.cc:304] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:20.670073 17159 leader_election.cc:304] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
W20260709 12:30:20.685668 17159 leader_election.cc:343] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.692232 17604 consensus_queue.cc:1048] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.693215 17623 raft_consensus.cc:2749] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.692122 17868 raft_consensus.cc:2804] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.706453 17391 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:20.707197 17391 ts_tablet_manager.cc:1403] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.374s	user 0.009s	sys 0.010s
I20260709 12:30:20.708789 17604 raft_consensus.cc:493] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.709373 17604 raft_consensus.cc:515] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.721807 17390 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:20.722766 17390 ts_tablet_manager.cc:1403] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.247s	user 0.019s	sys 0.000s
I20260709 12:30:20.709024 17868 raft_consensus.cc:697] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:20.759574 17018 catalog_manager.cc:5697] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:20.768821 17868 consensus_queue.cc:237] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:20.775701 17391 raft_consensus.cc:359] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.795379 17391 raft_consensus.cc:385] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.795708 17391 raft_consensus.cc:740] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.801817 17390 raft_consensus.cc:359] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.802716 17390 raft_consensus.cc:385] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.803042 17390 raft_consensus.cc:740] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.803838 17390 consensus_queue.cc:260] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.806532 17390 ts_tablet_manager.cc:1434] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.002s
I20260709 12:30:20.807607 17390 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:20.809017 17604 leader_election.cc:290] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:20.814800 17390 tablet_bootstrap.cc:654] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.817442 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:20.818722 17269 raft_consensus.cc:2468] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
W20260709 12:30:20.824417 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 407529us
I20260709 12:30:20.796340 17391 consensus_queue.cc:260] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.838725 17776 raft_consensus.cc:2749] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.845116 17307 leader_election.cc:304] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:20.848570 17819 raft_consensus.cc:2804] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.849012 17819 raft_consensus.cc:493] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.849345 17819 raft_consensus.cc:3060] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.856009 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
W20260709 12:30:20.857870 17307 leader_election.cc:343] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.867915 17819 raft_consensus.cc:515] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.871001 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:20.872052 17269 raft_consensus.cc:3060] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.875298 17391 ts_tablet_manager.cc:1434] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.168s	user 0.004s	sys 0.004s
I20260709 12:30:20.876578 17391 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:20.877905 17117 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
W20260709 12:30:20.879463 17307 leader_election.cc:343] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.870208 17819 leader_election.cc:290] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:20.901507 17391 tablet_bootstrap.cc:654] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.912811 17017 catalog_manager.cc:5697] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.872830 17161 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:20.938057 17390 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:20.943836 17390 ts_tablet_manager.cc:1403] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.136s	user 0.016s	sys 0.043s
I20260709 12:30:20.956559 17871 raft_consensus.cc:2749] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.958534 17871 raft_consensus.cc:493] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.959239 17391 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:20.959815 17391 ts_tablet_manager.cc:1403] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.083s	user 0.026s	sys 0.006s
I20260709 12:30:20.959028 17871 raft_consensus.cc:515] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.966327 17871 leader_election.cc:290] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:20.937386 17269 raft_consensus.cc:2468] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:20.967973 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90bb36eac77e43ad8fc0568e85913a36" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:20.970296 17391 raft_consensus.cc:359] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:20.971203 17391 raft_consensus.cc:385] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.971439 17307 leader_election.cc:304] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
W20260709 12:30:20.972664 17159 leader_election.cc:343] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.971536 17391 raft_consensus.cc:740] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
W20260709 12:30:20.971377 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 126466us
I20260709 12:30:20.975097 17744 raft_consensus.cc:2804] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.975734 17744 raft_consensus.cc:697] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:20.976562 17744 consensus_queue.cc:237] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.978268 17390 raft_consensus.cc:359] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.979592 17390 raft_consensus.cc:385] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.979945 17390 raft_consensus.cc:740] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.980772 17390 consensus_queue.cc:260] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.981920 17776 raft_consensus.cc:493] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.982784 17776 raft_consensus.cc:515] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:20.984704 17390 ts_tablet_manager.cc:1434] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:30:20.987759 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:20.990334 17776 leader_election.cc:290] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:20.993472 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90bb36eac77e43ad8fc0568e85913a36" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:20.994277 17342 raft_consensus.cc:2468] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 0.
I20260709 12:30:21.008608 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a100ea0ba8a74d14bdfc772d6059f39c" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:21.010632 17084 leader_election.cc:304] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:21.011922 17776 raft_consensus.cc:2749] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.017247 17790 consensus_queue.cc:1048] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.019595 17161 leader_election.cc:304] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:20.973811 17391 consensus_queue.cc:260] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:21.021476 17871 raft_consensus.cc:2804] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:21.021920 17871 raft_consensus.cc:493] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:21.022284 17871 raft_consensus.cc:3060] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.028374 17391 ts_tablet_manager.cc:1434] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.000s
I20260709 12:30:21.029606 17391 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:21.022306 17390 tablet_bootstrap.cc:492] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a: Bootstrap starting.
I20260709 12:30:21.036680 17871 raft_consensus.cc:515] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:21.038631 17390 tablet_bootstrap.cc:654] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.039659 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90bb36eac77e43ad8fc0568e85913a36" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:21.040413 17338 raft_consensus.cc:3060] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.042753 17871 leader_election.cc:290] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:21.049185 17233 consensus_peers.cc:597] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:21.051400 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "90bb36eac77e43ad8fc0568e85913a36" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:21.065008 17391 tablet_bootstrap.cc:654] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:21.066781 17159 leader_election.cc:343] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.093152 17338 raft_consensus.cc:2468] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:21.094893 17161 leader_election.cc:304] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:21.096192 17871 raft_consensus.cc:2804] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:21.096743 17871 raft_consensus.cc:697] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:21.097658 17871 consensus_queue.cc:237] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:21.108011 17015 catalog_manager.cc:5697] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:21.121824 17391 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:21.122604 17391 ts_tablet_manager.cc:1403] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.093s	user 0.014s	sys 0.015s
I20260709 12:30:21.123886 17390 tablet_bootstrap.cc:492] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a: No bootstrap required, opened a new log
I20260709 12:30:21.124639 17390 ts_tablet_manager.cc:1403] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a: Time spent bootstrapping tablet: real 0.103s	user 0.015s	sys 0.002s
I20260709 12:30:21.125928 17391 raft_consensus.cc:359] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.126753 17391 raft_consensus.cc:385] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.127053 17391 raft_consensus.cc:740] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.128264 17390 raft_consensus.cc:359] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:21.129024 17390 raft_consensus.cc:385] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.129344 17390 raft_consensus.cc:740] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.130187 17390 consensus_queue.cc:260] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:21.132974 17390 ts_tablet_manager.cc:1434] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:30:21.138810 17015 catalog_manager.cc:5697] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 0 to 1, leader changed from <none> to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:21.144421 17391 consensus_queue.cc:260] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.145493 17776 raft_consensus.cc:493] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.146219 17776 raft_consensus.cc:515] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.148288 17391 ts_tablet_manager.cc:1434] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:30:21.150426 17776 leader_election.cc:290] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:21.152684 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:21.152891 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51de8b9a56eb439e9c25052d3d9587f9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:21.153625 17269 raft_consensus.cc:2393] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:21.155313 17084 leader_election.cc:304] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:21.156165 17866 raft_consensus.cc:3060] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.165400 17391 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:21.172142 17879 raft_consensus.cc:493] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.172788 17879 raft_consensus.cc:515] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.164530 17866 raft_consensus.cc:2749] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.175078 17879 leader_election.cc:290] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:21.177105 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:21.177873 17269 raft_consensus.cc:2468] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 0.
I20260709 12:30:21.181636 17391 tablet_bootstrap.cc:654] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.184916 17307 leader_election.cc:304] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:21.186158 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:21.192569 17604 raft_consensus.cc:2804] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:21.193032 17604 raft_consensus.cc:493] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:21.193346 17604 raft_consensus.cc:3060] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:21.192571 17307 leader_election.cc:343] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:21.198567 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:21.205030 17604 raft_consensus.cc:515] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.209123 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:21.210029 17269 raft_consensus.cc:3060] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.220172 17604 leader_election.cc:290] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:21.258322 17112 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:21.266829 17391 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:21.267295 17269 raft_consensus.cc:2468] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:21.267520 17391 ts_tablet_manager.cc:1403] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.112s	user 0.021s	sys 0.034s
I20260709 12:30:21.269290 17307 leader_election.cc:304] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:21.270701 17879 raft_consensus.cc:2804] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:21.271219 17879 raft_consensus.cc:697] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:21.272086 17879 consensus_queue.cc:237] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.272625 17391 raft_consensus.cc:359] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:21.273780 17391 raft_consensus.cc:385] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.274128 17391 raft_consensus.cc:740] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.274997 17391 consensus_queue.cc:260] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:21.276281 17307 leader_election.cc:343] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:21.283563 17391 ts_tablet_manager.cc:1434] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.002s
I20260709 12:30:21.284744 17391 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:21.291785 17391 tablet_bootstrap.cc:654] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.292462 17115 raft_consensus.cc:3060] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.300818 17391 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:21.301674 17391 ts_tablet_manager.cc:1403] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.002s
I20260709 12:30:21.310580 17018 catalog_manager.cc:5697] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 1 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:21.315017 17391 raft_consensus.cc:359] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:21.315799 17391 raft_consensus.cc:385] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.316078 17391 raft_consensus.cc:740] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.316939 17776 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.317566 17776 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:21.319782 17776 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:21.320147 17391 consensus_queue.cc:260] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:21.321447 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:21.322345 17344 raft_consensus.cc:2393] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:21.324869 17391 ts_tablet_manager.cc:1434] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.005s
I20260709 12:30:21.326004 17391 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:21.324937 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:21.328783 17190 raft_consensus.cc:2393] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate c75b62c746bd4d90a28483bc9d5e3c84 in this term.
I20260709 12:30:21.338289 17085 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:21.339094 17391 tablet_bootstrap.cc:654] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.339303 17776 raft_consensus.cc:3060] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.347131 17344 raft_consensus.cc:3060] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.362514 17117 raft_consensus.cc:3060] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.373208 17871 consensus_queue.cc:1048] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.375012 17776 raft_consensus.cc:2749] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.390110 17391 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:21.390738 17391 ts_tablet_manager.cc:1403] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.065s	user 0.018s	sys 0.018s
I20260709 12:30:21.397392 17391 raft_consensus.cc:359] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:21.399405 17391 raft_consensus.cc:385] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.400081 17391 raft_consensus.cc:740] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.401422 17391 consensus_queue.cc:260] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:21.404963 17391 ts_tablet_manager.cc:1434] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:30:21.407395 17391 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:21.390470 17871 consensus_queue.cc:1048] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:30:21.422132 17391 tablet_bootstrap.cc:654] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:21.469782 17391 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: No bootstrap required, opened a new log
I20260709 12:30:21.470377 17391 ts_tablet_manager.cc:1403] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.004s
I20260709 12:30:21.480432 17391 raft_consensus.cc:359] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.481177 17391 raft_consensus.cc:385] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:21.481463 17391 raft_consensus.cc:740] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:21.482132 17391 consensus_queue.cc:260] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.484828 17391 ts_tablet_manager.cc:1434] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:30:21.505764 17116 raft_consensus.cc:3060] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.516587 17117 raft_consensus.cc:3060] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.528537 17878 consensus_queue.cc:1048] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.548290 17866 raft_consensus.cc:493] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:21.548930 17866 raft_consensus.cc:515] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:21.551347 17866 leader_election.cc:290] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:21.559414 17270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:21.566742 17879 consensus_queue.cc:1048] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.570958 17270 raft_consensus.cc:2393] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 79e92a05527f4a0b98b8352cb68a1010 in current term 1: Already voted for candidate 7c0f44cba62f491392d41ca89376139a in this term.
I20260709 12:30:21.560725 17760 consensus_queue.cc:1048] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.553484 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d14098588e6e4421bbf71e16dea54da0" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:21.576912 17086 leader_election.cc:304] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:21.577916 17673 raft_consensus.cc:3060] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:21.604624 17744 consensus_queue.cc:1048] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:21.621898 17673 raft_consensus.cc:2749] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:21.994082 17819 consensus_queue.cc:1048] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:30:22.009946 17879 consensus_queue.cc:1048] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260709 12:30:22.052369 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 45490.5us
I20260709 12:30:22.244553 17879 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:22.245434 17879 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:22.248615 17879 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:22.249441 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:22.249509 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010" is_pre_election: true
I20260709 12:30:22.250599 17193 raft_consensus.cc:2468] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:22.251089 17115 raft_consensus.cc:2468] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:22.252295 17308 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:22.253507 17879 raft_consensus.cc:2804] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:30:22.253517 17066 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:22.254078 17879 raft_consensus.cc:493] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:22.254479 17879 raft_consensus.cc:3060] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:22.262010 17879 raft_consensus.cc:515] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:22.264915 17879 leader_election.cc:290] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:22.264906 17115 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
I20260709 12:30:22.265693 17115 raft_consensus.cc:3060] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:22.265664 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec4250a950114da1bce4b46571dc97cc" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:22.266726 17193 raft_consensus.cc:3060] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:22.274858 17115 raft_consensus.cc:2468] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 2.
I20260709 12:30:22.276229 17307 leader_election.cc:304] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:22.276952 17879 raft_consensus.cc:2804] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:22.277576 17879 raft_consensus.cc:697] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:22.278584 17193 raft_consensus.cc:2468] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 2.
I20260709 12:30:22.278936 17879 consensus_queue.cc:237] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:22.291432 17015 catalog_manager.cc:5697] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 0 to 2, leader changed from <none> to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 2 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:22.781394 17115 raft_consensus.cc:1275] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Refusing update from remote peer 7c0f44cba62f491392d41ca89376139a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:22.783095 17604 consensus_queue.cc:1048] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:22.794426 17193 raft_consensus.cc:1275] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 7c0f44cba62f491392d41ca89376139a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:22.798076 17879 consensus_queue.cc:1048] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:24.252416  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
W20260709 12:30:24.259357 17159 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
W20260709 12:30:24.265992 17159 consensus_peers.cc:597] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.284281 17233 consensus_peers.cc:597] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.306998 17159 consensus_peers.cc:597] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.316254 17233 consensus_peers.cc:597] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.320156 17233 consensus_peers.cc:597] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.342031 17307 consensus_peers.cc:597] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.343223  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:24.345973  5137 tablet_replica.cc:333] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.347438  5137 raft_consensus.cc:2243] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.348027  5137 raft_consensus.cc:2272] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.352408  5137 tablet_replica.cc:333] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.353484  5137 raft_consensus.cc:2243] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.354108  5137 raft_consensus.cc:2272] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.357321  5137 tablet_replica.cc:333] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.358191  5137 raft_consensus.cc:2243] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.358794  5137 raft_consensus.cc:2272] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.361969  5137 tablet_replica.cc:333] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.362885  5137 raft_consensus.cc:2243] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.363562  5137 raft_consensus.cc:2272] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.366854  5137 tablet_replica.cc:333] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.367743  5137 raft_consensus.cc:2243] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.368409  5137 raft_consensus.cc:2272] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.371662  5137 tablet_replica.cc:333] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.372633  5137 raft_consensus.cc:2243] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.373323  5137 raft_consensus.cc:2272] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.376801  5137 tablet_replica.cc:333] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.377786  5137 raft_consensus.cc:2243] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.378444  5137 raft_consensus.cc:2272] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.381697  5137 tablet_replica.cc:333] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.382728  5137 raft_consensus.cc:2243] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.383322  5137 raft_consensus.cc:2272] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.386439  5137 tablet_replica.cc:333] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.387339  5137 raft_consensus.cc:2243] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.388019  5137 raft_consensus.cc:2272] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.391137  5137 tablet_replica.cc:333] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.392133  5137 raft_consensus.cc:2243] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:24.393795 17159 consensus_peers.cc:597] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.394064  5137 raft_consensus.cc:2272] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:24.395684 17159 consensus_peers.cc:597] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.397284  5137 tablet_replica.cc:333] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.397922  5137 raft_consensus.cc:2243] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.398334  5137 raft_consensus.cc:2272] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.401425  5137 tablet_replica.cc:333] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.402233  5137 raft_consensus.cc:2243] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.402937  5137 raft_consensus.cc:2272] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:24.405035 17159 consensus_peers.cc:597] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.405961  5137 tablet_replica.cc:333] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.406791  5137 raft_consensus.cc:2243] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.407949  5137 raft_consensus.cc:2272] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.410192  5137 tablet_replica.cc:333] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.410846  5137 raft_consensus.cc:2243] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.411360  5137 raft_consensus.cc:2272] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.413688  5137 tablet_replica.cc:333] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.414304  5137 raft_consensus.cc:2243] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.414862  5137 raft_consensus.cc:2272] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.417718  5137 tablet_replica.cc:333] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.418541  5137 raft_consensus.cc:2243] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.419386  5137 raft_consensus.cc:2272] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.422780  5137 tablet_replica.cc:333] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.423620  5137 raft_consensus.cc:2243] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.424365  5137 raft_consensus.cc:2272] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.428591  5137 tablet_replica.cc:333] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.429540  5137 raft_consensus.cc:2243] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.430054  5137 raft_consensus.cc:2272] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.432581  5137 tablet_replica.cc:333] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.433122  5137 raft_consensus.cc:2243] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.433503  5137 raft_consensus.cc:2272] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.435931  5137 tablet_replica.cc:333] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.436546  5137 raft_consensus.cc:2243] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.436983  5137 raft_consensus.cc:2272] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.439242  5137 tablet_replica.cc:333] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.439857  5137 raft_consensus.cc:2243] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.441033  5137 raft_consensus.cc:2272] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.443222  5137 tablet_replica.cc:333] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.443729  5137 raft_consensus.cc:2243] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.444571  5137 raft_consensus.cc:2272] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.447120  5137 tablet_replica.cc:333] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.447696  5137 raft_consensus.cc:2243] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.448617  5137 raft_consensus.cc:2272] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:24.449928 17307 consensus_peers.cc:597] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.451757  5137 tablet_replica.cc:333] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.452462  5137 raft_consensus.cc:2243] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.453111  5137 raft_consensus.cc:2272] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.455695  5137 tablet_replica.cc:333] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.456369  5137 raft_consensus.cc:2243] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.456876  5137 raft_consensus.cc:2272] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:24.456985 17307 consensus_peers.cc:597] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.459233  5137 tablet_replica.cc:333] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.459892  5137 raft_consensus.cc:2243] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.460670  5137 raft_consensus.cc:2272] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.462922  5137 tablet_replica.cc:333] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.463457  5137 raft_consensus.cc:2243] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.463873  5137 raft_consensus.cc:2272] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.466426  5137 tablet_replica.cc:333] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.467377  5137 raft_consensus.cc:2243] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.468520  5137 raft_consensus.cc:2272] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.471071  5137 tablet_replica.cc:333] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.471647  5137 raft_consensus.cc:2243] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.472095  5137 raft_consensus.cc:2272] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.474081  5137 tablet_replica.cc:333] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.474735  5137 raft_consensus.cc:2243] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.475631  5137 raft_consensus.cc:2272] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.478797  5137 tablet_replica.cc:333] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.479483  5137 raft_consensus.cc:2243] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.480080  5137 raft_consensus.cc:2272] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:24.481011 17233 consensus_peers.cc:597] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.483093  5137 tablet_replica.cc:333] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.484040  5137 raft_consensus.cc:2243] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.484638  5137 raft_consensus.cc:2272] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.487881  5137 tablet_replica.cc:333] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
W20260709 12:30:24.488230 17159 consensus_peers.cc:597] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.488591  5137 raft_consensus.cc:2243] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.489214  5137 raft_consensus.cc:2272] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.491842  5137 tablet_replica.cc:333] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.492565  5137 raft_consensus.cc:2243] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.493124  5137 raft_consensus.cc:2272] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.496057  5137 tablet_replica.cc:333] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
W20260709 12:30:24.496274 17307 consensus_peers.cc:597] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.496790  5137 raft_consensus.cc:2243] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.497426  5137 raft_consensus.cc:2272] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.500669  5137 tablet_replica.cc:333] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.501361  5137 raft_consensus.cc:2243] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.502377  5137 raft_consensus.cc:2272] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.505352  5137 tablet_replica.cc:333] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.506124  5137 raft_consensus.cc:2243] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.506714  5137 raft_consensus.cc:2272] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.509727  5137 tablet_replica.cc:333] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.510434  5137 raft_consensus.cc:2243] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.510874  5137 raft_consensus.cc:2272] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.512977  5137 tablet_replica.cc:333] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.513566  5137 raft_consensus.cc:2243] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.513983  5137 raft_consensus.cc:2272] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.516331  5137 tablet_replica.cc:333] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.516906  5137 raft_consensus.cc:2243] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.517524  5137 raft_consensus.cc:2272] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.520056  5137 tablet_replica.cc:333] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.520627  5137 raft_consensus.cc:2243] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.521196  5137 raft_consensus.cc:2272] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.523561  5137 tablet_replica.cc:333] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.524080  5137 raft_consensus.cc:2243] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.524596  5137 raft_consensus.cc:2272] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.526866  5137 tablet_replica.cc:333] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.527329  5137 raft_consensus.cc:2243] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.527694  5137 raft_consensus.cc:2272] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.529932  5137 tablet_replica.cc:333] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.530427  5137 raft_consensus.cc:2243] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:24.530907  5137 raft_consensus.cc:2272] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:24.533725  5137 tablet_replica.cc:333] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: stopping tablet replica
I20260709 12:30:24.534332  5137 raft_consensus.cc:2243] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:24.535277  5137 raft_consensus.cc:2272] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:24.570457 17159 consensus_peers.cc:597] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.572300 17307 consensus_peers.cc:597] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.588877 17307 consensus_peers.cc:597] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.590543 17307 consensus_peers.cc:597] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.596980 17307 consensus_peers.cc:597] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.602880 17307 consensus_peers.cc:597] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.613673 17307 connection.cc:573] client connection to 127.5.4.65:34625 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:30:24.617259 17159 consensus_peers.cc:597] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.621223 17307 consensus_peers.cc:597] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.624187 17307 consensus_peers.cc:597] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.625022 17307 consensus_peers.cc:597] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.628854  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
W20260709 12:30:24.637073 17307 consensus_peers.cc:597] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.638630 17233 consensus_peers.cc:597] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.655759 17159 consensus_peers.cc:597] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.664721 17233 consensus_peers.cc:597] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.673556 17307 consensus_peers.cc:597] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.677992 17159 consensus_peers.cc:597] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.699875 17159 consensus_peers.cc:597] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.707424 17307 consensus_peers.cc:597] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:24.721428 17307 consensus_peers.cc:597] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.754360  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:24.767568 17946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:24.776260 17947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:24.776621 17949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:24.782579  5137 server_base.cc:1061] running on GCE node
W20260709 12:30:24.790205 17307 consensus_peers.cc:597] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.791735  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:24.792184  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:24.792436  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600224792416 us; error 0 us; skew 500 ppm
I20260709 12:30:24.793275  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:24.799157  5137 webserver.cc:533] Webserver started at http://127.5.4.65:33081/ using document root <none> and password file <none>
I20260709 12:30:24.799886  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:24.800158  5137 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:30:24.806667  5137 fs_manager.cc:714] Time spent opening directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:30:24.841382 17959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:24.845309  5137 fs_manager.cc:730] Time spent opening block manager: real 0.012s	user 0.000s	sys 0.002s
I20260709 12:30:24.845851  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "79e92a05527f4a0b98b8352cb68a1010"
format_stamp: "Formatted at 2026-07-09 12:30:08 on dist-test-slave-lg8f"
I20260709 12:30:24.846555  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:30:24.888353 17159 consensus_peers.cc:597] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:24.900503  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:24.902510  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:24.905171  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:24.912279 17968 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:30:25.217185  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:30:25.217510  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.307s	user 0.008s	sys 0.000s
I20260709 12:30:25.217854  5137 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:30:25.225133 17968 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.247340 17968 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.248238 17968 tablet_bootstrap.cc:492] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.248936 17968 ts_tablet_manager.cc:1403] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.024s	user 0.008s	sys 0.008s
I20260709 12:30:25.251971 17968 raft_consensus.cc:359] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.252780 17968 raft_consensus.cc:740] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.253568 17968 consensus_queue.cc:260] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.258667 17968 ts_tablet_manager.cc:1434] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.009s	user 0.012s	sys 0.000s
I20260709 12:30:25.259647 17968 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.284324 17968 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.285409 17968 tablet_bootstrap.cc:492] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.286105 17968 ts_tablet_manager.cc:1403] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.027s	user 0.010s	sys 0.008s
I20260709 12:30:25.288594 17968 raft_consensus.cc:359] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.289294 17968 raft_consensus.cc:740] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.290210 17968 consensus_queue.cc:260] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.292352 17968 ts_tablet_manager.cc:1434] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:25.294441 17968 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.317369 17968 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.318413 17968 tablet_bootstrap.cc:492] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.319150 17968 ts_tablet_manager.cc:1403] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.003s
I20260709 12:30:25.321564 17968 raft_consensus.cc:359] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.322296 17968 raft_consensus.cc:740] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.323002 17968 consensus_queue.cc:260] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.325331 17968 ts_tablet_manager.cc:1434] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:30:25.326151 17968 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.341451 17968 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.342523 17968 tablet_bootstrap.cc:492] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.343205 17968 ts_tablet_manager.cc:1403] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.017s	user 0.012s	sys 0.004s
I20260709 12:30:25.345624 17968 raft_consensus.cc:359] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.346345 17968 raft_consensus.cc:740] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.347210 17968 consensus_queue.cc:260] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.352336 17968 ts_tablet_manager.cc:1434] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:30:25.353103 17968 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.368197 17952 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.368430 17968 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.368973 17952 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.369663 17968 tablet_bootstrap.cc:492] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.370384 17968 ts_tablet_manager.cc:1403] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:30:25.370812 17879 raft_consensus.cc:493] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.371155 17952 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:25.371416 17879 raft_consensus.cc:515] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.373293 17879 leader_election.cc:290] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:25.372973 17968 raft_consensus.cc:359] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.373761 17968 raft_consensus.cc:740] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.374475 17968 consensus_queue.cc:260] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.376574 17968 ts_tablet_manager.cc:1434] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:25.377393 17968 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:25.378226 17307 leader_election.cc:336] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
W20260709 12:30:25.378979 17307 leader_election.cc:336] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.383391 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:25.385866 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:25.385357 17308 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:25.389107 17308 leader_election.cc:304] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:25.389863 17952 raft_consensus.cc:2749] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.390537 17879 raft_consensus.cc:2749] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.393370 17968 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.394418 17968 tablet_bootstrap.cc:492] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.395110 17968 ts_tablet_manager.cc:1403] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.018s	user 0.011s	sys 0.004s
I20260709 12:30:25.397588 17968 raft_consensus.cc:359] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.398407 17968 raft_consensus.cc:740] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.399118 17968 consensus_queue.cc:260] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.400977 17968 ts_tablet_manager.cc:1434] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:30:25.401753 17968 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.416743 17968 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.417765 17968 tablet_bootstrap.cc:492] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.418435 17968 ts_tablet_manager.cc:1403] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.004s
I20260709 12:30:25.420900 17968 raft_consensus.cc:359] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.421630 17968 raft_consensus.cc:740] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.422569 17968 consensus_queue.cc:260] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.424774 17968 ts_tablet_manager.cc:1434] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:25.425595 17968 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.488467 17968 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.489482 17968 tablet_bootstrap.cc:492] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.490154 17968 ts_tablet_manager.cc:1403] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.065s	user 0.032s	sys 0.019s
I20260709 12:30:25.492669 17968 raft_consensus.cc:359] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.493410 17968 raft_consensus.cc:740] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.494169 17968 consensus_queue.cc:260] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.496110 17968 ts_tablet_manager.cc:1434] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:25.496927 17968 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.509063  5137 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:30:25.509647  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.292s	user 0.250s	sys 0.014s
I20260709 12:30:25.527325 17968 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.528369 17968 tablet_bootstrap.cc:492] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.529188 17968 ts_tablet_manager.cc:1403] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.015s
I20260709 12:30:25.533267 17968 raft_consensus.cc:359] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.534858 17968 raft_consensus.cc:740] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.535864 17968 consensus_queue.cc:260] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.538816 17968 ts_tablet_manager.cc:1434] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 12:30:25.540009 17968 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.551942 17960 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.552944 17960 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.556622 17968 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.557934 17968 tablet_bootstrap.cc:492] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.559271 17968 ts_tablet_manager.cc:1403] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 12:30:25.558943 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:25.559964 17339 raft_consensus.cc:2468] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:25.560166 17960 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.562227 17161 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.562455 17968 raft_consensus.cc:359] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.563251 17968 raft_consensus.cc:740] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
W20260709 12:30:25.563634 17159 leader_election.cc:336] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.564117 17968 consensus_queue.cc:260] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.565653 17960 raft_consensus.cc:2804] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.566212 17960 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.566955 17968 ts_tablet_manager.cc:1434] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:25.566776 17960 raft_consensus.cc:3060] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.567991 17968 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.584087 17960 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.584810 17952 raft_consensus.cc:493] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.585680 17952 raft_consensus.cc:515] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.589709 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:25.596707 17952 leader_election.cc:290] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:25.600299 17307 leader_election.cc:336] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.600996 17307 leader_election.cc:304] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.601956 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:25.602712 17339 raft_consensus.cc:3060] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.602403 17952 raft_consensus.cc:2749] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.603794 17960 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.605345 17960 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.606313 17960 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:25.607156 17159 leader_election.cc:336] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.610262 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:25.613709 17339 raft_consensus.cc:2468] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 2.
I20260709 12:30:25.614691 17960 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.619781 17161 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.622566 17968 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:30:25.622635 17159 leader_election.cc:336] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.622792 17952 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.623306 17159 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:25.623764 17968 tablet_bootstrap.cc:492] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.624348 17937 raft_consensus.cc:2749] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.624624 17968 ts_tablet_manager.cc:1403] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.057s	user 0.030s	sys 0.016s
I20260709 12:30:25.623556 17952 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.626775 17952 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.624586 17960 raft_consensus.cc:2804] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:25.628053 17968 raft_consensus.cc:359] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.628744 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:25.629083 17968 raft_consensus.cc:740] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.629281 17956 raft_consensus.cc:493] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.629878 17956 raft_consensus.cc:515] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.630118 17968 consensus_queue.cc:260] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.632419 17956 leader_election.cc:290] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:25.632529 17968 ts_tablet_manager.cc:1434] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:30:25.633513 17968 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.633698 17960 raft_consensus.cc:697] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:25.633867 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:25.637737 17960 consensus_queue.cc:237] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:25.640182 17159 leader_election.cc:336] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.642283 17159 leader_election.cc:304] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c75b62c746bd4d90a28483bc9d5e3c84; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.643144 17937 raft_consensus.cc:2749] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.649161 17936 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.649791 17936 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.652667 17938 raft_consensus.cc:493] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.653251 17938 raft_consensus.cc:515] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:25.656208 17307 leader_election.cc:336] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.656741 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:25.656816 17307 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.659673 17952 raft_consensus.cc:2749] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.662863 17938 leader_election.cc:290] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.659209 17936 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:25.665365 17956 raft_consensus.cc:493] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.665925 17956 raft_consensus.cc:515] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.668130 17956 leader_election.cc:290] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.672549 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:25.675721 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:25.678150 17233 leader_election.cc:336] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.678731 17339 raft_consensus.cc:2468] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:25.678912 17233 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:25.681005 17936 raft_consensus.cc:2749] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:25.681416 17161 leader_election.cc:304] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.683444 17956 raft_consensus.cc:2804] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.683837 17956 raft_consensus.cc:493] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.684188 17956 raft_consensus.cc:3060] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.694546 17957 raft_consensus.cc:493] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
W20260709 12:30:25.694758 17159 leader_election.cc:336] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.695253 17957 raft_consensus.cc:515] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.696831 17968 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.697901 17968 tablet_bootstrap.cc:492] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.699112 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:25.701345 17952 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.701967 17952 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.704028 17952 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.707228 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:25.708343 17233 leader_election.cc:336] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.708770 17233 leader_election.cc:304] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
W20260709 12:30:25.709318 17233 leader_election.cc:336] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.709568 18007 raft_consensus.cc:2749] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.709679 17233 leader_election.cc:304] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:25.710599 17957 leader_election.cc:290] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:25.711397 18007 raft_consensus.cc:2749] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.714111 17956 raft_consensus.cc:515] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.713871 17015 catalog_manager.cc:5697] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:25.716864 17956 leader_election.cc:290] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.717733 17968 ts_tablet_manager.cc:1403] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.084s	user 0.025s	sys 0.017s
I20260709 12:30:25.717976 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55a24caeb8124f77a6a9d21f8bed1831" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:25.720218 17339 raft_consensus.cc:3060] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:25.721800 17307 leader_election.cc:336] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.722384 17307 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.723295 17969 raft_consensus.cc:2749] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:25.723492 17968 raft_consensus.cc:359] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.724298 17968 raft_consensus.cc:740] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.725172 17968 consensus_queue.cc:260] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.729959 17968 ts_tablet_manager.cc:1434] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:30:25.731354 17968 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.741374 17879 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.749176 17879 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.751272 17879 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
W20260709 12:30:25.753082 17159 leader_election.cc:336] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.729784 17339 raft_consensus.cc:2468] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 2.
I20260709 12:30:25.760710 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:25.762673 17193 raft_consensus.cc:2468] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:25.764009 17308 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.765091 17879 raft_consensus.cc:2804] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.765275 17161 leader_election.cc:304] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.764757 17968 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:25.766928 17968 tablet_bootstrap.cc:492] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:25.766963 17937 raft_consensus.cc:2804] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:25.767823 17968 ts_tablet_manager.cc:1403] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.037s	user 0.027s	sys 0.004s
I20260709 12:30:25.767855 17937 raft_consensus.cc:697] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:25.769580 17937 consensus_queue.cc:237] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.771109 17968 raft_consensus.cc:359] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:25.772042 17307 leader_election.cc:336] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.772007 17968 raft_consensus.cc:740] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:25.775234 17879 raft_consensus.cc:493] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.775797 17952 raft_consensus.cc:493] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.775959 17879 raft_consensus.cc:3060] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.776759 17952 raft_consensus.cc:515] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.788578 17952 leader_election.cc:290] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.794157 18007 raft_consensus.cc:493] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.794795 18007 raft_consensus.cc:515] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.797220 18007 leader_election.cc:290] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:25.798774 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:25.799484 17193 raft_consensus.cc:2468] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:25.800822 17234 leader_election.cc:304] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.802554 17957 raft_consensus.cc:2804] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.803016 17957 raft_consensus.cc:493] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.803401 17957 raft_consensus.cc:3060] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.804011 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:25.804708 17269 raft_consensus.cc:2468] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 1.
I20260709 12:30:25.801991 17968 consensus_queue.cc:260] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.810396 17879 raft_consensus.cc:515] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.811259 17968 ts_tablet_manager.cc:1434] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.000s
I20260709 12:30:25.814126 17307 leader_election.cc:304] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
W20260709 12:30:25.811340 17233 leader_election.cc:336] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.812275 17957 raft_consensus.cc:515] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.819456 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44f0f66af5d044fdb87c6320a84cbf8e" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:25.820115 17193 raft_consensus.cc:3060] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.821082 17952 raft_consensus.cc:2804] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.821520 17952 raft_consensus.cc:493] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.821895 17952 raft_consensus.cc:3060] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.816202 17879 leader_election.cc:290] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:25.826776 17957 leader_election.cc:290] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
W20260709 12:30:25.829308 17233 leader_election.cc:336] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.816057 17968 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:25.839563 17193 raft_consensus.cc:2468] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 2.
I20260709 12:30:25.841116 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ed704c39774ac88e3d32548f26b242" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:25.842022 17190 raft_consensus.cc:3060] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:25.840366 17307 leader_election.cc:336] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.830451 17952 raft_consensus.cc:515] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.846244 17937 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.846940 17937 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:25.848505 17307 leader_election.cc:336] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.848516 17234 leader_election.cc:304] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.849300 17937 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:25.850277 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44c804e5f1864c799cd5d1d49734db98" candidate_uuid: "7c0f44cba62f491392d41ca89376139a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:25.850941 17269 raft_consensus.cc:3060] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.849088 17952 leader_election.cc:290] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.854305 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:25.855027 17267 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 2.
I20260709 12:30:25.855188 18008 raft_consensus.cc:2804] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:25.855719 18008 raft_consensus.cc:697] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:25.856520 18008 consensus_queue.cc:237] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.856962 17159 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.859596 17960 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:25.860034 17960 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.860463 17960 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:25.862568 17269 raft_consensus.cc:2468] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 2.
I20260709 12:30:25.865012 18007 raft_consensus.cc:493] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.865767 18007 raft_consensus.cc:515] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.868525 18007 leader_election.cc:290] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.869017 17307 leader_election.cc:304] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:25.870204 17952 raft_consensus.cc:2804] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:25.870903 17952 raft_consensus.cc:697] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:25.872114 17952 consensus_queue.cc:237] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:25.873440 17159 leader_election.cc:336] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
W20260709 12:30:25.874590 17233 leader_election.cc:336] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.877810 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:25.878525 17344 raft_consensus.cc:2468] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:25.881876 17235 leader_election.cc:304] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.884068 17190 raft_consensus.cc:2468] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c0f44cba62f491392d41ca89376139a in term 2.
I20260709 12:30:25.884858 17957 raft_consensus.cc:2804] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.889256 17957 raft_consensus.cc:493] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.889881 17957 raft_consensus.cc:3060] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.889936 17308 leader_election.cc:304] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.892397 17879 raft_consensus.cc:2804] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:25.893054 17879 raft_consensus.cc:697] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Becoming Leader. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:25.894121 17879 consensus_queue.cc:237] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:25.886165 17960 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:25.886626 17307 leader_election.cc:336] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.924602 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:25.925436 17267 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:25.929975 17936 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.930564 17936 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.931687 17956 raft_consensus.cc:493] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.932420 17956 raft_consensus.cc:515] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.932953 17936 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:25.943181 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:25.943501 17233 leader_election.cc:336] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:25.943979 17339 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:25.944185 17956 leader_election.cc:290] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:25.946717 17235 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:25.948823 17267 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 3.
I20260709 12:30:25.969144 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:25.961745 18008 raft_consensus.cc:2804] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:25.973163 17267 raft_consensus.cc:2468] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 1.
I20260709 12:30:25.973268 18008 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:25.975749 17159 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.914353 17960 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:25.974097 18008 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:25.980547 17017 catalog_manager.cc:5697] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:25.983042 17956 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:25.983567 17956 raft_consensus.cc:697] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:25.984455 17956 consensus_queue.cc:237] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:25.985594 18008 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:25.976938 17159 leader_election.cc:304] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:25.996255 17018 catalog_manager.cc:5697] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 2 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:25.991194 17960 raft_consensus.cc:2804] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:26.016719 17960 raft_consensus.cc:493] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:26.039757 17960 raft_consensus.cc:3060] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:26.019840 17957 raft_consensus.cc:515] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.039476 18007 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:26.058770 18007 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.061247 18007 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:26.069150 18022 log.cc:927] Time spent T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:30:26.070577 17233 leader_election.cc:336] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:26.073133 18008 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:26.079984 17233 leader_election.cc:336] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:26.089126 17339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:26.089927 17339 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:26.090976 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:26.091948 17968 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.091660 17338 raft_consensus.cc:2468] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 1.
I20260709 12:30:26.093083 17968 tablet_bootstrap.cc:492] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.093452 17235 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:26.092422 17960 raft_consensus.cc:515] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.095567 18007 raft_consensus.cc:2804] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:26.096014 18007 raft_consensus.cc:493] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:26.096122 17960 leader_election.cc:290] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: Requested vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:26.096438 18007 raft_consensus.cc:3060] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:26.093952 17968 ts_tablet_manager.cc:1403] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.278s	user 0.063s	sys 0.105s
I20260709 12:30:26.105438 17968 raft_consensus.cc:359] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.112143 17339 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 2.
I20260709 12:30:26.113431 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1ca4d4100a154e058fd6b5525d3e280f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:26.113667 17957 leader_election.cc:290] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:26.113653 17018 catalog_manager.cc:5697] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to 7c0f44cba62f491392d41ca89376139a (127.5.4.68). New cstate: current_term: 2 leader_uuid: "7c0f44cba62f491392d41ca89376139a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:26.118393 17338 raft_consensus.cc:3060] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:26.106238 17968 raft_consensus.cc:740] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.121810 17235 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
W20260709 12:30:26.120616 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 143712us
W20260709 12:30:26.123909 17159 leader_election.cc:336] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
W20260709 12:30:26.124701 17159 leader_election.cc:336] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:26.126289 17936 raft_consensus.cc:2804] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:26.121040 17968 consensus_queue.cc:260] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.127358 17936 raft_consensus.cc:697] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:26.128399 17936 consensus_queue.cc:237] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.130751 17968 ts_tablet_manager.cc:1434] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:30:26.131707 17968 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:26.135084 17338 raft_consensus.cc:2468] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 2.
W20260709 12:30:26.133222 17233 leader_election.cc:336] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:26.139312 18007 raft_consensus.cc:515] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.141574 18007 leader_election.cc:290] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:26.153585 17235 leader_election.cc:304] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:26.155009 18008 raft_consensus.cc:2804] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:26.155544 18008 raft_consensus.cc:697] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:26.156515 18008 consensus_queue.cc:237] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.158301 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd33570a634744e89bbf929e83441909" candidate_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:26.159305 17267 raft_consensus.cc:3060] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:26.160166 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e9b921f32d04442e89c1b7ea73df020f" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:26.160946 17341 raft_consensus.cc:3060] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:26.162963 17338 raft_consensus.cc:1275] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer c75b62c746bd4d90a28483bc9d5e3c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:26.164803 17233 leader_election.cc:336] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:26.170488 17341 raft_consensus.cc:2468] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 2.
I20260709 12:30:26.171702 17267 raft_consensus.cc:2468] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c75b62c746bd4d90a28483bc9d5e3c84 in term 2.
I20260709 12:30:26.186396 17235 leader_election.cc:304] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:26.189352 17936 raft_consensus.cc:2804] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:26.189100 17956 consensus_queue.cc:1048] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:26.189906 17936 raft_consensus.cc:697] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:26.182327 17018 catalog_manager.cc:5697] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:26.195123 17936 consensus_queue.cc:237] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.197532 17968 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.198716 17968 tablet_bootstrap.cc:492] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.199393 17968 ts_tablet_manager.cc:1403] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.068s	user 0.026s	sys 0.006s
I20260709 12:30:26.206919 17337 raft_consensus.cc:1275] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer c75b62c746bd4d90a28483bc9d5e3c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.208833 17159 leader_election.cc:304] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:26.209699 17960 raft_consensus.cc:2804] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:26.202039 17968 raft_consensus.cc:359] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.211103 17968 raft_consensus.cc:740] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.212357 17968 consensus_queue.cc:260] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.213330 17956 consensus_queue.cc:1048] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:26.214666 17960 raft_consensus.cc:697] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 LEADER]: Becoming Leader. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:26.214810 17968 ts_tablet_manager.cc:1434] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
W20260709 12:30:26.215307 17159 leader_election.cc:336] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:26.215948 17968 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:26.215814 17960 consensus_queue.cc:237] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.224560 17017 catalog_manager.cc:5697] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:26.229019 17017 catalog_manager.cc:5697] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 2 to 3, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:26.254297 17159 consensus_peers.cc:597] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.255019 17159 consensus_peers.cc:597] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.261411 17190 raft_consensus.cc:1275] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 6c1eb70ad94940b89118a32a370a4310: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.263316 18008 consensus_queue.cc:1048] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:26.246335 17018 catalog_manager.cc:5697] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:26.270654 17018 catalog_manager.cc:5697] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:26.294836 17018 catalog_manager.cc:5697] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 reported cstate change: term changed from 1 to 2, leader changed from 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65) to c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:26.315990 17968 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.317137 17968 tablet_bootstrap.cc:492] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.317850 17968 ts_tablet_manager.cc:1403] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.102s	user 0.015s	sys 0.004s
I20260709 12:30:26.320513 17968 raft_consensus.cc:359] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.321161 17968 raft_consensus.cc:740] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.321918 17968 consensus_queue.cc:260] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.333055 17968 ts_tablet_manager.cc:1434] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:30:26.334187 17968 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:26.355490 17307 consensus_peers.cc:597] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.362715 17233 consensus_peers.cc:597] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.377249 17968 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.378271 17968 tablet_bootstrap.cc:492] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.378929 17968 ts_tablet_manager.cc:1403] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.045s	user 0.018s	sys 0.004s
I20260709 12:30:26.381682 17968 raft_consensus.cc:359] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.391999 17267 raft_consensus.cc:1275] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Refusing update from remote peer c75b62c746bd4d90a28483bc9d5e3c84: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:30:26.395579 17956 consensus_queue.cc:1048] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:26.401345 17269 raft_consensus.cc:1275] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 7c0f44cba62f491392d41ca89376139a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.403479 17969 consensus_queue.cc:1048] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:26.382397 17968 raft_consensus.cc:740] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.410480 17968 consensus_queue.cc:260] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.412792 17968 ts_tablet_manager.cc:1434] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 12:30:26.413606 17968 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:26.506705 17307 consensus_peers.cc:597] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.519701 17190 raft_consensus.cc:1275] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 7c0f44cba62f491392d41ca89376139a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.520206 17968 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.521332 17968 tablet_bootstrap.cc:492] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.522080 17968 ts_tablet_manager.cc:1403] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.109s	user 0.026s	sys 0.064s
I20260709 12:30:26.525029 17968 raft_consensus.cc:359] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.525784 17968 raft_consensus.cc:740] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.526698 17968 consensus_queue.cc:260] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.529301 17968 ts_tablet_manager.cc:1434] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.010s	sys 0.000s
I20260709 12:30:26.530181 17968 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:26.529807 17952 consensus_queue.cc:1048] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:26.543151 17307 consensus_peers.cc:597] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:26.539959 17159 consensus_peers.cc:597] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.566357 17971 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.567226 17971 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.576200 17968 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.577306 17968 tablet_bootstrap.cc:492] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.578908 17968 ts_tablet_manager.cc:1403] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.049s	user 0.015s	sys 0.010s
I20260709 12:30:26.580178 17971 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:26.581609 17968 raft_consensus.cc:359] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:26.581473 17159 consensus_peers.cc:597] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.604753 17968 raft_consensus.cc:740] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.607899 17968 consensus_queue.cc:260] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.610154 17968 ts_tablet_manager.cc:1434] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.031s	user 0.004s	sys 0.000s
I20260709 12:30:26.611212 17968 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:26.654477 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:26.664816 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:26.666121 17233 consensus_peers.cc:597] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.667939 17962 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:26.669129 17968 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.669247 17971 raft_consensus.cc:2749] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:26.673228 17233 consensus_peers.cc:597] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.672801 17269 raft_consensus.cc:1275] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer c75b62c746bd4d90a28483bc9d5e3c84: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.674772 17968 tablet_bootstrap.cc:492] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.675019 17956 consensus_queue.cc:1048] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:30:26.679129 17968 ts_tablet_manager.cc:1403] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.068s	user 0.018s	sys 0.014s
I20260709 12:30:26.682245 17968 raft_consensus.cc:359] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.683388 17968 raft_consensus.cc:740] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.685290 17968 consensus_queue.cc:260] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.688242 17971 raft_consensus.cc:493] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.688732 17337 raft_consensus.cc:1275] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 6c1eb70ad94940b89118a32a370a4310: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.689254 17968 ts_tablet_manager.cc:1434] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:30:26.690303 17968 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:26.688766 17971 raft_consensus.cc:515] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.691094 17936 consensus_queue.cc:1048] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:26.693977 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:26.697965 17971 leader_election.cc:290] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
W20260709 12:30:26.702224 17233 consensus_peers.cc:597] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:26.702966 17233 consensus_peers.cc:597] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.706887 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:26.709286 17962 leader_election.cc:304] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:26.714720 17971 raft_consensus.cc:2749] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:26.727583 17971 raft_consensus.cc:493] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.728169 17971 raft_consensus.cc:515] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.731321 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:26.734519 17971 leader_election.cc:290] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:26.740067 17338 raft_consensus.cc:1275] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 6c1eb70ad94940b89118a32a370a4310: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:26.745774 18008 consensus_queue.cc:1048] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:26.752715 17266 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "058259c8483642bba034c3bd3f5ba90e" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:26.771377 17971 raft_consensus.cc:493] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.772285 17971 raft_consensus.cc:515] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
W20260709 12:30:26.778715 17159 consensus_peers.cc:597] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:26.780767 17971 leader_election.cc:290] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:26.781961 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:26.788914 17962 leader_election.cc:304] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:26.790330 17971 raft_consensus.cc:2749] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:26.809387 17159 consensus_peers.cc:597] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.815196 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ecf98a8bc05f404f8fb4149773b1c384" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:26.815775 17341 raft_consensus.cc:1275] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 6c1eb70ad94940b89118a32a370a4310: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:26.818778 17307 consensus_peers.cc:597] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.819032 17964 leader_election.cc:304] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:26.820830 17971 raft_consensus.cc:2749] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:26.829978 18008 consensus_queue.cc:1048] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:26.832244 17971 raft_consensus.cc:493] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.832818 17971 raft_consensus.cc:515] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.835083 17971 leader_election.cc:290] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:26.836676 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
W20260709 12:30:26.845252 17159 consensus_peers.cc:597] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:26.854722 17233 consensus_peers.cc:597] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.856592 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52b7abb65c454a8f9dca603b58f2cc0a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:26.867262 17962 leader_election.cc:304] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:26.868170 17971 raft_consensus.cc:2749] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:26.876819 17971 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.877343 17971 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:26.879726 17971 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:26.882737 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:26.889789 17159 consensus_peers.cc:597] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:26.895358 17233 consensus_peers.cc:597] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.901367 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:26.910501 17962 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:26.911890 17971 raft_consensus.cc:2749] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:26.918664 17307 consensus_peers.cc:597] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:26.929208  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:34625
I20260709 12:30:26.931035 17971 raft_consensus.cc:493] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.931692 17971 raft_consensus.cc:515] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.926288 18091 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:34625 every 8 connection(s)
I20260709 12:30:26.934376 17971 leader_election.cc:290] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:26.935118 17971 raft_consensus.cc:493] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:26.935679 17971 raft_consensus.cc:515] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:26.938019 17971 leader_election.cc:290] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:26.950943 17968 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:26.952354 17968 tablet_bootstrap.cc:492] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:26.953210 17968 ts_tablet_manager.cc:1403] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.263s	user 0.069s	sys 0.073s
I20260709 12:30:26.956530 17968 raft_consensus.cc:359] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.957401 17968 raft_consensus.cc:740] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:26.966807 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:26.976135 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:26.958204 17968 consensus_queue.cc:260] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:26.979947 17968 ts_tablet_manager.cc:1434] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.002s
I20260709 12:30:26.978255 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9af739be394434c9d30f15785e5e692" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:26.994676 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f61a82e0eef4a83ad047d6dd2ca75f8" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:27.011482 17964 leader_election.cc:304] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:27.020615 17964 leader_election.cc:304] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 7c0f44cba62f491392d41ca89376139a, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:27.024797 17971 raft_consensus.cc:2749] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:27.026753 18092 raft_consensus.cc:2749] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:27.028069 17968 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:27.037488 17307 consensus_peers.cc:597] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.039276 18025 raft_consensus.cc:3060] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.043182 18033 raft_consensus.cc:3060] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:27.044363 17159 consensus_peers.cc:597] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:27.049768 17159 consensus_peers.cc:597] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.051710 18094 heartbeater.cc:344] Connected to a master server at 127.5.4.126:38383
I20260709 12:30:27.052233 18094 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:27.053093 18094 heartbeater.cc:507] Master 127.5.4.126:38383 requested a full tablet report, sending...
I20260709 12:30:27.056833 18099 raft_consensus.cc:493] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:27.057439 18099 raft_consensus.cc:515] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.059930 18099 leader_election.cc:290] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:27.061499 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:27.064591 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3374bfc110c642518353f59ecdcd8a05" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
W20260709 12:30:27.067579 17307 consensus_peers.cc:597] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.067786 17962 leader_election.cc:304] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:27.069100 18092 raft_consensus.cc:2749] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:27.076182 17307 consensus_peers.cc:597] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:27.079537 17307 consensus_peers.cc:597] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.089833 17968 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.090981 17968 tablet_bootstrap.cc:492] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.091694 17968 ts_tablet_manager.cc:1403] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.064s	user 0.022s	sys 0.020s
I20260709 12:30:27.094419 17968 raft_consensus.cc:359] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.096988 17968 raft_consensus.cc:740] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.097777 17968 consensus_queue.cc:260] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.100039 17968 ts_tablet_manager.cc:1434] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:27.100844 17968 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:27.120210 17159 consensus_peers.cc:597] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.120745 17968 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.121865 17968 tablet_bootstrap.cc:492] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.122617 17968 ts_tablet_manager.cc:1403] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.011s
I20260709 12:30:27.125226 17968 raft_consensus.cc:359] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.125972 17968 raft_consensus.cc:740] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.123145 17018 ts_manager.cc:194] Re-registered known tserver with Master: 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:27.126744 17968 consensus_queue.cc:260] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.128904 17968 ts_tablet_manager.cc:1434] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 12:30:27.129714 17968 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:27.144007 17307 consensus_peers.cc:597] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.149477 17968 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.150316 17968 tablet_bootstrap.cc:492] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.150887 17968 ts_tablet_manager.cc:1403] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.001s
I20260709 12:30:27.153100 17968 raft_consensus.cc:359] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.154198 17968 raft_consensus.cc:740] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.159157 17968 consensus_queue.cc:260] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.161187 17968 ts_tablet_manager.cc:1434] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:30:27.161978 17968 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.177198 17968 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.178349 18092 raft_consensus.cc:493] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:27.179026 18092 raft_consensus.cc:515] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:27.179960 17233 consensus_peers.cc:597] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.178386 17968 tablet_bootstrap.cc:492] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.181002 18092 leader_election.cc:290] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:27.181107 17018 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33606
I20260709 12:30:27.181622 17968 ts_tablet_manager.cc:1403] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:30:27.182140 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" is_pre_election: true
I20260709 12:30:27.182646 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c1eb70ad94940b89118a32a370a4310" is_pre_election: true
I20260709 12:30:27.184429 17968 raft_consensus.cc:359] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.185225 17968 raft_consensus.cc:740] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.185935 17968 consensus_queue.cc:260] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.187718 17968 ts_tablet_manager.cc:1434] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:30:27.188256 18094 heartbeater.cc:499] Master 127.5.4.126:38383 was elected leader, sending a full tablet report...
I20260709 12:30:27.188346 17968 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.184626 17962 leader_election.cc:304] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:27.190070 18092 raft_consensus.cc:2749] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:30:27.192296 17159 consensus_peers.cc:597] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.205374 17968 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.206172 17968 tablet_bootstrap.cc:492] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.206816 17968 ts_tablet_manager.cc:1403] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 12:30:27.209751 17968 raft_consensus.cc:359] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.210270 17968 raft_consensus.cc:740] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.210913 17968 consensus_queue.cc:260] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.212697 17968 ts_tablet_manager.cc:1434] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:30:27.213464 17968 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:27.223032 17159 consensus_peers.cc:597] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:27.232569 17968 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.234910 17968 tablet_bootstrap.cc:492] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.235702 17968 ts_tablet_manager.cc:1403] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 12:30:27.238911 17968 raft_consensus.cc:359] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.239624 17968 raft_consensus.cc:740] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.240430 17968 consensus_queue.cc:260] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.242650 17968 ts_tablet_manager.cc:1434] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:27.243583 17968 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.260233 17968 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.261286 17968 tablet_bootstrap.cc:492] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.261976 17968 ts_tablet_manager.cc:1403] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.019s	user 0.002s	sys 0.012s
I20260709 12:30:27.264823 17968 raft_consensus.cc:359] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.265552 17968 raft_consensus.cc:740] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.266256 17968 consensus_queue.cc:260] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.269165 17968 ts_tablet_manager.cc:1434] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:27.270153 17968 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.284989 17968 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.286119 17968 tablet_bootstrap.cc:492] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.286876 17968 ts_tablet_manager.cc:1403] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.007s
I20260709 12:30:27.289206 17968 raft_consensus.cc:359] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.289945 17968 raft_consensus.cc:740] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.290712 17968 consensus_queue.cc:260] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.292424 17968 ts_tablet_manager.cc:1434] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:30:27.293699 17968 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.312268 17968 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.313282 17968 tablet_bootstrap.cc:492] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.313954 17968 ts_tablet_manager.cc:1403] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.000s
I20260709 12:30:27.316135 17968 raft_consensus.cc:359] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.316922 17968 raft_consensus.cc:740] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.317724 17968 consensus_queue.cc:260] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.320233 17968 ts_tablet_manager.cc:1434] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:30:27.321482 18033 raft_consensus.cc:3060] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.321626 17968 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.353371 17968 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.354800 17968 tablet_bootstrap.cc:492] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.355470 17968 ts_tablet_manager.cc:1403] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.004s
I20260709 12:30:27.358140 17968 raft_consensus.cc:359] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.358816 17968 raft_consensus.cc:740] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.359844 17968 consensus_queue.cc:260] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.368511 17968 ts_tablet_manager.cc:1434] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:30:27.369408 17968 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.387269 17968 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.388345 17968 tablet_bootstrap.cc:492] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.392014 17968 ts_tablet_manager.cc:1403] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.008s
I20260709 12:30:27.400036 17968 raft_consensus.cc:359] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.400898 17968 raft_consensus.cc:740] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.401757 17968 consensus_queue.cc:260] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.404228 17968 ts_tablet_manager.cc:1434] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.005s
I20260709 12:30:27.405256 17968 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.436728 17968 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.437573 17968 tablet_bootstrap.cc:492] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.438207 17968 ts_tablet_manager.cc:1403] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.033s	user 0.018s	sys 0.002s
I20260709 12:30:27.441040 17968 raft_consensus.cc:359] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.441723 17968 raft_consensus.cc:740] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.442698 17968 consensus_queue.cc:260] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.444835 17968 ts_tablet_manager.cc:1434] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:27.445740 17968 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.462726 17968 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.464812 17968 tablet_bootstrap.cc:492] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.466583 17968 ts_tablet_manager.cc:1403] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:30:27.469340 17968 raft_consensus.cc:359] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.469952 17968 raft_consensus.cc:740] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.470578 17968 consensus_queue.cc:260] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.472546 17968 ts_tablet_manager.cc:1434] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:27.474255 17968 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.490240 17968 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.491305 17968 tablet_bootstrap.cc:492] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.494625 17968 ts_tablet_manager.cc:1403] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.003s
I20260709 12:30:27.496937 17968 raft_consensus.cc:359] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.499502 18033 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:27.497674 17968 raft_consensus.cc:740] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.509908 17968 consensus_queue.cc:260] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.513669 17968 ts_tablet_manager.cc:1434] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:30:27.521116 17968 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.536607 18033 raft_consensus.cc:3060] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.539834 17968 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.541019 17968 tablet_bootstrap.cc:492] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.541769 17968 ts_tablet_manager.cc:1403] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 12:30:27.544721 17968 raft_consensus.cc:359] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.545363 17968 raft_consensus.cc:740] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.548965 18025 raft_consensus.cc:3060] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.546068 17968 consensus_queue.cc:260] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.552385 17968 ts_tablet_manager.cc:1434] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:30:27.553692 17968 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.576085 17968 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.577183 17968 tablet_bootstrap.cc:492] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.578012 17968 ts_tablet_manager.cc:1403] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 12:30:27.580550 17968 raft_consensus.cc:359] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.581121 17968 raft_consensus.cc:740] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.581817 17968 consensus_queue.cc:260] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.585193 17968 ts_tablet_manager.cc:1434] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:30:27.586103 17968 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.608345 17968 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.609845 17968 tablet_bootstrap.cc:492] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.610656 17968 ts_tablet_manager.cc:1403] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 12:30:27.613796 17968 raft_consensus.cc:359] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.614543 17968 raft_consensus.cc:740] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.615746 17968 consensus_queue.cc:260] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.617383 17968 ts_tablet_manager.cc:1434] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:30:27.618608 17968 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.637095 17968 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.637949 17968 tablet_bootstrap.cc:492] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.638693 17968 ts_tablet_manager.cc:1403] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.000s
I20260709 12:30:27.640765 17968 raft_consensus.cc:359] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.641420 17968 raft_consensus.cc:740] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.642179 17968 consensus_queue.cc:260] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.644686 17968 ts_tablet_manager.cc:1434] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:30:27.645613 17968 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.679212 17968 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.680469 17968 tablet_bootstrap.cc:492] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.681314 17968 ts_tablet_manager.cc:1403] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.036s	user 0.026s	sys 0.007s
I20260709 12:30:27.683542 17968 raft_consensus.cc:359] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.684083 17968 raft_consensus.cc:740] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.684794 17968 consensus_queue.cc:260] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.686815 17968 ts_tablet_manager.cc:1434] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:30:27.689011 17968 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.704195 17968 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.705253 17968 tablet_bootstrap.cc:492] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.705986 17968 ts_tablet_manager.cc:1403] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 12:30:27.708786 17968 raft_consensus.cc:359] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.709362 17968 raft_consensus.cc:740] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.710032 17968 consensus_queue.cc:260] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.711982 17968 ts_tablet_manager.cc:1434] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.003s
I20260709 12:30:27.712841 17968 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.722043 18025 raft_consensus.cc:3060] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.732714 17968 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.733893 17968 tablet_bootstrap.cc:492] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.734934 17968 ts_tablet_manager.cc:1403] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.004s
I20260709 12:30:27.738845 17968 raft_consensus.cc:359] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.739554 17968 raft_consensus.cc:740] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.740391 17968 consensus_queue.cc:260] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:27.741511 18018 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.744246 17968 ts_tablet_manager.cc:1434] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:27.745609 17968 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
W20260709 12:30:27.810554 17308 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 61883.2us
I20260709 12:30:27.882505 17968 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.884290 17968 tablet_bootstrap.cc:492] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.885525 17968 ts_tablet_manager.cc:1403] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.140s	user 0.019s	sys 0.023s
I20260709 12:30:27.888741 17968 raft_consensus.cc:359] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.889737 17968 raft_consensus.cc:740] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.890548 17968 consensus_queue.cc:260] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.892702 17968 ts_tablet_manager.cc:1434] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:27.893759 17968 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.911609 17968 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.912671 17968 tablet_bootstrap.cc:492] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.915463 17968 ts_tablet_manager.cc:1403] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.003s
I20260709 12:30:27.918630 17968 raft_consensus.cc:359] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.919368 17968 raft_consensus.cc:740] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.920192 17968 consensus_queue.cc:260] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:27.922935 17968 ts_tablet_manager.cc:1434] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:27.923784 17968 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap starting.
I20260709 12:30:27.952461 18025 raft_consensus.cc:3060] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:27.968030 17968 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:27.969138 17968 tablet_bootstrap.cc:492] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Bootstrap complete.
I20260709 12:30:27.969859 17968 ts_tablet_manager.cc:1403] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.008s
I20260709 12:30:27.976585 17968 raft_consensus.cc:359] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.979463 17968 raft_consensus.cc:740] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Initialized, Role: FOLLOWER
I20260709 12:30:27.980491 17968 consensus_queue.cc:260] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:27.983033 17968 ts_tablet_manager.cc:1434] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.005s
I20260709 12:30:28.452433 18018 raft_consensus.cc:3060] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.028317  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:37.087297 17190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868"
dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:54840
I20260709 12:30:37.087949 17190 raft_consensus.cc:606] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:37.100793 17267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038"
dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:42412
I20260709 12:30:37.101322 17267 raft_consensus.cc:606] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:37.104377 17267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa"
dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:42412
I20260709 12:30:37.105032 17267 raft_consensus.cc:606] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:37.107309 17267 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97"
dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:42412
I20260709 12:30:37.107731 17267 raft_consensus.cc:606] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:37.121615 17338 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216"
dest_uuid: "7c0f44cba62f491392d41ca89376139a"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:59386
I20260709 12:30:37.122184 17338 raft_consensus.cc:606] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:37.126248 17269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133"
dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:42412
I20260709 12:30:37.126917 17269 raft_consensus.cc:606] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:37.128105  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:30:37.129987  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:37.131227  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:37.174320 18150 raft_consensus.cc:993] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.174880 18150 raft_consensus.cc:1081] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.176461 18018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49628
I20260709 12:30:37.177002 18018 raft_consensus.cc:493] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:37.177408 18018 raft_consensus.cc:3060] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.183615 18018 raft_consensus.cc:515] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.185240 18018 leader_election.cc:290] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:37.186350 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:37.186664 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d06ba733fa2c4e5385d5839f67039a97" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:37.187321 17190 raft_consensus.cc:3060] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.187417 17269 raft_consensus.cc:3055] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:37.187812 17269 raft_consensus.cc:740] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:37.188868 17269 consensus_queue.cc:260] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.190704 17269 raft_consensus.cc:3060] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.196986 17190 raft_consensus.cc:2468] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.198618 17963 leader_election.cc:304] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:37.199637 17269 raft_consensus.cc:2468] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.200759 18163 raft_consensus.cc:2804] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:37.202612 18163 raft_consensus.cc:697] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:37.204149 18163 consensus_queue.cc:237] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.225306 17017 catalog_manager.cc:5697] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:37.346316 18150 raft_consensus.cc:993] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.346853 18150 raft_consensus.cc:1081] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.348380 18018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49628
I20260709 12:30:37.348868 18018 raft_consensus.cc:493] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:37.349164 18018 raft_consensus.cc:3060] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.355149 18018 raft_consensus.cc:515] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.356762 18018 leader_election.cc:290] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:37.357750 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:37.357775 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70b2a46ca14b57b55e8eaec58cd038" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:37.358319 17267 raft_consensus.cc:3055] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:37.358682 17190 raft_consensus.cc:3060] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.358741 17267 raft_consensus.cc:740] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:37.359809 17267 consensus_queue.cc:260] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.361208 17267 raft_consensus.cc:3060] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.364560 18143 raft_consensus.cc:993] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.365264 18156 raft_consensus.cc:1081] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.367054 17190 raft_consensus.cc:2468] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.367367 18029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49614
I20260709 12:30:37.368005 18029 raft_consensus.cc:493] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:37.368350 17963 leader_election.cc:304] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:37.368368 18029 raft_consensus.cc:3060] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.369287 18163 raft_consensus.cc:2804] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:37.370326 18163 raft_consensus.cc:697] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:37.370268 17267 raft_consensus.cc:2468] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.371454 18163 consensus_queue.cc:237] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.377426 18029 raft_consensus.cc:515] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:37.379941 18029 leader_election.cc:290] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:37.381975 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:37.382020 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d85bc0ab53bf4a66af3a1b42ff166216" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:37.382771 17267 raft_consensus.cc:3060] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.382891 17338 raft_consensus.cc:3055] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:37.383383 17338 raft_consensus.cc:740] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:37.384308 17338 consensus_queue.cc:260] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:37.385828 17338 raft_consensus.cc:3060] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.393527 17267 raft_consensus.cc:2468] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.395215 17338 raft_consensus.cc:2468] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.395397 17962 leader_election.cc:304] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:37.396607 18163 raft_consensus.cc:2804] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:37.397619 18163 raft_consensus.cc:697] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:37.398911 18163 consensus_queue.cc:237] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:37.403045 17017 catalog_manager.cc:5697] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:37.421095 18171 raft_consensus.cc:993] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.423161 18141 raft_consensus.cc:1081] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.428131 18018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49628
I20260709 12:30:37.430938 18018 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:37.431907 18018 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.437455 17017 catalog_manager.cc:5697] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 7c0f44cba62f491392d41ca89376139a (127.5.4.68) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:37.443392 18018 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:37.445768 18152 raft_consensus.cc:993] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.446244 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:37.446358 18018 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:37.447371 17267 raft_consensus.cc:3055] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:37.447736 17267 raft_consensus.cc:740] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:37.446506 18152 raft_consensus.cc:1081] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.448563 17267 consensus_queue.cc:260] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:37.450093 17267 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.450369 18018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49644
I20260709 12:30:37.451079 18018 raft_consensus.cc:493] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:37.451438 18018 raft_consensus.cc:3060] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.454530 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:37.455150 17338 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.459436 18174 raft_consensus.cc:993] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.460186 17267 raft_consensus.cc:2468] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.459846 18018 raft_consensus.cc:515] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.460134 18176 raft_consensus.cc:1081] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:37.468035 17338 raft_consensus.cc:2468] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.468135 17962 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:37.469609 18018 leader_election.cc:290] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:37.469803 18163 raft_consensus.cc:2804] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:37.470296 18163 raft_consensus.cc:697] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:37.470832 18011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49628
I20260709 12:30:37.471526 18011 raft_consensus.cc:493] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:37.471215 18163 consensus_queue.cc:237] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:37.471962 18011 raft_consensus.cc:3060] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.471832 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:37.472537 17267 raft_consensus.cc:3060] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.474435 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eac394a27ae4bd88c0948a7c1ad1868" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:37.475310 17190 raft_consensus.cc:3055] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:37.475651 17190 raft_consensus.cc:740] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:37.476517 17190 consensus_queue.cc:260] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.477943 17190 raft_consensus.cc:3060] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.482295 17267 raft_consensus.cc:2468] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.483692 17962 leader_election.cc:304] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:37.483065 17017 catalog_manager.cc:5697] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:37.484562 18163 raft_consensus.cc:2804] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:37.485036 18163 raft_consensus.cc:697] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:37.485934 18163 consensus_queue.cc:237] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.509491 18011 raft_consensus.cc:515] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.514681 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:37.515005 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "901d47a673fa40b7a58a7bea321288aa" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:37.515344 17193 raft_consensus.cc:3060] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.515709 17267 raft_consensus.cc:3055] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:37.516043 17267 raft_consensus.cc:740] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:37.516872 17267 consensus_queue.cc:260] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.518325 17267 raft_consensus.cc:3060] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:37.517563 18011 leader_election.cc:290] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:37.520999 17190 raft_consensus.cc:2468] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.527068 17267 raft_consensus.cc:2468] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.528187 17962 leader_election.cc:304] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:37.529055 18163 raft_consensus.cc:2804] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:37.529562 17193 raft_consensus.cc:2468] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:37.529604 18163 raft_consensus.cc:697] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:37.530917 18163 consensus_queue.cc:237] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:37.530604 17017 catalog_manager.cc:5697] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:37.554600 17017 catalog_manager.cc:5697] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:37.681655 17269 raft_consensus.cc:1275] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.683045 18163 consensus_queue.cc:1048] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:37.695179 17193 raft_consensus.cc:1275] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.697032 18163 consensus_queue.cc:1048] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:37.791061 17193 raft_consensus.cc:1275] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.792906 18163 consensus_queue.cc:1048] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:37.804811 17267 raft_consensus.cc:1275] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.806840 18187 consensus_queue.cc:1048] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:37.850271 17341 raft_consensus.cc:1275] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.851773 18187 consensus_queue.cc:1048] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:37.859274 17338 raft_consensus.cc:1275] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.860656 18163 consensus_queue.cc:1048] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:37.864262 17267 raft_consensus.cc:1275] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.866644 18163 consensus_queue.cc:1048] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:37.888993 17269 raft_consensus.cc:1275] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.891155 18190 consensus_queue.cc:1048] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:37.909693 17193 raft_consensus.cc:1275] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.912067 18164 consensus_queue.cc:1048] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:37.936986 17269 raft_consensus.cc:1275] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:37.938977 18164 consensus_queue.cc:1048] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:38.104803 17267 raft_consensus.cc:1275] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:38.106550 18164 consensus_queue.cc:1048] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:38.109582 17190 raft_consensus.cc:1275] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:38.110951 18191 consensus_queue.cc:1048] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:38.171690  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:38.210273 17341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b709ddd01784d588100b08085474cd7"
dest_uuid: "7c0f44cba62f491392d41ca89376139a"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:59386
I20260709 12:30:38.211014 17341 raft_consensus.cc:606] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:38.213749 17341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c"
dest_uuid: "7c0f44cba62f491392d41ca89376139a"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:59386
I20260709 12:30:38.214362 17341 raft_consensus.cc:606] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:38.217064 17190 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323"
dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:54840
I20260709 12:30:38.217530 17190 raft_consensus.cc:606] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:38.220410 17341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b"
dest_uuid: "7c0f44cba62f491392d41ca89376139a"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:59386
I20260709 12:30:38.221057 17341 raft_consensus.cc:606] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:38.223742 17341 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403"
dest_uuid: "7c0f44cba62f491392d41ca89376139a"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:59386
I20260709 12:30:38.224184 17341 raft_consensus.cc:606] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:38.225209  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:30:38.226569  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:38.227274  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:38.322698 18152 raft_consensus.cc:993] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.323243 18155 raft_consensus.cc:1081] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.324928 18011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49644
I20260709 12:30:38.325583 18011 raft_consensus.cc:493] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:38.326052 18011 raft_consensus.cc:3060] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.334611 18011 raft_consensus.cc:515] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.337775 18011 leader_election.cc:290] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:38.338317 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:38.338809 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "675c55f0ab604b0cb4f3205d8bedf323" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:38.339025 17338 raft_consensus.cc:3060] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.339608 17190 raft_consensus.cc:3055] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:38.340065 17190 raft_consensus.cc:740] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:38.341269 17190 consensus_queue.cc:260] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.343812 17190 raft_consensus.cc:3060] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.346967 17338 raft_consensus.cc:2468] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.348675 17964 leader_election.cc:304] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:38.349898 18187 raft_consensus.cc:2804] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:38.350433 18187 raft_consensus.cc:697] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:38.351439 18187 consensus_queue.cc:237] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.356297 17190 raft_consensus.cc:2468] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.373927 17018 catalog_manager.cc:5697] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:38.374414 18156 raft_consensus.cc:993] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.375228 18143 raft_consensus.cc:1081] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.377010 18011 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49614
I20260709 12:30:38.377581 18011 raft_consensus.cc:493] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:38.377972 18011 raft_consensus.cc:3060] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.385298 18011 raft_consensus.cc:515] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:38.387539 18011 leader_election.cc:290] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:38.388217 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:38.388360 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5b31c8286684684ae24797a44fc5403" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:38.389187 17341 raft_consensus.cc:3055] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:38.389497 17190 raft_consensus.cc:3060] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.389606 17341 raft_consensus.cc:740] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:38.390615 17341 consensus_queue.cc:260] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:38.393522 17341 raft_consensus.cc:3060] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.400002 17190 raft_consensus.cc:2468] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.401532 17963 leader_election.cc:304] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:38.402568 18187 raft_consensus.cc:2804] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:38.403086 18187 raft_consensus.cc:697] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:38.404000 18187 consensus_queue.cc:237] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:38.404582 17341 raft_consensus.cc:2468] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.406338 18143 raft_consensus.cc:993] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.406937 18143 raft_consensus.cc:1081] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.412369 18029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49614
I20260709 12:30:38.412984 18029 raft_consensus.cc:493] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:38.413406 18029 raft_consensus.cc:3060] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.420390 17018 catalog_manager.cc:5697] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 7c0f44cba62f491392d41ca89376139a (127.5.4.68) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:38.420944 18029 raft_consensus.cc:515] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:38.423038 18029 leader_election.cc:290] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:38.423599 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:38.424160 17190 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d2404ce16f84a8a9429e9b50a14d97b" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:38.424340 17341 raft_consensus.cc:3055] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:38.424823 17341 raft_consensus.cc:740] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:38.424847 17190 raft_consensus.cc:3060] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.426002 17341 consensus_queue.cc:260] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:38.431046 17341 raft_consensus.cc:3060] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.434212 17190 raft_consensus.cc:2468] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.435720 17963 leader_election.cc:304] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84; no voters: 
I20260709 12:30:38.436766 18187 raft_consensus.cc:2804] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:38.437587 18187 raft_consensus.cc:697] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:38.438350 17341 raft_consensus.cc:2468] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.438633 18187 consensus_queue.cc:237] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:38.454694 17018 catalog_manager.cc:5697] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 7c0f44cba62f491392d41ca89376139a (127.5.4.68) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:38.466143 18156 raft_consensus.cc:993] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.466692 18156 raft_consensus.cc:1081] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.468189 18029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49614
I20260709 12:30:38.468765 18029 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:38.469102 18029 raft_consensus.cc:3060] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.475005 18029 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.476507 18029 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:38.477231 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:38.477479 17341 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:38.478052 17267 raft_consensus.cc:3060] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.478135 17341 raft_consensus.cc:3055] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:38.478664 17341 raft_consensus.cc:740] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:38.479394 17341 consensus_queue.cc:260] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.480449 17341 raft_consensus.cc:3060] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.486857 17341 raft_consensus.cc:2468] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.487737 17964 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:38.488678 18187 raft_consensus.cc:2804] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:38.489213 18187 raft_consensus.cc:697] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:38.489975 18187 consensus_queue.cc:237] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.490666 17267 raft_consensus.cc:2468] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.501853 17018 catalog_manager.cc:5697] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 7c0f44cba62f491392d41ca89376139a (127.5.4.68) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:38.512614 18143 raft_consensus.cc:993] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.513151 18143 raft_consensus.cc:1081] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:38.514675 18029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b709ddd01784d588100b08085474cd7"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49614
I20260709 12:30:38.515194 18029 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:38.515540 18029 raft_consensus.cc:3060] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.520676 18029 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.522063 18029 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:38.523093 17267 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:38.523334 17338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b709ddd01784d588100b08085474cd7" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:38.523870 17267 raft_consensus.cc:3060] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.524022 17338 raft_consensus.cc:3055] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:38.524492 17338 raft_consensus.cc:740] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7c0f44cba62f491392d41ca89376139a, State: Running, Role: LEADER
I20260709 12:30:38.525250 17338 consensus_queue.cc:260] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.526377 17338 raft_consensus.cc:3060] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:38.531030 17267 raft_consensus.cc:2468] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.531952 17338 raft_consensus.cc:2468] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:38.532264 17962 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:38.533166 18187 raft_consensus.cc:2804] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:38.533684 18187 raft_consensus.cc:697] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:38.534574 18187 consensus_queue.cc:237] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:38.543850 17018 catalog_manager.cc:5697] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from 7c0f44cba62f491392d41ca89376139a (127.5.4.68) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:39.335588  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:39.443411 17194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9"
dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:54840
I20260709 12:30:39.444547 17194 raft_consensus.cc:606] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:39.478951 17265 raft_consensus.cc:1275] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.485520 17266 raft_consensus.cc:1275] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:30:39.495702 17234 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44307, user_credentials={real_user=slave}} blocked reactor thread for 73567us
I20260709 12:30:39.501744 17195 raft_consensus.cc:1275] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.506735 17191 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde"
dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
mode: GRACEFUL
new_leader_uuid: "6c1eb70ad94940b89118a32a370a4310"
 from {username='slave'} at 127.0.0.1:54840
I20260709 12:30:39.507085 17194 raft_consensus.cc:1275] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.507498 17191 raft_consensus.cc:606] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Received request to transfer leadership to TS 6c1eb70ad94940b89118a32a370a4310
I20260709 12:30:39.515031 17191 raft_consensus.cc:1275] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.515702 17194 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead"
dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:54840
I20260709 12:30:39.516420 17194 raft_consensus.cc:606] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:39.520363  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:30:39.522418  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:39.523097 18227 consensus_queue.cc:1048] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.525063 18216 consensus_queue.cc:1048] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:30:39.532706 17962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 46154.3us
W20260709 12:30:39.527304 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:43265, user_credentials={real_user=slave}} blocked reactor thread for 112475us
I20260709 12:30:39.528821 18219 consensus_queue.cc:1048] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.540498  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:30:39.541792 17307 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 122046us
I20260709 12:30:39.545418 18217 raft_consensus.cc:993] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: : Instructing follower 6c1eb70ad94940b89118a32a370a4310 to start an election
I20260709 12:30:39.546067 18200 raft_consensus.cc:1081] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Signalling peer 6c1eb70ad94940b89118a32a370a4310 to start an election
I20260709 12:30:39.550190 17344 raft_consensus.cc:1275] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.551473 17341 raft_consensus.cc:1275] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.555455 18228 consensus_queue.cc:1048] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.559288 17341 raft_consensus.cc:1275] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.561062 17341 raft_consensus.cc:1275] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.563014 17341 raft_consensus.cc:1275] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:39.568303 18216 consensus_queue.cc:1048] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.576272 17269 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde"
dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
 from {username='slave'} at 127.0.0.1:58918
I20260709 12:30:39.576951 17269 raft_consensus.cc:493] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:39.577363 17269 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:39.601723 18224 raft_consensus.cc:993] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:39.605341 18217 raft_consensus.cc:1081] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:39.610622 17269 raft_consensus.cc:515] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:39.618010 18229 consensus_queue.cc:1048] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.618302 18227 consensus_queue.cc:1048] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.618348 18228 consensus_queue.cc:1048] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.620069 18187 consensus_queue.cc:1048] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.622946 18234 consensus_queue.cc:1048] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:39.642237 18200 raft_consensus.cc:993] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:39.657006 18224 raft_consensus.cc:1081] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:39.658671 18029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49644
I20260709 12:30:39.659456 18029 raft_consensus.cc:493] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:39.659869 18029 raft_consensus.cc:3060] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:39.688777 17193 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:39.689985 17193 raft_consensus.cc:3055] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:30:39.690408 17193 raft_consensus.cc:740] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:39.692768 17193 consensus_queue.cc:260] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:39.695044 17193 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:39.707967 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "419b4941c01a4ffd8c32d71f505b8cde" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:39.708756 17342 raft_consensus.cc:3060] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:39.706176 17269 leader_election.cc:290] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 4 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
W20260709 12:30:39.736270 17159 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:30:39.753943 17342 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 4.
I20260709 12:30:39.763870 17235 leader_election.cc:304] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:39.764999 18176 raft_consensus.cc:2804] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:39.765513 18176 raft_consensus.cc:697] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 4 LEADER]: Becoming Leader. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:39.766374 18176 consensus_queue.cc:237] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:39.772049 18029 raft_consensus.cc:515] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:39.806933 18029 leader_election.cc:290] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:39.809567 17193 raft_consensus.cc:2468] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 4.
I20260709 12:30:39.839157 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:39.840322 17269 raft_consensus.cc:3060] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:39.843061 17195 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc02a3ce008e456caafa4f8ba81dcead" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:39.843935 17195 raft_consensus.cc:3055] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:39.844245 17195 raft_consensus.cc:740] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:39.845230 17195 consensus_queue.cc:260] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:39.872555 18029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49644
I20260709 12:30:39.873643 18029 raft_consensus.cc:493] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:39.874048 18029 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:39.875321 17195 raft_consensus.cc:3060] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:39.890141 17269 raft_consensus.cc:2468] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:39.929713 17195 raft_consensus.cc:2468] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 2.
I20260709 12:30:39.941752 18029 raft_consensus.cc:515] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:39.958653 18029 leader_election.cc:290] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 4 election: Requested vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:39.959266 17962 leader_election.cc:304] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:39.967440 18234 raft_consensus.cc:2804] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:39.967995 18234 raft_consensus.cc:697] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:39.968863 18234 consensus_queue.cc:237] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:39.976998 17015 catalog_manager.cc:5697] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 reported cstate change: term changed from 3 to 4, leader changed from c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66) to 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67). New cstate: current_term: 4 leader_uuid: "6c1eb70ad94940b89118a32a370a4310" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:40.013108 17159 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:34625, user_credentials={real_user=slave}} blocked reactor thread for 58040.5us
I20260709 12:30:40.028770 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:40.029647 17269 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:40.030880 18011 raft_consensus.cc:1240] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 4 FOLLOWER]: Rejecting Update request from peer c75b62c746bd4d90a28483bc9d5e3c84 for earlier term 3. Current term is 4. Ops: []
I20260709 12:30:40.042380 17269 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 4.
I20260709 12:30:40.043711 17962 leader_election.cc:304] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:40.045336 17191 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "09bd2f3de39744c28ad21f75b37293b9" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "c75b62c746bd4d90a28483bc9d5e3c84"
I20260709 12:30:40.045976 17191 raft_consensus.cc:3055] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:30:40.046286 17191 raft_consensus.cc:740] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c75b62c746bd4d90a28483bc9d5e3c84, State: Running, Role: LEADER
I20260709 12:30:40.047087 17191 consensus_queue.cc:260] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:40.048213 17191 raft_consensus.cc:3060] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:40.050463 18227 raft_consensus.cc:2804] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:40.054755 18227 raft_consensus.cc:697] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 4 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:40.056457 18227 consensus_queue.cc:237] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:40.058466 17018 catalog_manager.cc:5697] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 1 to 2, leader changed from c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:40.080596 17191 raft_consensus.cc:2468] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 4.
I20260709 12:30:40.112087 17018 catalog_manager.cc:5697] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 3 to 4, leader changed from c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 4 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:40.268962 17193 raft_consensus.cc:1275] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Refusing update from remote peer 6c1eb70ad94940b89118a32a370a4310: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:30:40.270599 18257 consensus_queue.cc:1048] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:40.296700 17339 raft_consensus.cc:1275] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 4 FOLLOWER]: Refusing update from remote peer 6c1eb70ad94940b89118a32a370a4310: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:30:40.298456 18220 consensus_queue.cc:1048] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:30:40.343633 17050 debug-util.cc:398] Leaking SignalData structure 0x7b080024e3e0 after lost signal to thread 5140
W20260709 12:30:40.345207 17050 debug-util.cc:398] Leaking SignalData structure 0x7b08001e78e0 after lost signal to thread 17053
W20260709 12:30:40.346916 17050 debug-util.cc:398] Leaking SignalData structure 0x7b08001c48a0 after lost signal to thread 17220
W20260709 12:30:40.348196 17050 debug-util.cc:398] Leaking SignalData structure 0x7b0800174d40 after lost signal to thread 17294
W20260709 12:30:40.349329 17050 debug-util.cc:398] Leaking SignalData structure 0x7b08003f03c0 after lost signal to thread 17368
W20260709 12:30:40.350734 17050 debug-util.cc:398] Leaking SignalData structure 0x7b08000e24a0 after lost signal to thread 18091
I20260709 12:30:40.369432 17265 raft_consensus.cc:1275] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:40.371119 18228 consensus_queue.cc:1048] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:40.430655 17193 raft_consensus.cc:1275] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:40.433921 18228 consensus_queue.cc:1048] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:40.499786 17265 raft_consensus.cc:1275] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:30:40.501294 18228 consensus_queue.cc:1048] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:40.520212 17193 raft_consensus.cc:1275] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:30:40.523051 18216 consensus_queue.cc:1048] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:40.810878  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:40.860569 17269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13be6c315f194afc80d89f01777e105a"
dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
mode: GRACEFUL
new_leader_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:42412
I20260709 12:30:40.861193 17269 raft_consensus.cc:606] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Received request to transfer leadership to TS 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:40.863229  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:30:40.864982  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:40.865793  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:41.313802 18211 raft_consensus.cc:993] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: : Instructing follower 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:41.314509 18220 raft_consensus.cc:1081] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Signalling peer 79e92a05527f4a0b98b8352cb68a1010 to start an election
I20260709 12:30:41.318212 18018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13be6c315f194afc80d89f01777e105a"
dest_uuid: "79e92a05527f4a0b98b8352cb68a1010"
 from {username='slave'} at 127.0.0.1:49628
I20260709 12:30:41.318917 18018 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:41.319301 18018 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:41.327585 18018 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:41.331588 17269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c1eb70ad94940b89118a32a370a4310"
I20260709 12:30:41.332227 17269 raft_consensus.cc:3055] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:41.332530 17269 raft_consensus.cc:740] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c1eb70ad94940b89118a32a370a4310, State: Running, Role: LEADER
I20260709 12:30:41.332726 18018 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 3 election: Requested vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585)
I20260709 12:30:41.333328 17269 consensus_queue.cc:260] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:41.334647 17269 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:41.336684 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13be6c315f194afc80d89f01777e105a" candidate_uuid: "79e92a05527f4a0b98b8352cb68a1010" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:41.337301 17342 raft_consensus.cc:3060] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:41.352315 17342 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 3.
I20260709 12:30:41.361248 17269 raft_consensus.cc:2468] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e92a05527f4a0b98b8352cb68a1010 in term 3.
I20260709 12:30:41.362700 17962 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010; no voters: 
I20260709 12:30:41.364248 18219 raft_consensus.cc:2804] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:41.364759 18219 raft_consensus.cc:697] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 3 LEADER]: Becoming Leader. State: Replica: 79e92a05527f4a0b98b8352cb68a1010, State: Running, Role: LEADER
I20260709 12:30:41.366405 18219 consensus_queue.cc:237] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:41.379037 17015 catalog_manager.cc:5697] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 reported cstate change: term changed from 2 to 3, leader changed from 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67) to 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "79e92a05527f4a0b98b8352cb68a1010" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:41.797845 17342 raft_consensus.cc:1275] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:41.799685 18219 consensus_queue.cc:1048] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:41.835740 17269 raft_consensus.cc:1275] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Refusing update from remote peer 79e92a05527f4a0b98b8352cb68a1010: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:41.848960 18228 consensus_queue.cc:1048] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:41.926277  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:41.971298  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:41.973289  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:41.974072  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:42.253898  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:42.400264  5137 tablet_server.cc:179] TabletServer@127.5.4.65:34625 shutting down...
W20260709 12:30:42.410034 17159 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 227 similar messages]
W20260709 12:30:42.442415 17159 consensus_peers.cc:597] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.466447 17307 consensus_peers.cc:597] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.469380 17307 consensus_peers.cc:597] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.470255 17307 consensus_peers.cc:597] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.484899 17233 consensus_peers.cc:597] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.490613  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:42.491745  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.492756  5137 raft_consensus.cc:2243] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.494045  5137 raft_consensus.cc:2272] T 27bba673d23842c790eb0140a01cee0c P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.498541 17307 consensus_peers.cc:597] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.498852  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.500598  5137 raft_consensus.cc:2243] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.501425  5137 raft_consensus.cc:2272] T d14098588e6e4421bbf71e16dea54da0 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.505558 17307 consensus_peers.cc:597] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.507138  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.509850  5137 raft_consensus.cc:2243] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.510527  5137 raft_consensus.cc:2272] T 458688757584439eb735f41c9b0e9eaa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.515450  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.516598  5137 raft_consensus.cc:2243] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.517441  5137 raft_consensus.cc:2272] T fd33570a634744e89bbf929e83441909 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.521307  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.522348  5137 raft_consensus.cc:2243] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.523082  5137 raft_consensus.cc:2272] T 44f0f66af5d044fdb87c6320a84cbf8e P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.526616  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.527657  5137 raft_consensus.cc:2243] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.528252  5137 raft_consensus.cc:2272] T 6ec669aa1bf9434f9b198f6d8e768348 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.535600  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.536852  5137 raft_consensus.cc:2243] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.538277  5137 raft_consensus.cc:2272] T 6c70b2a46ca14b57b55e8eaec58cd038 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.542311  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.543489  5137 raft_consensus.cc:2243] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.544318  5137 raft_consensus.cc:2272] T 90bb36eac77e43ad8fc0568e85913a36 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.548501  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.549499  5137 raft_consensus.cc:2243] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.550297  5137 raft_consensus.cc:2272] T 1ca4d4100a154e058fd6b5525d3e280f P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.551326 17159 consensus_peers.cc:597] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.554057  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.554905  5137 raft_consensus.cc:2243] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.555454  5137 raft_consensus.cc:2272] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.559039  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.559998  5137 raft_consensus.cc:2243] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.561332  5137 raft_consensus.cc:2272] T 0eac394a27ae4bd88c0948a7c1ad1868 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.564833  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.565608  5137 raft_consensus.cc:2243] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.566167  5137 raft_consensus.cc:2272] T b6a9d0d27df74b748881bc3ef8245a62 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.569458  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.570456  5137 raft_consensus.cc:2243] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.571112  5137 raft_consensus.cc:2272] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.574682  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.575491  5137 raft_consensus.cc:2243] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.575990  5137 raft_consensus.cc:2272] T cd19f1dc408f4801b7e7d672696228cc P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.579051  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.579860  5137 raft_consensus.cc:2243] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.581122  5137 raft_consensus.cc:2272] T 9d2404ce16f84a8a9429e9b50a14d97b P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.585361  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.586412  5137 raft_consensus.cc:2243] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.587055  5137 raft_consensus.cc:2272] T ec4e7b8f0ed341e0b42e5d3872c70010 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.591286  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.592248  5137 raft_consensus.cc:2243] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.593130  5137 raft_consensus.cc:2272] T a100ea0ba8a74d14bdfc772d6059f39c P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.598202  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.599090  5137 raft_consensus.cc:2243] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.599825  5137 raft_consensus.cc:2272] T 44c804e5f1864c799cd5d1d49734db98 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.603986  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.605157  5137 raft_consensus.cc:2243] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.605849  5137 raft_consensus.cc:2272] T 9614ba0ca1894975b6696b3724157ee6 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.609812  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.610847  5137 raft_consensus.cc:2243] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.612521  5137 raft_consensus.cc:2272] T cc02a3ce008e456caafa4f8ba81dcead P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.616231  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.617260  5137 raft_consensus.cc:2243] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:30:42.617873 17233 consensus_peers.cc:597] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.618670  5137 raft_consensus.cc:2272] T fb995bc24cf54980b8977e69501ac133 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.622964  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.623689  5137 raft_consensus.cc:2243] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:42.623734 17307 consensus_peers.cc:597] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.624400  5137 raft_consensus.cc:2272] T 80ed704c39774ac88e3d32548f26b242 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.630999  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.632424  5137 raft_consensus.cc:2243] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.633337  5137 raft_consensus.cc:2272] T 51de8b9a56eb439e9c25052d3d9587f9 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.638769  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.639664  5137 raft_consensus.cc:2243] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.640352  5137 raft_consensus.cc:2272] T f9af739be394434c9d30f15785e5e692 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.643900 17307 consensus_peers.cc:597] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.713740  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.714986  5137 raft_consensus.cc:2243] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:42.723802 17161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.68:38585, user_credentials={real_user=slave}} blocked reactor thread for 72812.6us
I20260709 12:30:42.745565  5137 raft_consensus.cc:2272] T ec4250a950114da1bce4b46571dc97cc P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.749966  5137 tablet_replica.cc:333] stopping tablet replica
W20260709 12:30:42.757498 17159 consensus_peers.cc:597] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.758456  5137 raft_consensus.cc:2243] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:42.758682 17307 consensus_peers.cc:597] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.759258 17159 consensus_peers.cc:597] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.760360  5137 raft_consensus.cc:2272] T 058259c8483642bba034c3bd3f5ba90e P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.760975 17159 consensus_peers.cc:597] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.761868 17307 consensus_peers.cc:597] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.764271  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.765327  5137 raft_consensus.cc:2243] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.766781  5137 raft_consensus.cc:2272] T d5b31c8286684684ae24797a44fc5403 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.767297 17307 consensus_peers.cc:597] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.771525  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.772624  5137 raft_consensus.cc:2243] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.773478  5137 raft_consensus.cc:2272] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.780359  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.785601  5137 raft_consensus.cc:2243] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.786290  5137 raft_consensus.cc:2272] T 52b7abb65c454a8f9dca603b58f2cc0a P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.790199 17159 consensus_peers.cc:597] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.791826  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.792872  5137 raft_consensus.cc:2243] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:42.793036 17233 consensus_peers.cc:597] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.790669 17307 consensus_peers.cc:597] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.793531  5137 raft_consensus.cc:2272] T 9ba13b6455354f64a81d0be356cf1982 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.798955 17233 consensus_peers.cc:597] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.800413  5137 tablet_replica.cc:333] stopping tablet replica
W20260709 12:30:42.801618 17159 consensus_peers.cc:597] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.801677  5137 raft_consensus.cc:2243] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.803282  5137 raft_consensus.cc:2272] T 55a24caeb8124f77a6a9d21f8bed1831 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.803561 17307 consensus_peers.cc:597] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.807003  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.808009  5137 raft_consensus.cc:2243] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.809514  5137 raft_consensus.cc:2272] T 675c55f0ab604b0cb4f3205d8bedf323 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.812992  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.813843  5137 raft_consensus.cc:2243] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.814608  5137 raft_consensus.cc:2272] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.818099  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.819103  5137 raft_consensus.cc:2243] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.819751  5137 raft_consensus.cc:2272] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.824178  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.825057  5137 raft_consensus.cc:2243] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.825640  5137 raft_consensus.cc:2272] T 3374bfc110c642518353f59ecdcd8a05 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.830204  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.831033  5137 raft_consensus.cc:2243] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.832984  5137 raft_consensus.cc:2272] T d06ba733fa2c4e5385d5839f67039a97 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.836934  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.837806  5137 raft_consensus.cc:2243] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.839154  5137 raft_consensus.cc:2272] T 13be6c315f194afc80d89f01777e105a P 79e92a05527f4a0b98b8352cb68a1010 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.839211 17159 consensus_peers.cc:597] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.842274  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.843220  5137 raft_consensus.cc:2243] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.843915  5137 raft_consensus.cc:2272] T 56a32f727d4a47368863c010ce8508c8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.846303 17307 consensus_peers.cc:597] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.847807  5137 tablet_replica.cc:333] stopping tablet replica
W20260709 12:30:42.848505 17159 consensus_peers.cc:597] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:42.850095  5137 raft_consensus.cc:2243] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.851517  5137 raft_consensus.cc:2272] T d85bc0ab53bf4a66af3a1b42ff166216 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.854425  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.855093  5137 raft_consensus.cc:2243] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.855562  5137 raft_consensus.cc:2272] T 5746c5442b934f38910321caaa1db9a8 P 79e92a05527f4a0b98b8352cb68a1010 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.859026  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.859869  5137 raft_consensus.cc:2243] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.860972  5137 raft_consensus.cc:2272] T 901d47a673fa40b7a58a7bea321288aa P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.864145  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.864774  5137 raft_consensus.cc:2243] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.865765  5137 raft_consensus.cc:2272] T 0b709ddd01784d588100b08085474cd7 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.869364  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.870342  5137 raft_consensus.cc:2243] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:42.871719  5137 raft_consensus.cc:2272] T 09bd2f3de39744c28ad21f75b37293b9 P 79e92a05527f4a0b98b8352cb68a1010 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.875206  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.875811  5137 raft_consensus.cc:2243] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.876430  5137 raft_consensus.cc:2272] T ecf98a8bc05f404f8fb4149773b1c384 P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:42.879213  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:42.879765  5137 raft_consensus.cc:2243] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:42.880436  5137 raft_consensus.cc:2272] T e9b921f32d04442e89c1b7ea73df020f P 79e92a05527f4a0b98b8352cb68a1010 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:42.904274 17159 consensus_peers.cc:597] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.913770 17307 consensus_peers.cc:597] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.936437 17233 consensus_peers.cc:597] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.952680 17159 consensus_peers.cc:597] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:42.995615 17159 consensus_peers.cc:597] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 -> Peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Couldn't send request to peer 79e92a05527f4a0b98b8352cb68a1010. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.000602  5137 tablet_server.cc:196] TabletServer@127.5.4.65:34625 shutdown complete.
I20260709 12:30:43.190788  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
I20260709 12:30:43.276108  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:43.279983  5137 tablet_replica.cc:333] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.281414  5137 raft_consensus.cc:2243] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.282855  5137 raft_consensus.cc:2272] T 9b33f790d3fc4b0099ea91c45cc6fd11 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.285812  5137 tablet_replica.cc:333] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.287192  5137 raft_consensus.cc:2243] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.287859  5137 raft_consensus.cc:2272] T e58121712c0045238ff325078dcf0653 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.291247  5137 tablet_replica.cc:333] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
W20260709 12:30:43.291112 17234 consensus_peers.cc:597] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.292356  5137 raft_consensus.cc:2243] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.292928  5137 raft_consensus.cc:2272] T d06ba733fa2c4e5385d5839f67039a97 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.295526  5137 tablet_replica.cc:333] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.296278  5137 raft_consensus.cc:2243] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.297416  5137 raft_consensus.cc:2272] T 3374bfc110c642518353f59ecdcd8a05 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.300580  5137 tablet_replica.cc:333] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.301321  5137 raft_consensus.cc:2243] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.301801  5137 raft_consensus.cc:2272] T 458688757584439eb735f41c9b0e9eaa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.304473  5137 tablet_replica.cc:333] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.305146  5137 raft_consensus.cc:2243] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.305603  5137 raft_consensus.cc:2272] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.308338  5137 tablet_replica.cc:333] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.309023  5137 raft_consensus.cc:2243] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.309499  5137 raft_consensus.cc:2272] T bc38664bb0494fe080bd97ce12c2481b P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.315201  5137 tablet_replica.cc:333] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.319774  5137 raft_consensus.cc:2243] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.320559  5137 raft_consensus.cc:2272] T cc02a3ce008e456caafa4f8ba81dcead P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.324004  5137 tablet_replica.cc:333] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.325001  5137 raft_consensus.cc:2243] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.325678  5137 raft_consensus.cc:2272] T 6c70b2a46ca14b57b55e8eaec58cd038 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.329031  5137 tablet_replica.cc:333] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.329959  5137 raft_consensus.cc:2243] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.331182  5137 raft_consensus.cc:2272] T 90bb36eac77e43ad8fc0568e85913a36 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.334233  5137 tablet_replica.cc:333] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.335153  5137 raft_consensus.cc:2243] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.335784  5137 raft_consensus.cc:2272] T ce19f767e060412c8bc15f391d74b87f P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.339186  5137 tablet_replica.cc:333] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.340128  5137 raft_consensus.cc:2243] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.340795  5137 raft_consensus.cc:2272] T 44f0f66af5d044fdb87c6320a84cbf8e P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:43.341979 17234 consensus_peers.cc:597] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.344925 17308 consensus_peers.cc:597] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.345468  5137 tablet_replica.cc:333] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.346515  5137 raft_consensus.cc:2243] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.347203  5137 raft_consensus.cc:2272] T 6ec669aa1bf9434f9b198f6d8e768348 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.350517  5137 tablet_replica.cc:333] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.351389  5137 raft_consensus.cc:2243] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.352068  5137 raft_consensus.cc:2272] T 419b4941c01a4ffd8c32d71f505b8cde P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.355588  5137 tablet_replica.cc:333] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
W20260709 12:30:43.356055 17234 consensus_peers.cc:597] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.356509  5137 raft_consensus.cc:2243] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.358218  5137 raft_consensus.cc:2272] T 6e37e6c15d9f4c37b87d6f494bdc66bf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.361636  5137 tablet_replica.cc:333] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
W20260709 12:30:43.362574 17308 consensus_peers.cc:597] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.362733  5137 raft_consensus.cc:2243] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.364286  5137 raft_consensus.cc:2272] T 1711901c61714bc1bbb4f2bec64dadb3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.367036  5137 tablet_replica.cc:333] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.367678  5137 raft_consensus.cc:2243] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.368244  5137 raft_consensus.cc:2272] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.371625  5137 tablet_replica.cc:333] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.372524  5137 raft_consensus.cc:2243] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.373116  5137 raft_consensus.cc:2272] T cd19f1dc408f4801b7e7d672696228cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.376590  5137 tablet_replica.cc:333] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.377498  5137 raft_consensus.cc:2243] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.378146  5137 raft_consensus.cc:2272] T 80ed704c39774ac88e3d32548f26b242 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.381244  5137 tablet_replica.cc:333] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
W20260709 12:30:43.381934 17308 consensus_peers.cc:597] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.382277  5137 raft_consensus.cc:2243] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.383147  5137 raft_consensus.cc:2272] T 2d81d3402a7848f886f20c683e7874f3 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.386785  5137 tablet_replica.cc:333] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.387713  5137 raft_consensus.cc:2243] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.388936  5137 raft_consensus.cc:2272] T ecf98a8bc05f404f8fb4149773b1c384 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.392102  5137 tablet_replica.cc:333] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.392819  5137 raft_consensus.cc:2243] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.393421  5137 raft_consensus.cc:2272] T 09bd2f3de39744c28ad21f75b37293b9 P c75b62c746bd4d90a28483bc9d5e3c84 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.395994  5137 tablet_replica.cc:333] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.396656  5137 raft_consensus.cc:2243] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.397269  5137 raft_consensus.cc:2272] T 9ba13b6455354f64a81d0be356cf1982 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.399981  5137 tablet_replica.cc:333] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.400590  5137 raft_consensus.cc:2243] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.401525  5137 raft_consensus.cc:2272] T 55a24caeb8124f77a6a9d21f8bed1831 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.404210  5137 tablet_replica.cc:333] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.404788  5137 raft_consensus.cc:2243] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.405292  5137 raft_consensus.cc:2272] T 0af900dfc2784f8781502931a17eaa79 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.408209  5137 tablet_replica.cc:333] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.408993  5137 raft_consensus.cc:2243] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.409644  5137 raft_consensus.cc:2272] T 675c55f0ab604b0cb4f3205d8bedf323 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:43.410629 17308 consensus_peers.cc:597] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.412758  5137 tablet_replica.cc:333] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.413539  5137 raft_consensus.cc:2243] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.414070  5137 raft_consensus.cc:2272] T 35817ffe9ce34abd82d949dc55a8ae78 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.416671  5137 tablet_replica.cc:333] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.417312  5137 raft_consensus.cc:2243] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.418325  5137 raft_consensus.cc:2272] T 440c7aeff35f4f3988c3eebfef1c6acf P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.420795  5137 tablet_replica.cc:333] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.421409  5137 raft_consensus.cc:2243] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.422111  5137 raft_consensus.cc:2272] T fd26acf9f75043088e62624ad9f61c2e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.425303  5137 tablet_replica.cc:333] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.426198  5137 raft_consensus.cc:2243] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.427326  5137 raft_consensus.cc:2272] T 52b7abb65c454a8f9dca603b58f2cc0a P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.430562  5137 tablet_replica.cc:333] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.431622  5137 raft_consensus.cc:2243] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.432896  5137 raft_consensus.cc:2272] T fd33570a634744e89bbf929e83441909 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.436688  5137 tablet_replica.cc:333] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.437796  5137 raft_consensus.cc:2243] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.438726  5137 raft_consensus.cc:2272] T d62bfda640da40f9ab6e7b1da9479a5c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:43.439949 17234 consensus_peers.cc:597] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.442193  5137 tablet_replica.cc:333] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.443105  5137 raft_consensus.cc:2243] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.443737  5137 raft_consensus.cc:2272] T 795b495c58894c1eb861e953546a6e6d P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.446669  5137 tablet_replica.cc:333] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.447412  5137 raft_consensus.cc:2243] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.448575  5137 raft_consensus.cc:2272] T 5746c5442b934f38910321caaa1db9a8 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.452001  5137 tablet_replica.cc:333] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.453027  5137 raft_consensus.cc:2243] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.453622  5137 raft_consensus.cc:2272] T 901d47a673fa40b7a58a7bea321288aa P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.456928  5137 tablet_replica.cc:333] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.457810  5137 raft_consensus.cc:2243] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.459244  5137 raft_consensus.cc:2272] T 0148e46ec1b2484e902b01edfda2976c P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.462563  5137 tablet_replica.cc:333] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.463388  5137 raft_consensus.cc:2243] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.464008  5137 raft_consensus.cc:2272] T 0eac394a27ae4bd88c0948a7c1ad1868 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.467209  5137 tablet_replica.cc:333] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.467864  5137 raft_consensus.cc:2243] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.468345  5137 raft_consensus.cc:2272] T d5b31c8286684684ae24797a44fc5403 P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.470762  5137 tablet_replica.cc:333] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.471326  5137 raft_consensus.cc:2243] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.472194  5137 raft_consensus.cc:2272] T 058259c8483642bba034c3bd3f5ba90e P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.474670  5137 tablet_replica.cc:333] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.475201  5137 raft_consensus.cc:2243] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.476065  5137 raft_consensus.cc:2272] T ec4e7b8f0ed341e0b42e5d3872c70010 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.478411  5137 tablet_replica.cc:333] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.479050  5137 raft_consensus.cc:2243] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.479553  5137 raft_consensus.cc:2272] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.482074  5137 tablet_replica.cc:333] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.482743  5137 raft_consensus.cc:2243] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.483211  5137 raft_consensus.cc:2272] T ec4250a950114da1bce4b46571dc97cc P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.485981  5137 tablet_replica.cc:333] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.486663  5137 raft_consensus.cc:2243] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.487567  5137 raft_consensus.cc:2272] T f9af739be394434c9d30f15785e5e692 P c75b62c746bd4d90a28483bc9d5e3c84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.490173  5137 tablet_replica.cc:333] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84: stopping tablet replica
I20260709 12:30:43.490842  5137 raft_consensus.cc:2243] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.491420  5137 raft_consensus.cc:2272] T 9d2404ce16f84a8a9429e9b50a14d97b P c75b62c746bd4d90a28483bc9d5e3c84 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:43.524921 17234 consensus_peers.cc:597] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.530426 17308 consensus_peers.cc:597] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.533658 17234 consensus_peers.cc:597] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.547591 17234 consensus_peers.cc:597] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.549975 17234 consensus_peers.cc:597] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.579967 17234 consensus_peers.cc:597] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.584872 17308 consensus_peers.cc:597] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.608153 17234 consensus_peers.cc:597] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.612551  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
W20260709 12:30:43.632238 17308 consensus_peers.cc:597] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.638577 17234 consensus_peers.cc:597] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.675060 17308 consensus_peers.cc:597] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.704922 18220 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.707113 18290 raft_consensus.cc:493] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.707752 18290 raft_consensus.cc:515] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:43.710294 18290 leader_election.cc:290] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:43.705788 18220 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
W20260709 12:30:43.716230 17233 leader_election.cc:336] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
W20260709 12:30:43.722129 17234 leader_election.cc:336] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
I20260709 12:30:43.722913 17234 leader_election.cc:304] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:43.723932 18290 raft_consensus.cc:2749] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:43.724149 17342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27bba673d23842c790eb0140a01cee0c" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
W20260709 12:30:43.725257 17234 consensus_peers.cc:597] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 -> Peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Couldn't send request to peer c75b62c746bd4d90a28483bc9d5e3c84. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.746129  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
W20260709 12:30:43.754757 17233 leader_election.cc:336] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.755330 17233 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, 7c0f44cba62f491392d41ca89376139a
I20260709 12:30:43.752146 18220 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:43.757978 18220 raft_consensus.cc:2749] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:43.790076 18311 raft_consensus.cc:493] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.790781 18311 raft_consensus.cc:515] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
W20260709 12:30:43.792521 17307 consensus_peers.cc:597] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.793164 18311 leader_election.cc:290] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
W20260709 12:30:43.798676 17307 consensus_peers.cc:597] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:43.800789 17234 leader_election.cc:336] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
I20260709 12:30:43.810045 18311 raft_consensus.cc:493] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
W20260709 12:30:43.810307 17233 leader_election.cc:336] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.812033 18311 raft_consensus.cc:515] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:43.813289 17233 leader_election.cc:304] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:43.814185 18312 raft_consensus.cc:2749] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:43.815315 18311 leader_election.cc:290] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:43.816047 18209 raft_consensus.cc:493] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.816601 18223 raft_consensus.cc:493] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.817229 18223 raft_consensus.cc:515] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:43.819471 18223 leader_election.cc:290] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:43.816684 18209 raft_consensus.cc:515] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:43.822741 18209 leader_election.cc:290] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:43.823678 18209 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
W20260709 12:30:43.823933 17233 leader_election.cc:336] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.824846 18209 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:43.827121 18209 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:43.829967 17308 leader_election.cc:336] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
W20260709 12:30:43.832086 17234 leader_election.cc:336] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
I20260709 12:30:43.832697 17234 leader_election.cc:304] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:43.833498 18311 raft_consensus.cc:2749] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:43.840553 17307 leader_election.cc:336] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.840957 17307 leader_election.cc:304] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:43.841624 18209 raft_consensus.cc:2749] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:43.842679 17307 leader_election.cc:336] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
W20260709 12:30:43.843158 17307 leader_election.cc:336] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
W20260709 12:30:43.843966 17307 leader_election.cc:336] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:30:43.844357 17307 leader_election.cc:304] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:43.845018 18209 raft_consensus.cc:2749] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:43.845999 17307 leader_election.cc:336] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:30:43.846343 17307 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:43.847016 18209 raft_consensus.cc:2749] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:43.866453  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:43.867865  5137 tablet_replica.cc:333] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.869061  5137 raft_consensus.cc:2243] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.869807  5137 raft_consensus.cc:2272] T d14098588e6e4421bbf71e16dea54da0 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.873739  5137 tablet_replica.cc:333] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.874687  5137 raft_consensus.cc:2243] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.875351  5137 raft_consensus.cc:2272] T 51de8b9a56eb439e9c25052d3d9587f9 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.879128  5137 tablet_replica.cc:333] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.880125  5137 raft_consensus.cc:2243] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.880743  5137 raft_consensus.cc:2272] T 9b33f790d3fc4b0099ea91c45cc6fd11 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.884341  5137 tablet_replica.cc:333] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.885406  5137 raft_consensus.cc:2243] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.886708  5137 raft_consensus.cc:2272] T e58121712c0045238ff325078dcf0653 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.890515  5137 tablet_replica.cc:333] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.891476  5137 raft_consensus.cc:2243] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.892192  5137 raft_consensus.cc:2272] T d06ba733fa2c4e5385d5839f67039a97 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.895959  5137 tablet_replica.cc:333] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.896908  5137 raft_consensus.cc:2243] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.898170  5137 raft_consensus.cc:2272] T bc38664bb0494fe080bd97ce12c2481b P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.901769  5137 tablet_replica.cc:333] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.902877  5137 raft_consensus.cc:2243] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.903535  5137 raft_consensus.cc:2272] T 3374bfc110c642518353f59ecdcd8a05 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.906960  5137 tablet_replica.cc:333] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.907928  5137 raft_consensus.cc:2243] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.908598  5137 raft_consensus.cc:2272] T 0b709ddd01784d588100b08085474cd7 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.912302  5137 tablet_replica.cc:333] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.913269  5137 raft_consensus.cc:2243] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.913981  5137 raft_consensus.cc:2272] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.917322  5137 tablet_replica.cc:333] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.918188  5137 raft_consensus.cc:2243] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.918933  5137 raft_consensus.cc:2272] T cc02a3ce008e456caafa4f8ba81dcead P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:43.922117 17307 consensus_peers.cc:597] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.922626  5137 tablet_replica.cc:333] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.923668  5137 raft_consensus.cc:2243] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.925014  5137 raft_consensus.cc:2272] T 44f0f66af5d044fdb87c6320a84cbf8e P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.925693 18311 raft_consensus.cc:493] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.926216 18311 raft_consensus.cc:515] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } }
I20260709 12:30:43.928171 18311 leader_election.cc:290] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:43.928680  5137 tablet_replica.cc:333] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.929762  5137 raft_consensus.cc:2243] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.930474  5137 raft_consensus.cc:2272] T 6c70b2a46ca14b57b55e8eaec58cd038 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:43.932016 17233 leader_election.cc:336] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.934286  5137 tablet_replica.cc:333] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
W20260709 12:30:43.934826 17234 leader_election.cc:336] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
I20260709 12:30:43.935226 17234 leader_election.cc:304] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:43.935448  5137 raft_consensus.cc:2243] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.935804 18311 raft_consensus.cc:2749] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:43.936914  5137 raft_consensus.cc:2272] T ce19f767e060412c8bc15f391d74b87f P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.940413  5137 tablet_replica.cc:333] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.941370  5137 raft_consensus.cc:2243] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.942692  5137 raft_consensus.cc:2272] T b6a9d0d27df74b748881bc3ef8245a62 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.946223  5137 tablet_replica.cc:333] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.947168  5137 raft_consensus.cc:2243] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.948364  5137 raft_consensus.cc:2272] T 419b4941c01a4ffd8c32d71f505b8cde P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.951941  5137 tablet_replica.cc:333] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.953101  5137 raft_consensus.cc:2243] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.954025  5137 raft_consensus.cc:2272] T 6e37e6c15d9f4c37b87d6f494bdc66bf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.957854  5137 tablet_replica.cc:333] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
W20260709 12:30:43.958287 17307 consensus_peers.cc:597] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:43.958913  5137 raft_consensus.cc:2243] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.960356  5137 raft_consensus.cc:2272] T 2d81d3402a7848f886f20c683e7874f3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.964308  5137 tablet_replica.cc:333] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.965255  5137 raft_consensus.cc:2243] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.965904  5137 raft_consensus.cc:2272] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.968343 18311 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.968842 18311 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:43.969689  5137 tablet_replica.cc:333] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.970800  5137 raft_consensus.cc:2243] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.971580 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a" is_pre_election: true
I20260709 12:30:43.972122  5137 raft_consensus.cc:2272] T cd19f1dc408f4801b7e7d672696228cc P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.972295 17344 raft_consensus.cc:2468] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 2.
I20260709 12:30:43.973819 17235 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 
I20260709 12:30:43.974730 18312 raft_consensus.cc:2804] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:43.975122 18312 raft_consensus.cc:493] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:43.975456 18312 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:43.976101  5137 tablet_replica.cc:333] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.976243 18311 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:43.976099 17233 leader_election.cc:336] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.982043 18312 raft_consensus.cc:515] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:43.982930  5137 raft_consensus.cc:2243] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.984869 17344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb995bc24cf54980b8977e69501ac133" candidate_uuid: "6c1eb70ad94940b89118a32a370a4310" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "7c0f44cba62f491392d41ca89376139a"
I20260709 12:30:43.985553 17344 raft_consensus.cc:3060] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:43.986395  5137 raft_consensus.cc:2272] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.986969 18312 leader_election.cc:290] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 election: Requested vote from peers 7c0f44cba62f491392d41ca89376139a (127.5.4.68:38585), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:43.992064  5137 tablet_replica.cc:333] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:43.992983  5137 raft_consensus.cc:2243] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:43.994094  5137 raft_consensus.cc:2272] T 1ca4d4100a154e058fd6b5525d3e280f P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:43.997349  5137 tablet_replica.cc:333] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
W20260709 12:30:43.997382 17233 leader_election.cc:336] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:43.998322  5137 raft_consensus.cc:2243] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:43.999096  5137 raft_consensus.cc:2272] T 27bba673d23842c790eb0140a01cee0c P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.002193  5137 tablet_replica.cc:333] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.002993  5137 raft_consensus.cc:2243] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.004060 17344 raft_consensus.cc:2468] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c1eb70ad94940b89118a32a370a4310 in term 3.
I20260709 12:30:44.006238 17235 leader_election.cc:304] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c1eb70ad94940b89118a32a370a4310, 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010
W20260709 12:30:44.006946 17235 raft_consensus.cc:2686] T fb995bc24cf54980b8977e69501ac133 P 6c1eb70ad94940b89118a32a370a4310: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:30:44.004230  5137 raft_consensus.cc:2272] T d62bfda640da40f9ab6e7b1da9479a5c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:44.008834 17307 consensus_peers.cc:597] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:44.009603 17307 consensus_peers.cc:597] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:44.010291  5137 tablet_replica.cc:333] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.010994  5137 raft_consensus.cc:2243] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.011883  5137 raft_consensus.cc:2272] T 795b495c58894c1eb861e953546a6e6d P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.014348  5137 tablet_replica.cc:333] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.014950  5137 raft_consensus.cc:2243] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.015421  5137 raft_consensus.cc:2272] T 56a32f727d4a47368863c010ce8508c8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.018000  5137 tablet_replica.cc:333] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.018772  5137 raft_consensus.cc:2243] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.019694  5137 raft_consensus.cc:2272] T 35817ffe9ce34abd82d949dc55a8ae78 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.022472  5137 tablet_replica.cc:333] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.023226  5137 raft_consensus.cc:2243] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.023698  5137 raft_consensus.cc:2272] T 13be6c315f194afc80d89f01777e105a P 6c1eb70ad94940b89118a32a370a4310 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.025748  5137 tablet_replica.cc:333] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.026321  5137 raft_consensus.cc:2243] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.027180  5137 raft_consensus.cc:2272] T d85bc0ab53bf4a66af3a1b42ff166216 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.029314  5137 tablet_replica.cc:333] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.029913  5137 raft_consensus.cc:2243] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.030412  5137 raft_consensus.cc:2272] T 0eac394a27ae4bd88c0948a7c1ad1868 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.032956  5137 tablet_replica.cc:333] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.033751  5137 raft_consensus.cc:2243] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.034422  5137 raft_consensus.cc:2272] T 5746c5442b934f38910321caaa1db9a8 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.037086  5137 tablet_replica.cc:333] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.037818  5137 raft_consensus.cc:2243] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.038364  5137 raft_consensus.cc:2272] T 0148e46ec1b2484e902b01edfda2976c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.040934  5137 tablet_replica.cc:333] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.041518  5137 raft_consensus.cc:2243] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.042021  5137 raft_consensus.cc:2272] T 901d47a673fa40b7a58a7bea321288aa P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.044324  5137 tablet_replica.cc:333] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.044862  5137 raft_consensus.cc:2243] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.045260  5137 raft_consensus.cc:2272] T a100ea0ba8a74d14bdfc772d6059f39c P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.047358  5137 tablet_replica.cc:333] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.047911  5137 raft_consensus.cc:2243] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.048349  5137 raft_consensus.cc:2272] T 44c804e5f1864c799cd5d1d49734db98 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.050374  5137 tablet_replica.cc:333] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.050940  5137 raft_consensus.cc:2243] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.051360  5137 raft_consensus.cc:2272] T 440c7aeff35f4f3988c3eebfef1c6acf P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.053442  5137 tablet_replica.cc:333] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.053996  5137 raft_consensus.cc:2243] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.054895  5137 raft_consensus.cc:2272] T fd26acf9f75043088e62624ad9f61c2e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.056958  5137 tablet_replica.cc:333] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.057448  5137 raft_consensus.cc:2243] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.057901  5137 raft_consensus.cc:2272] T 52b7abb65c454a8f9dca603b58f2cc0a P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.060125  5137 tablet_replica.cc:333] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.060657  5137 raft_consensus.cc:2243] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.061129  5137 raft_consensus.cc:2272] T fd33570a634744e89bbf929e83441909 P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.063419  5137 tablet_replica.cc:333] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.063081 18297 raft_consensus.cc:493] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:44.064118  5137 raft_consensus.cc:2243] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.064050 18297 raft_consensus.cc:515] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } } peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } }
I20260709 12:30:44.065173  5137 raft_consensus.cc:2272] T 0af900dfc2784f8781502931a17eaa79 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.066843 18297 leader_election.cc:290] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625), c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307)
I20260709 12:30:44.067845  5137 tablet_replica.cc:333] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.068689  5137 raft_consensus.cc:2243] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.069197  5137 raft_consensus.cc:2272] T 9614ba0ca1894975b6696b3724157ee6 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.072206  5137 tablet_replica.cc:333] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
W20260709 12:30:44.072363 17308 leader_election.cc:336] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
I20260709 12:30:44.072894  5137 raft_consensus.cc:2243] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:44.073271 17307 leader_election.cc:336] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:44.073522  5137 raft_consensus.cc:2272] T 1711901c61714bc1bbb4f2bec64dadb3 P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.073742 17307 leader_election.cc:304] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:44.074611 18297 raft_consensus.cc:2749] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:44.076143  5137 tablet_replica.cc:333] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.076732  5137 raft_consensus.cc:2243] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.077188  5137 raft_consensus.cc:2272] T 058259c8483642bba034c3bd3f5ba90e P 6c1eb70ad94940b89118a32a370a4310 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.079322  5137 tablet_replica.cc:333] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.079824  5137 raft_consensus.cc:2243] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.080274  5137 raft_consensus.cc:2272] T 09bd2f3de39744c28ad21f75b37293b9 P 6c1eb70ad94940b89118a32a370a4310 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.082479  5137 tablet_replica.cc:333] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310: stopping tablet replica
I20260709 12:30:44.083017  5137 raft_consensus.cc:2243] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.083737  5137 raft_consensus.cc:2272] T e9b921f32d04442e89c1b7ea73df020f P 6c1eb70ad94940b89118a32a370a4310 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.131170 18297 raft_consensus.cc:493] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:44.131990 18297 raft_consensus.cc:515] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:44.134914 18297 leader_election.cc:290] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:44.140635 17307 leader_election.cc:336] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:30:44.142825 17307 leader_election.cc:336] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:44.143275 17307 leader_election.cc:304] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:44.144079 18223 raft_consensus.cc:2749] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:44.168067  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
W20260709 12:30:44.215919 17307 consensus_peers.cc:597] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.67:43265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:44.231623 18223 raft_consensus.cc:493] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:44.232271 18223 raft_consensus.cc:515] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:44.235610 18223 leader_election.cc:290] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:44.240618 17308 leader_election.cc:336] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
W20260709 12:30:44.241429 17307 leader_election.cc:336] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:44.241871 17307 leader_election.cc:304] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 79e92a05527f4a0b98b8352cb68a1010, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:44.242686 18223 raft_consensus.cc:2749] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:30:44.267175 18223 raft_consensus.cc:493] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:44.267848 18223 raft_consensus.cc:515] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:44.269893 18209 raft_consensus.cc:493] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 79e92a05527f4a0b98b8352cb68a1010)
I20260709 12:30:44.270668 18209 raft_consensus.cc:515] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } } peers { permanent_uuid: "79e92a05527f4a0b98b8352cb68a1010" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34625 } }
I20260709 12:30:44.272984 18209 leader_election.cc:290] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
I20260709 12:30:44.282600  5137 tablet_server.cc:179] TabletServer@127.5.4.68:0 shutting down...
I20260709 12:30:44.292567 18223 leader_election.cc:290] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265), 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625)
W20260709 12:30:44.296146 17307 leader_election.cc:336] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Network error: Client connection negotiation failed: client connection to 127.5.4.67:43265: connect: Connection refused (error 111)
W20260709 12:30:44.296726 17307 leader_election.cc:336] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Network error: Client connection negotiation failed: client connection to 127.5.4.67:43265: connect: Connection refused (error 111)
W20260709 12:30:44.300990 17307 leader_election.cc:336] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:44.301477 17307 leader_election.cc:304] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:44.302361 18313 raft_consensus.cc:2749] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:30:44.302560 17307 leader_election.cc:336] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 79e92a05527f4a0b98b8352cb68a1010 (127.5.4.65:34625): Network error: Client connection negotiation failed: client connection to 127.5.4.65:34625: connect: Connection refused (error 111)
I20260709 12:30:44.303289 17307 leader_election.cc:304] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, 79e92a05527f4a0b98b8352cb68a1010
I20260709 12:30:44.304117 18313 raft_consensus.cc:2749] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:30:44.321501 17307 consensus_peers.cc:597] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a -> Peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Couldn't send request to peer 6c1eb70ad94940b89118a32a370a4310. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.67:43265: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:44.362846  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:44.364002  5137 tablet_replica.cc:333] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.365176  5137 raft_consensus.cc:2243] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.366460  5137 raft_consensus.cc:2272] T d14098588e6e4421bbf71e16dea54da0 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.369460  5137 tablet_replica.cc:333] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.370097  5137 raft_consensus.cc:2243] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.370656  5137 raft_consensus.cc:2272] T e58121712c0045238ff325078dcf0653 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.373149  5137 tablet_replica.cc:333] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.373767  5137 raft_consensus.cc:2243] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.374599  5137 raft_consensus.cc:2272] T bc38664bb0494fe080bd97ce12c2481b P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.377553  5137 tablet_replica.cc:333] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.378265  5137 raft_consensus.cc:2243] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.379002  5137 raft_consensus.cc:2272] T 0b709ddd01784d588100b08085474cd7 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.382069  5137 tablet_replica.cc:333] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.382997  5137 raft_consensus.cc:2243] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.384166  5137 raft_consensus.cc:2272] T b7ffe1eaf86b4aa8bfa40bb0defe1cfd P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.387043  5137 tablet_replica.cc:333] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.387943  5137 raft_consensus.cc:2243] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.389068  5137 raft_consensus.cc:2272] T 458688757584439eb735f41c9b0e9eaa P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.391423  5137 tablet_replica.cc:333] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.392019  5137 raft_consensus.cc:2243] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.392912  5137 raft_consensus.cc:2272] T 2f61a82e0eef4a83ad047d6dd2ca75f8 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.395318  5137 tablet_replica.cc:333] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.395905  5137 raft_consensus.cc:2243] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.396430  5137 raft_consensus.cc:2272] T 419b4941c01a4ffd8c32d71f505b8cde P 7c0f44cba62f491392d41ca89376139a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.398873  5137 tablet_replica.cc:333] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.399497  5137 raft_consensus.cc:2243] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.399978  5137 raft_consensus.cc:2272] T d5b31c8286684684ae24797a44fc5403 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.402298  5137 tablet_replica.cc:333] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.402967  5137 raft_consensus.cc:2243] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.403432  5137 raft_consensus.cc:2272] T 90bb36eac77e43ad8fc0568e85913a36 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.405757  5137 tablet_replica.cc:333] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.406375  5137 raft_consensus.cc:2243] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.406894  5137 raft_consensus.cc:2272] T ce19f767e060412c8bc15f391d74b87f P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.409310  5137 tablet_replica.cc:333] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.409921  5137 raft_consensus.cc:2243] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.410842  5137 raft_consensus.cc:2272] T 6ec669aa1bf9434f9b198f6d8e768348 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.413183  5137 tablet_replica.cc:333] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.413789  5137 raft_consensus.cc:2243] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.414695  5137 raft_consensus.cc:2272] T 80ed704c39774ac88e3d32548f26b242 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.416978  5137 tablet_replica.cc:333] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.417596  5137 raft_consensus.cc:2243] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.418032  5137 raft_consensus.cc:2272] T b6a9d0d27df74b748881bc3ef8245a62 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.420895  5137 tablet_replica.cc:333] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.421587  5137 raft_consensus.cc:2243] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.422152  5137 raft_consensus.cc:2272] T 2d81d3402a7848f886f20c683e7874f3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.425024  5137 tablet_replica.cc:333] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.425683  5137 raft_consensus.cc:2243] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.426594  5137 raft_consensus.cc:2272] T 56a32f727d4a47368863c010ce8508c8 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.429734  5137 tablet_replica.cc:333] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.430402  5137 raft_consensus.cc:2243] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.430985  5137 raft_consensus.cc:2272] T 13be6c315f194afc80d89f01777e105a P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.433384  5137 tablet_replica.cc:333] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.434016  5137 raft_consensus.cc:2243] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.434577  5137 raft_consensus.cc:2272] T d85bc0ab53bf4a66af3a1b42ff166216 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.436919  5137 tablet_replica.cc:333] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.437503  5137 raft_consensus.cc:2243] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.437958  5137 raft_consensus.cc:2272] T 35817ffe9ce34abd82d949dc55a8ae78 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.440357  5137 tablet_replica.cc:333] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.440960  5137 raft_consensus.cc:2243] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.441416  5137 raft_consensus.cc:2272] T 27bba673d23842c790eb0140a01cee0c P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.443771  5137 tablet_replica.cc:333] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.444373  5137 raft_consensus.cc:2243] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.444830  5137 raft_consensus.cc:2272] T d62bfda640da40f9ab6e7b1da9479a5c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.447686  5137 tablet_replica.cc:333] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.448347  5137 raft_consensus.cc:2243] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.448757  5137 raft_consensus.cc:2272] T 795b495c58894c1eb861e953546a6e6d P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.451395  5137 tablet_replica.cc:333] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.451915  5137 raft_consensus.cc:2243] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.452415  5137 raft_consensus.cc:2272] T 1ca4d4100a154e058fd6b5525d3e280f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.454934  5137 tablet_replica.cc:333] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.455487  5137 raft_consensus.cc:2243] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.455884  5137 raft_consensus.cc:2272] T ec4e7b8f0ed341e0b42e5d3872c70010 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.458117  5137 tablet_replica.cc:333] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.458722  5137 raft_consensus.cc:2243] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.459609  5137 raft_consensus.cc:2272] T 47941e6ecfe34dfeb4aeb7f68e020eb5 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.461829  5137 tablet_replica.cc:333] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.462466  5137 raft_consensus.cc:2243] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.463014  5137 raft_consensus.cc:2272] T 9d2404ce16f84a8a9429e9b50a14d97b P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.465184  5137 tablet_replica.cc:333] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.465744  5137 raft_consensus.cc:2243] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.466650  5137 raft_consensus.cc:2272] T 51de8b9a56eb439e9c25052d3d9587f9 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.468842  5137 tablet_replica.cc:333] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.469954  5137 raft_consensus.cc:2243] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.470886  5137 raft_consensus.cc:2272] T ec4250a950114da1bce4b46571dc97cc P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.473143  5137 tablet_replica.cc:333] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.473721  5137 raft_consensus.cc:2243] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.474184  5137 raft_consensus.cc:2272] T f9af739be394434c9d30f15785e5e692 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.476608  5137 tablet_replica.cc:333] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.477217  5137 raft_consensus.cc:2243] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.477640  5137 raft_consensus.cc:2272] T e9b921f32d04442e89c1b7ea73df020f P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.479902  5137 tablet_replica.cc:333] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.480468  5137 raft_consensus.cc:2243] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.480944  5137 raft_consensus.cc:2272] T fb995bc24cf54980b8977e69501ac133 P 7c0f44cba62f491392d41ca89376139a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.483673  5137 tablet_replica.cc:333] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.484342  5137 raft_consensus.cc:2243] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.485257  5137 raft_consensus.cc:2272] T 9ba13b6455354f64a81d0be356cf1982 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.487833  5137 tablet_replica.cc:333] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.488440  5137 raft_consensus.cc:2243] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.488911  5137 raft_consensus.cc:2272] T 675c55f0ab604b0cb4f3205d8bedf323 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.491266  5137 tablet_replica.cc:333] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.491834  5137 raft_consensus.cc:2243] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.492336  5137 raft_consensus.cc:2272] T 0af900dfc2784f8781502931a17eaa79 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.494710  5137 tablet_replica.cc:333] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.495435  5137 raft_consensus.cc:2243] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.495437 18313 raft_consensus.cc:493] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c75b62c746bd4d90a28483bc9d5e3c84)
I20260709 12:30:44.496485  5137 raft_consensus.cc:2272] T 55a24caeb8124f77a6a9d21f8bed1831 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.496449 18313 raft_consensus.cc:515] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c75b62c746bd4d90a28483bc9d5e3c84" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44307 } } peers { permanent_uuid: "6c1eb70ad94940b89118a32a370a4310" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 43265 } } peers { permanent_uuid: "7c0f44cba62f491392d41ca89376139a" member_type: VOTER last_known_addr { host: "127.5.4.68" port: 38585 } }
I20260709 12:30:44.499387 18313 leader_election.cc:290] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307), 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265)
I20260709 12:30:44.499807  5137 tablet_replica.cc:333] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.500713  5137 raft_consensus.cc:2243] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.501389  5137 raft_consensus.cc:2272] T 0148e46ec1b2484e902b01edfda2976c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.504660  5137 tablet_replica.cc:333] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
W20260709 12:30:44.505005 17307 leader_election.cc:336] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6c1eb70ad94940b89118a32a370a4310 (127.5.4.67:43265): Network error: Client connection negotiation failed: client connection to 127.5.4.67:43265: connect: Connection refused (error 111)
I20260709 12:30:44.505481  5137 raft_consensus.cc:2243] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:30:44.505656 17308 leader_election.cc:336] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c75b62c746bd4d90a28483bc9d5e3c84 (127.5.4.66:44307): Network error: Client connection negotiation failed: client connection to 127.5.4.66:44307: connect: Connection refused (error 111)
I20260709 12:30:44.506232 17308 leader_election.cc:304] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c0f44cba62f491392d41ca89376139a; no voters: 6c1eb70ad94940b89118a32a370a4310, c75b62c746bd4d90a28483bc9d5e3c84
I20260709 12:30:44.506608  5137 raft_consensus.cc:2272] T a100ea0ba8a74d14bdfc772d6059f39c P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.507085 18313 raft_consensus.cc:2749] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:30:44.509805  5137 tablet_replica.cc:333] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.510681  5137 raft_consensus.cc:2243] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.511771  5137 raft_consensus.cc:2272] T 44c804e5f1864c799cd5d1d49734db98 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.514299  5137 tablet_replica.cc:333] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.514889  5137 raft_consensus.cc:2243] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.515343  5137 raft_consensus.cc:2272] T ecf98a8bc05f404f8fb4149773b1c384 P 7c0f44cba62f491392d41ca89376139a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.517607  5137 tablet_replica.cc:333] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.518150  5137 raft_consensus.cc:2243] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.519042  5137 raft_consensus.cc:2272] T 60f10e5fb23f4dc08b18d7e805d0a0f4 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.521203  5137 tablet_replica.cc:333] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.521744  5137 raft_consensus.cc:2243] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.522734  5137 raft_consensus.cc:2272] T 9614ba0ca1894975b6696b3724157ee6 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.525048  5137 tablet_replica.cc:333] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.525563  5137 raft_consensus.cc:2243] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.526017  5137 raft_consensus.cc:2272] T 1711901c61714bc1bbb4f2bec64dadb3 P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.528287  5137 tablet_replica.cc:333] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.528798  5137 raft_consensus.cc:2243] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.529268  5137 raft_consensus.cc:2272] T 440c7aeff35f4f3988c3eebfef1c6acf P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.531690  5137 tablet_replica.cc:333] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a: stopping tablet replica
I20260709 12:30:44.532176  5137 raft_consensus.cc:2243] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:44.532616  5137 raft_consensus.cc:2272] T fd26acf9f75043088e62624ad9f61c2e P 7c0f44cba62f491392d41ca89376139a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.620271  5137 tablet_server.cc:196] TabletServer@127.5.4.68:0 shutdown complete.
I20260709 12:30:44.725857  5137 master.cc:562] Master@127.5.4.126:38383 shutting down...
I20260709 12:30:44.772666  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:44.773481  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:44.773968  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P ee328dfad2034db3a72398b337bc4cb7: stopping tablet replica
I20260709 12:30:45.319533  5137 master.cc:584] Master@127.5.4.126:38383 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (36755 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:30:45.423188  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:44499
I20260709 12:30:45.424744  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:45.432384 18315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:45.432493 18314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:45.432432 18317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:45.433907  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:45.435276  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:45.435482  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:45.435678  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600245435660 us; error 0 us; skew 500 ppm
I20260709 12:30:45.436331  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:45.439791  5137 webserver.cc:533] Webserver started at http://127.5.4.126:43875/ using document root <none> and password file <none>
I20260709 12:30:45.440380  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:45.440580  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:45.440902  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:45.442229  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "071688f3f2324b91baab385d069c2681"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.448472  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:30:45.452298 18322 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.453289  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:30:45.453645  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "071688f3f2324b91baab385d069c2681"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.454010  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:45.487434  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:45.488802  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:45.532981  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:44499
I20260709 12:30:45.533134 18373 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:44499 every 8 connection(s)
I20260709 12:30:45.537411 18374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:45.549427 18374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681: Bootstrap starting.
I20260709 12:30:45.555135 18374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:45.560453 18374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681: No bootstrap required, opened a new log
I20260709 12:30:45.563103 18374 raft_consensus.cc:359] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071688f3f2324b91baab385d069c2681" member_type: VOTER }
I20260709 12:30:45.563561 18374 raft_consensus.cc:385] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:45.563824 18374 raft_consensus.cc:740] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 071688f3f2324b91baab385d069c2681, State: Initialized, Role: FOLLOWER
I20260709 12:30:45.564546 18374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071688f3f2324b91baab385d069c2681" member_type: VOTER }
I20260709 12:30:45.565107 18374 raft_consensus.cc:399] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:45.565357 18374 raft_consensus.cc:493] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:45.565632 18374 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:45.571089 18374 raft_consensus.cc:515] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071688f3f2324b91baab385d069c2681" member_type: VOTER }
I20260709 12:30:45.571672 18374 leader_election.cc:304] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 071688f3f2324b91baab385d069c2681; no voters: 
I20260709 12:30:45.573014 18374 leader_election.cc:290] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:45.573415 18377 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:45.575057 18377 raft_consensus.cc:697] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 1 LEADER]: Becoming Leader. State: Replica: 071688f3f2324b91baab385d069c2681, State: Running, Role: LEADER
I20260709 12:30:45.575744 18377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071688f3f2324b91baab385d069c2681" member_type: VOTER }
I20260709 12:30:45.576462 18374 sys_catalog.cc:565] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:45.580400 18378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "071688f3f2324b91baab385d069c2681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071688f3f2324b91baab385d069c2681" member_type: VOTER } }
I20260709 12:30:45.580617 18379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 071688f3f2324b91baab385d069c2681. Latest consensus state: current_term: 1 leader_uuid: "071688f3f2324b91baab385d069c2681" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "071688f3f2324b91baab385d069c2681" member_type: VOTER } }
I20260709 12:30:45.581605 18379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:45.581218 18378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:45.589934 18383 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:45.595106 18383 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:45.599509  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:45.605449 18383 catalog_manager.cc:1383] Generated new cluster ID: 8dd390015c3649fdbf024b0da3d486bf
I20260709 12:30:45.605772 18383 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:45.647300 18383 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:45.648644 18383 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:45.663903 18383 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681: Generated new TSK 0
I20260709 12:30:45.664520 18383 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:45.731716  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:45.738622 18395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:45.739971 18396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:45.741760  5137 server_base.cc:1061] running on GCE node
W20260709 12:30:45.741874 18398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:45.743400  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:45.743610  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:45.743832  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600245743815 us; error 0 us; skew 500 ppm
I20260709 12:30:45.744453  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:45.747138  5137 webserver.cc:533] Webserver started at http://127.5.4.65:43115/ using document root <none> and password file <none>
I20260709 12:30:45.747671  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:45.747879  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:45.748184  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:45.749402  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "dede9b011d384c4896611658022c757a"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.754175  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.005s
I20260709 12:30:45.758440 18403 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.759565  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:45.759970  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "dede9b011d384c4896611658022c757a"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.760263  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:45.787065  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:45.788488  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:45.790261  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:45.792856  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:45.793088  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.793376  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:45.793603  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.845724  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:46627
I20260709 12:30:45.845896 18465 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:46627 every 8 connection(s)
I20260709 12:30:45.851716  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:45.859390 18470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:45.861119 18471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:45.864328 18473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:45.864568  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:45.865756  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:45.865945  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:45.866163  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600245866148 us; error 0 us; skew 500 ppm
I20260709 12:30:45.867038  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:45.869932  5137 webserver.cc:533] Webserver started at http://127.5.4.66:43967/ using document root <none> and password file <none>
I20260709 12:30:45.870591  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:45.870812  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:45.871133  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:45.871673 18466 heartbeater.cc:344] Connected to a master server at 127.5.4.126:44499
I20260709 12:30:45.872231 18466 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:45.872383  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "1396e5332bb14f96bb184162cf048329"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.873133 18466 heartbeater.cc:507] Master 127.5.4.126:44499 requested a full tablet report, sending...
I20260709 12:30:45.875694 18339 ts_manager.cc:194] Registered new tserver with Master: dede9b011d384c4896611658022c757a (127.5.4.65:46627)
I20260709 12:30:45.878190  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:30:45.878295 18339 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47044
I20260709 12:30:45.882709 18478 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.883721  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:45.884016  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "1396e5332bb14f96bb184162cf048329"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.884286  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:45.903350  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:45.904685  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:45.906306  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:45.908872  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:45.909108  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.909370  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:45.909566  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.957937  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:41163
I20260709 12:30:45.958127 18540 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:41163 every 8 connection(s)
I20260709 12:30:45.963581  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:45.971261 18544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:45.973448 18545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:45.973925 18541 heartbeater.cc:344] Connected to a master server at 127.5.4.126:44499
I20260709 12:30:45.974637 18541 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:45.975927 18541 heartbeater.cc:507] Master 127.5.4.126:44499 requested a full tablet report, sending...
W20260709 12:30:45.977061 18547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:45.977175  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:45.978893  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:45.979208  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:45.979013 18339 ts_manager.cc:194] Registered new tserver with Master: 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:45.979563  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600245979544 us; error 0 us; skew 500 ppm
I20260709 12:30:45.980334  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:45.980983 18339 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47056
I20260709 12:30:45.984055  5137 webserver.cc:533] Webserver started at http://127.5.4.67:37301/ using document root <none> and password file <none>
I20260709 12:30:45.984728  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:45.984984  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:45.985287  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:45.986639  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "9972f1b44e7742beb5b7386e736ead4f"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.991700  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:30:45.995623 18552 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:45.996603  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:30:45.996891  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "9972f1b44e7742beb5b7386e736ead4f"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:45.997179  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:46.012166  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:46.013327  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:46.014986  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:46.017401  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:46.017601  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:46.017889  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:46.018057  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:46.062537  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:40539
I20260709 12:30:46.062678 18614 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:40539 every 8 connection(s)
I20260709 12:30:46.077519 18615 heartbeater.cc:344] Connected to a master server at 127.5.4.126:44499
I20260709 12:30:46.078017 18615 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:46.078951 18615 heartbeater.cc:507] Master 127.5.4.126:44499 requested a full tablet report, sending...
I20260709 12:30:46.081048 18339 ts_manager.cc:194] Registered new tserver with Master: 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:46.081867  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015849957s
I20260709 12:30:46.082582 18339 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47070
I20260709 12:30:46.112522 18339 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47072:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:46.115533 18339 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:46.194693 18580 tablet_service.cc:1511] Processing CreateTablet for tablet f87a4a1efe584b8e9c9a9b74fb367c40 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:30:46.196446 18580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87a4a1efe584b8e9c9a9b74fb367c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.206633 18579 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8ae5b9817d423fb60455f04d1bcc8d (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:30:46.208359 18579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8ae5b9817d423fb60455f04d1bcc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.210426 18578 tablet_service.cc:1511] Processing CreateTablet for tablet c9000dd337d14f7ba6de5fc47003bac4 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:30:46.215382 18574 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf4422aa7554e08855d77908f99763c (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:30:46.215551 18430 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8ae5b9817d423fb60455f04d1bcc8d (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:30:46.216382 18578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9000dd337d14f7ba6de5fc47003bac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.217195 18430 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8ae5b9817d423fb60455f04d1bcc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.217923 18429 tablet_service.cc:1511] Processing CreateTablet for tablet c9000dd337d14f7ba6de5fc47003bac4 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:30:46.219336 18429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9000dd337d14f7ba6de5fc47003bac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.211421 18573 tablet_service.cc:1511] Processing CreateTablet for tablet cb246f39e3c04375bb0364916c6c82cb (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:30:46.221885 18573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb246f39e3c04375bb0364916c6c82cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.211588 18577 tablet_service.cc:1511] Processing CreateTablet for tablet feb8927db5a34050961c0c096ca6a03c (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:30:46.224130 18577 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feb8927db5a34050961c0c096ca6a03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.212730 18576 tablet_service.cc:1511] Processing CreateTablet for tablet 08ff9c1df5704377bf35c7804397f431 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:30:46.228040 18576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ff9c1df5704377bf35c7804397f431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.213699 18572 tablet_service.cc:1511] Processing CreateTablet for tablet 9f58e58d872a43ba84789c32783e4a74 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:30:46.231712 18572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f58e58d872a43ba84789c32783e4a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.214039 18575 tablet_service.cc:1511] Processing CreateTablet for tablet b4f15bd8832b4b39abfaec4c976b8ec1 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:30:46.235206 18428 tablet_service.cc:1511] Processing CreateTablet for tablet feb8927db5a34050961c0c096ca6a03c (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:30:46.237044 18428 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feb8927db5a34050961c0c096ca6a03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.239595 18575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4f15bd8832b4b39abfaec4c976b8ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.247072 18574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf4422aa7554e08855d77908f99763c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.249114 18427 tablet_service.cc:1511] Processing CreateTablet for tablet 08ff9c1df5704377bf35c7804397f431 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:30:46.214511 18431 tablet_service.cc:1511] Processing CreateTablet for tablet f87a4a1efe584b8e9c9a9b74fb367c40 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:30:46.251279 18427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ff9c1df5704377bf35c7804397f431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.252638 18431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87a4a1efe584b8e9c9a9b74fb367c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.255534 18426 tablet_service.cc:1511] Processing CreateTablet for tablet b4f15bd8832b4b39abfaec4c976b8ec1 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:30:46.258381 18426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4f15bd8832b4b39abfaec4c976b8ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.294227 18425 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf4422aa7554e08855d77908f99763c (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:30:46.296314 18425 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf4422aa7554e08855d77908f99763c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.300103 18503 tablet_service.cc:1511] Processing CreateTablet for tablet feb8927db5a34050961c0c096ca6a03c (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:30:46.302251 18503 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet feb8927db5a34050961c0c096ca6a03c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.321376 18422 tablet_service.cc:1511] Processing CreateTablet for tablet dbeedc48c0f04eb4b10f9df20f06695a (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:30:46.304195 18502 tablet_service.cc:1511] Processing CreateTablet for tablet 08ff9c1df5704377bf35c7804397f431 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:30:46.323614 18422 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbeedc48c0f04eb4b10f9df20f06695a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.323678 18502 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 08ff9c1df5704377bf35c7804397f431. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.308684 18506 tablet_service.cc:1511] Processing CreateTablet for tablet f87a4a1efe584b8e9c9a9b74fb367c40 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:30:46.327836 18506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f87a4a1efe584b8e9c9a9b74fb367c40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.309260 18424 tablet_service.cc:1511] Processing CreateTablet for tablet cb246f39e3c04375bb0364916c6c82cb (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:30:46.331758 18424 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb246f39e3c04375bb0364916c6c82cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.333336 18571 tablet_service.cc:1511] Processing CreateTablet for tablet dbeedc48c0f04eb4b10f9df20f06695a (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:30:46.311278 18500 tablet_service.cc:1511] Processing CreateTablet for tablet 0bf4422aa7554e08855d77908f99763c (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:30:46.335315 18571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbeedc48c0f04eb4b10f9df20f06695a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.335752 18500 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bf4422aa7554e08855d77908f99763c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.312681 18423 tablet_service.cc:1511] Processing CreateTablet for tablet 9f58e58d872a43ba84789c32783e4a74 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:30:46.339658 18423 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f58e58d872a43ba84789c32783e4a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.296080 18505 tablet_service.cc:1511] Processing CreateTablet for tablet 3a8ae5b9817d423fb60455f04d1bcc8d (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:30:46.344396 18505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a8ae5b9817d423fb60455f04d1bcc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.313205 18501 tablet_service.cc:1511] Processing CreateTablet for tablet b4f15bd8832b4b39abfaec4c976b8ec1 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:30:46.347744 18501 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4f15bd8832b4b39abfaec4c976b8ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.315757 18499 tablet_service.cc:1511] Processing CreateTablet for tablet cb246f39e3c04375bb0364916c6c82cb (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:30:46.351763 18499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cb246f39e3c04375bb0364916c6c82cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.318837 18498 tablet_service.cc:1511] Processing CreateTablet for tablet 9f58e58d872a43ba84789c32783e4a74 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:30:46.355701 18498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f58e58d872a43ba84789c32783e4a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.298184 18504 tablet_service.cc:1511] Processing CreateTablet for tablet c9000dd337d14f7ba6de5fc47003bac4 (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:30:46.363600 18504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9000dd337d14f7ba6de5fc47003bac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.372845 18497 tablet_service.cc:1511] Processing CreateTablet for tablet dbeedc48c0f04eb4b10f9df20f06695a (DEFAULT_TABLE table=test-workload [id=4ced29d5bedd475dbea9e554d5f61065]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:30:46.375156 18497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbeedc48c0f04eb4b10f9df20f06695a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:46.510138 18631 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:46.518744 18631 tablet_bootstrap.cc:654] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.528225 18630 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.550477 18630 tablet_bootstrap.cc:654] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.555341 18633 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:46.557056 18631 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:46.557719 18631 ts_tablet_manager.cc:1403] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.020s
I20260709 12:30:46.560811 18631 raft_consensus.cc:359] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.561465 18631 raft_consensus.cc:385] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.562167 18631 raft_consensus.cc:740] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.563428 18631 consensus_queue.cc:260] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.569296 18466 heartbeater.cc:499] Master 127.5.4.126:44499 was elected leader, sending a full tablet report...
I20260709 12:30:46.570128 18631 ts_tablet_manager.cc:1434] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:30:46.570415 18630 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:46.571128 18630 ts_tablet_manager.cc:1403] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.043s	user 0.014s	sys 0.004s
I20260709 12:30:46.571266 18631 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:46.574690 18630 raft_consensus.cc:359] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.575591 18630 raft_consensus.cc:385] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.575980 18630 raft_consensus.cc:740] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.577090 18630 consensus_queue.cc:260] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.577610 18633 tablet_bootstrap.cc:654] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.578559 18631 tablet_bootstrap.cc:654] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.587689 18630 ts_tablet_manager.cc:1434] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.006s
I20260709 12:30:46.588992 18615 heartbeater.cc:499] Master 127.5.4.126:44499 was elected leader, sending a full tablet report...
W20260709 12:30:46.595094 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:30:46.601522 18467 tablet.cc:2406] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:30:46.611361 18630 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.611367 18633 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:46.612210 18631 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:46.612320 18633 ts_tablet_manager.cc:1403] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.057s	user 0.024s	sys 0.016s
I20260709 12:30:46.612937 18631 ts_tablet_manager.cc:1403] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.015s
I20260709 12:30:46.616034 18633 raft_consensus.cc:359] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.616824 18633 raft_consensus.cc:385] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.617108 18633 raft_consensus.cc:740] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.618005 18633 consensus_queue.cc:260] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.618067 18631 raft_consensus.cc:359] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:46.619081 18631 raft_consensus.cc:385] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.619410 18631 raft_consensus.cc:740] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.620288 18631 consensus_queue.cc:260] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:46.622967 18630 tablet_bootstrap.cc:654] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.626775 18541 heartbeater.cc:499] Master 127.5.4.126:44499 was elected leader, sending a full tablet report...
I20260709 12:30:46.627918 18633 ts_tablet_manager.cc:1434] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:30:46.629107 18633 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:46.629683 18631 ts_tablet_manager.cc:1434] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.016s	user 0.011s	sys 0.000s
I20260709 12:30:46.632813 18631 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:46.636277 18633 tablet_bootstrap.cc:654] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.640514 18631 tablet_bootstrap.cc:654] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.648027 18630 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:46.648095 18633 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:46.648887 18630 ts_tablet_manager.cc:1403] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.038s	user 0.017s	sys 0.005s
I20260709 12:30:46.649071 18633 ts_tablet_manager.cc:1403] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.020s	user 0.008s	sys 0.009s
I20260709 12:30:46.649317 18631 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:46.649921 18631 ts_tablet_manager.cc:1403] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:30:46.652463 18630 raft_consensus.cc:359] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.652773 18633 raft_consensus.cc:359] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.653395 18630 raft_consensus.cc:385] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.652961 18631 raft_consensus.cc:359] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.653643 18633 raft_consensus.cc:385] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.653792 18630 raft_consensus.cc:740] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.654062 18631 raft_consensus.cc:385] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.654106 18633 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.654727 18631 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.654901 18630 consensus_queue.cc:260] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.655581 18633 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.655889 18631 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.658342 18631 ts_tablet_manager.cc:1434] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:30:46.659458 18631 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:46.662824 18633 ts_tablet_manager.cc:1434] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.002s
I20260709 12:30:46.663957 18633 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:46.666699 18631 tablet_bootstrap.cc:654] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.670969 18633 tablet_bootstrap.cc:654] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.672336 18630 ts_tablet_manager.cc:1434] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.002s
I20260709 12:30:46.673583 18630 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.680830 18630 tablet_bootstrap.cc:654] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.685547 18631 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:46.686262 18631 ts_tablet_manager.cc:1403] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.020s
I20260709 12:30:46.686273 18633 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:46.687325 18633 ts_tablet_manager.cc:1403] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.007s
I20260709 12:30:46.689313 18631 raft_consensus.cc:359] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.690025 18631 raft_consensus.cc:385] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.690287 18631 raft_consensus.cc:740] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.691097 18631 consensus_queue.cc:260] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.691355 18633 raft_consensus.cc:359] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.692958 18633 raft_consensus.cc:385] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.693305 18631 ts_tablet_manager.cc:1434] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:30:46.693339 18633 raft_consensus.cc:740] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.694548 18631 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:46.694943 18633 consensus_queue.cc:260] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.696321 18630 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:46.696875 18630 ts_tablet_manager.cc:1403] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.017s
I20260709 12:30:46.700137 18630 raft_consensus.cc:359] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.700856 18630 raft_consensus.cc:385] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.701133 18630 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.701964 18630 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.702527 18631 tablet_bootstrap.cc:654] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.706635 18630 ts_tablet_manager.cc:1434] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:46.706918 18633 ts_tablet_manager.cc:1434] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.004s
I20260709 12:30:46.707746 18630 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.711519 18633 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:46.716099 18630 tablet_bootstrap.cc:654] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.717485 18631 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:46.718168 18631 ts_tablet_manager.cc:1403] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.024s	user 0.006s	sys 0.013s
I20260709 12:30:46.718744 18633 tablet_bootstrap.cc:654] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.721338 18635 raft_consensus.cc:493] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.721889 18635 raft_consensus.cc:515] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:46.724929 18633 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:46.725642 18633 ts_tablet_manager.cc:1403] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.005s
I20260709 12:30:46.729475 18630 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:46.729432 18633 raft_consensus.cc:359] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.729640 18631 raft_consensus.cc:359] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.730505 18633 raft_consensus.cc:385] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.730598 18630 ts_tablet_manager.cc:1403] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.003s
I20260709 12:30:46.730664 18631 raft_consensus.cc:385] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.731137 18633 raft_consensus.cc:740] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.731586 18631 raft_consensus.cc:740] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.732405 18633 consensus_queue.cc:260] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.732508 18631 consensus_queue.cc:260] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.735428 18633 ts_tablet_manager.cc:1434] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:30:46.736996 18633 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:46.739846 18635 leader_election.cc:290] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:46.743299 18635 raft_consensus.cc:493] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.743775 18635 raft_consensus.cc:515] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.746016 18635 leader_election.cc:290] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:46.746560 18633 tablet_bootstrap.cc:654] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.749109 18630 raft_consensus.cc:359] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.749944 18630 raft_consensus.cc:385] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.750344 18630 raft_consensus.cc:740] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.751302 18630 consensus_queue.cc:260] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.754321 18630 ts_tablet_manager.cc:1434] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.001s
I20260709 12:30:46.755759 18630 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.764138 18630 tablet_bootstrap.cc:654] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.779543 18633 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:46.780498 18633 ts_tablet_manager.cc:1403] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.044s	user 0.016s	sys 0.025s
I20260709 12:30:46.783260 18636 raft_consensus.cc:493] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.784077 18636 raft_consensus.cc:515] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.785099 18633 raft_consensus.cc:359] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.786725 18633 raft_consensus.cc:385] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.787168 18633 raft_consensus.cc:740] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.788657 18633 consensus_queue.cc:260] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.793992 18636 leader_election.cc:290] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:46.802874 18633 ts_tablet_manager.cc:1434] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 12:30:46.808055 18633 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:46.816387 18633 tablet_bootstrap.cc:654] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.819228 18636 raft_consensus.cc:493] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.820113 18636 raft_consensus.cc:515] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.823349 18636 leader_election.cc:290] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:46.860371 18635 raft_consensus.cc:493] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.861749 18630 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:46.861276 18635 raft_consensus.cc:515] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.862622 18630 ts_tablet_manager.cc:1403] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.107s	user 0.040s	sys 0.059s
I20260709 12:30:46.865245 18635 leader_election.cc:290] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:46.865871 18630 raft_consensus.cc:359] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.866228 18653 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.866775 18630 raft_consensus.cc:385] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.867149 18630 raft_consensus.cc:740] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.867180 18653 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.868115 18630 consensus_queue.cc:260] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.869056 18590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f87a4a1efe584b8e9c9a9b74fb367c40" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:46.870790 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:46.870950 18630 ts_tablet_manager.cc:1434] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.001s
I20260709 12:30:46.870994 18653 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:46.872201 18630 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.872187 18588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:46.879422 18630 tablet_bootstrap.cc:654] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:46.881788 18404 leader_election.cc:343] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:46.882731 18404 leader_election.cc:343] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:30:46.883561 18404 leader_election.cc:343] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.885206 18588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:46.886096 18588 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dede9b011d384c4896611658022c757a in term 0.
I20260709 12:30:46.887749 18404 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:46.888963 18653 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:46.889434 18653 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:46.889747 18653 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:46.891270 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:46.893487 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:46.893710 18630 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:46.894193 18441 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 0.
I20260709 12:30:46.894331 18630 ts_tablet_manager.cc:1403] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.005s
W20260709 12:30:46.895604 18553 leader_election.cc:343] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.896848 18553 leader_election.cc:304] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:46.898365 18636 raft_consensus.cc:2804] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:46.898269 18630 raft_consensus.cc:359] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.899034 18636 raft_consensus.cc:493] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:46.899293 18630 raft_consensus.cc:385] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.899533 18636 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:46.900224 18516 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f87a4a1efe584b8e9c9a9b74fb367c40" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:46.901648 18515 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:46.901978 18514 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
W20260709 12:30:46.905993 18405 leader_election.cc:343] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.901314 18630 raft_consensus.cc:740] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.902824 18513 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:46.906786 18405 leader_election.cc:304] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dede9b011d384c4896611658022c757a; no voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:46.903389 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:46.907998 18512 raft_consensus.cc:2468] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 0.
I20260709 12:30:46.903941 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
W20260709 12:30:46.908506 18405 leader_election.cc:343] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.909063 18511 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 0.
I20260709 12:30:46.909169 18405 leader_election.cc:304] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dede9b011d384c4896611658022c757a; no voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:46.910689 18635 raft_consensus.cc:2749] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:46.911332 18630 consensus_queue.cc:260] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.921717 18554 leader_election.cc:304] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:46.921016 18635 raft_consensus.cc:2749] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:46.921222 18405 leader_election.cc:343] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.925000 18636 raft_consensus.cc:515] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.925318 18405 leader_election.cc:304] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dede9b011d384c4896611658022c757a; no voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:46.925760 18631 ts_tablet_manager.cc:1434] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.207s	user 0.025s	sys 0.037s
I20260709 12:30:46.928159 18631 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:46.928805 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:46.929478 18441 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:46.930599 18653 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.934269 18588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:46.934945 18588 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:46.927179 18635 raft_consensus.cc:2749] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:30:46.937916 18405 leader_election.cc:343] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.938696 18441 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
I20260709 12:30:46.940212 18553 leader_election.cc:304] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:46.940593 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
W20260709 12:30:46.943485 18405 leader_election.cc:343] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:46.944362 18653 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 election: Requested vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:46.946231 18636 leader_election.cc:290] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:46.946717 18639 raft_consensus.cc:493] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:46.947474 18639 raft_consensus.cc:515] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.948894 18662 raft_consensus.cc:2804] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:46.949368 18662 raft_consensus.cc:493] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:46.949723 18662 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:46.949875 18636 raft_consensus.cc:2804] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:46.950428 18636 raft_consensus.cc:697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:46.950556 18633 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:46.951385 18633 ts_tablet_manager.cc:1403] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.144s	user 0.054s	sys 0.059s
I20260709 12:30:46.951383 18636 consensus_queue.cc:237] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.955260 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:46.956177 18511 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:46.988359 18588 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dede9b011d384c4896611658022c757a in term 1.
I20260709 12:30:46.988579 18630 ts_tablet_manager.cc:1434] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.094s	user 0.027s	sys 0.028s
I20260709 12:30:46.989629 18631 tablet_bootstrap.cc:654] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:46.990170 18404 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 1396e5332bb14f96bb184162cf048329
I20260709 12:30:46.990193 18630 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:46.991654 18653 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:46.992268 18653 raft_consensus.cc:697] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 LEADER]: Becoming Leader. State: Replica: dede9b011d384c4896611658022c757a, State: Running, Role: LEADER
I20260709 12:30:46.993230 18653 consensus_queue.cc:237] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.993269 18633 raft_consensus.cc:359] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:46.995410 18633 raft_consensus.cc:385] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:46.995756 18633 raft_consensus.cc:740] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:46.996603 18633 consensus_queue.cc:260] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.001802 18639 leader_election.cc:290] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.002267 18630 tablet_bootstrap.cc:654] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.004814 18662 raft_consensus.cc:515] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.019512 18633 ts_tablet_manager.cc:1434] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.068s	user 0.010s	sys 0.011s
I20260709 12:30:47.021289 18633 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:47.028252 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.028352 18662 leader_election.cc:290] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
W20260709 12:30:47.030975 18553 leader_election.cc:343] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.031406 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:47.032096 18512 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.036783 18633 tablet_bootstrap.cc:654] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.044888 18512 raft_consensus.cc:2468] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
I20260709 12:30:47.046525 18554 leader_election.cc:304] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:47.047304 18630 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:47.047866 18630 ts_tablet_manager.cc:1403] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.058s	user 0.019s	sys 0.013s
I20260709 12:30:47.047972 18662 raft_consensus.cc:2804] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.048545 18662 raft_consensus.cc:697] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:47.049492 18662 consensus_queue.cc:237] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.048849 18338 catalog_manager.cc:5697] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a reported cstate change: term changed from 0 to 1, leader changed from <none> to dede9b011d384c4896611658022c757a (127.5.4.65). New cstate: current_term: 1 leader_uuid: "dede9b011d384c4896611658022c757a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.051617 18630 raft_consensus.cc:359] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:47.053987 18630 raft_consensus.cc:385] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.055312 18630 raft_consensus.cc:740] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.056416 18630 consensus_queue.cc:260] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:47.056634 18337 catalog_manager.cc:5697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 1 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.060437 18630 ts_tablet_manager.cc:1434] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.002s
I20260709 12:30:47.065773 18668 raft_consensus.cc:493] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.066036 18630 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:47.066516 18668 raft_consensus.cc:515] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.064934 18511 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
I20260709 12:30:47.071029 18636 raft_consensus.cc:493] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.071570 18636 raft_consensus.cc:515] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.073884 18636 leader_election.cc:290] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:47.075639 18639 raft_consensus.cc:493] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.075816 18668 leader_election.cc:290] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.076231 18639 raft_consensus.cc:515] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.081316 18639 leader_election.cc:290] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.081241 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.078873 18631 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
W20260709 12:30:47.085827 18553 leader_election.cc:343] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.087525 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.087769 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.088603 18439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.088827 18440 raft_consensus.cc:2468] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 0.
W20260709 12:30:47.089884 18479 leader_election.cc:343] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.090761 18631 ts_tablet_manager.cc:1403] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.163s	user 0.027s	sys 0.037s
W20260709 12:30:47.090719 18479 leader_election.cc:343] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.091620 18479 leader_election.cc:304] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.092522 18639 raft_consensus.cc:2804] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.092901 18639 raft_consensus.cc:493] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.093230 18639 raft_consensus.cc:3060] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.094127 18631 raft_consensus.cc:359] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.094890 18631 raft_consensus.cc:385] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.095186 18631 raft_consensus.cc:740] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.088047 18633 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:47.096127 18633 ts_tablet_manager.cc:1403] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.075s	user 0.010s	sys 0.017s
I20260709 12:30:47.099332 18633 raft_consensus.cc:359] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.099985 18633 raft_consensus.cc:385] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.100286 18633 raft_consensus.cc:740] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.101074 18639 raft_consensus.cc:515] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.101132 18631 consensus_queue.cc:260] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.101351 18630 tablet_bootstrap.cc:654] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.104521 18631 ts_tablet_manager.cc:1434] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:30:47.105662 18631 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:47.088732 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
W20260709 12:30:47.107790 18554 leader_election.cc:343] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.108237 18554 leader_election.cc:304] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f; no voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a
I20260709 12:30:47.109863 18662 raft_consensus.cc:2749] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:47.111828 18588 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.112474 18631 tablet_bootstrap.cc:654] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.113451 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.113658 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.114027 18639 leader_election.cc:290] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.114183 18441 raft_consensus.cc:3060] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.114939 18590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.115657 18590 raft_consensus.cc:2468] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 0.
W20260709 12:30:47.118682 18479 leader_election.cc:343] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.119302 18479 leader_election.cc:304] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1396e5332bb14f96bb184162cf048329; no voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a
I20260709 12:30:47.120954 18479 leader_election.cc:304] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:47.121820 18441 raft_consensus.cc:2468] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:47.122044 18668 raft_consensus.cc:2749] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:47.123129 18668 raft_consensus.cc:2804] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.101395 18633 consensus_queue.cc:260] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.123701 18668 raft_consensus.cc:493] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.124150 18668 raft_consensus.cc:3060] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.125497 18633 ts_tablet_manager.cc:1434] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
W20260709 12:30:47.127372 18479 leader_election.cc:343] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.129158 18633 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:47.130157 18479 leader_election.cc:304] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.128444 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:47.134805 18679 raft_consensus.cc:2804] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.135372 18679 raft_consensus.cc:697] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:47.136287 18679 consensus_queue.cc:237] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.137010 18633 tablet_bootstrap.cc:654] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
W20260709 12:30:47.143163 18479 leader_election.cc:343] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.146076 18668 raft_consensus.cc:515] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.150216 18633 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:47.150856 18633 ts_tablet_manager.cc:1403] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
I20260709 12:30:47.151738 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.153098 18633 raft_consensus.cc:359] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
W20260709 12:30:47.153631 18479 leader_election.cc:343] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.153631 18633 raft_consensus.cc:385] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.154546 18633 raft_consensus.cc:740] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.155485 18633 consensus_queue.cc:260] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:47.157363 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:47.157963 18633 ts_tablet_manager.cc:1434] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:47.158104 18589 raft_consensus.cc:3060] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.159032 18633 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:47.161077 18336 catalog_manager.cc:5697] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 1 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.164729 18633 tablet_bootstrap.cc:654] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.165113 18631 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:47.165516 18662 raft_consensus.cc:493] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.165681 18631 ts_tablet_manager.cc:1403] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.006s
I20260709 12:30:47.166010 18662 raft_consensus.cc:515] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.168361 18662 leader_election.cc:290] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:47.168802 18668 leader_election.cc:290] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.168721 18631 raft_consensus.cc:359] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.169669 18631 raft_consensus.cc:385] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.169703 18589 raft_consensus.cc:2468] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:47.170011 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f15bd8832b4b39abfaec4c976b8ec1" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:47.171186 18511 raft_consensus.cc:2468] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 0.
I20260709 12:30:47.171263 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f15bd8832b4b39abfaec4c976b8ec1" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.171526 18479 leader_election.cc:304] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:47.170328 18631 raft_consensus.cc:740] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.171600 18630 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:47.172670 18554 leader_election.cc:304] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:47.172820 18630 ts_tablet_manager.cc:1403] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.110s	user 0.016s	sys 0.019s
I20260709 12:30:47.172945 18631 consensus_queue.cc:260] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.173627 18662 raft_consensus.cc:2804] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.174057 18662 raft_consensus.cc:493] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.174154 18441 raft_consensus.cc:2468] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 0.
I20260709 12:30:47.174453 18662 raft_consensus.cc:3060] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.176549 18668 raft_consensus.cc:2804] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.176316 18630 raft_consensus.cc:359] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.177111 18630 raft_consensus.cc:385] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.177407 18630 raft_consensus.cc:740] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.177469 18668 raft_consensus.cc:697] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:47.178359 18630 consensus_queue.cc:260] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.178653 18668 consensus_queue.cc:237] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.180905 18630 ts_tablet_manager.cc:1434] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:47.179978 18337 catalog_manager.cc:5697] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.182037 18630 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:47.182428 18662 raft_consensus.cc:515] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.186024 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f15bd8832b4b39abfaec4c976b8ec1" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:47.186666 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4f15bd8832b4b39abfaec4c976b8ec1" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.186731 18511 raft_consensus.cc:3060] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.187422 18441 raft_consensus.cc:3060] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.200949 18636 raft_consensus.cc:493] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.201783 18636 raft_consensus.cc:515] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.206329 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:47.207060 18512 raft_consensus.cc:2468] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 0.
I20260709 12:30:47.207260 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.207516 18633 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:47.208077 18554 leader_election.cc:304] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:47.208460 18633 ts_tablet_manager.cc:1403] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.050s	user 0.024s	sys 0.003s
I20260709 12:30:47.208720 18636 leader_election.cc:290] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:47.209069 18664 raft_consensus.cc:2804] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.209540 18664 raft_consensus.cc:493] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.209890 18664 raft_consensus.cc:3060] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:30:47.209976 18553 leader_election.cc:343] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.211457 18633 raft_consensus.cc:359] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.212441 18633 raft_consensus.cc:385] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.212777 18633 raft_consensus.cc:740] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.213770 18633 consensus_queue.cc:260] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.216607 18630 tablet_bootstrap.cc:654] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.216835 18633 ts_tablet_manager.cc:1434] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:30:47.218183 18633 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329: Bootstrap starting.
I20260709 12:30:47.223104 18664 raft_consensus.cc:515] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.225677 18633 tablet_bootstrap.cc:654] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.226984 18440 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.187525 18662 leader_election.cc:290] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:47.226320 18441 raft_consensus.cc:2468] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
W20260709 12:30:47.228595 18553 leader_election.cc:343] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:47.231086 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:47.231702 18512 raft_consensus.cc:3060] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.232621 18631 ts_tablet_manager.cc:1434] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.067s	user 0.005s	sys 0.005s
I20260709 12:30:47.236660 18631 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:47.248327 18512 raft_consensus.cc:2468] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
I20260709 12:30:47.249516 18631 tablet_bootstrap.cc:654] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.249723 18554 leader_election.cc:304] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:47.250689 18664 leader_election.cc:290] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:47.251343 18553 leader_election.cc:304] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.251612 18664 raft_consensus.cc:2804] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.252215 18662 raft_consensus.cc:2804] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.253530 18511 raft_consensus.cc:2468] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
I20260709 12:30:47.254031 18664 raft_consensus.cc:697] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:47.255033 18664 consensus_queue.cc:237] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.257704 18662 raft_consensus.cc:697] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:47.258759 18662 consensus_queue.cc:237] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.274168 18336 catalog_manager.cc:5697] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.276402 18633 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329: No bootstrap required, opened a new log
I20260709 12:30:47.277025 18633 ts_tablet_manager.cc:1403] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329: Time spent bootstrapping tablet: real 0.059s	user 0.024s	sys 0.019s
I20260709 12:30:47.276576 18337 catalog_manager.cc:5697] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 1 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.278309 18630 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:47.278899 18630 ts_tablet_manager.cc:1403] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.097s	user 0.012s	sys 0.001s
I20260709 12:30:47.280366 18633 raft_consensus.cc:359] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.281105 18633 raft_consensus.cc:385] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.281436 18633 raft_consensus.cc:740] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.281139 18630 raft_consensus.cc:359] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.282534 18630 raft_consensus.cc:385] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.282851 18630 raft_consensus.cc:740] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.282799 18633 consensus_queue.cc:260] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.283702 18631 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:47.284359 18631 ts_tablet_manager.cc:1403] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.048s	user 0.009s	sys 0.007s
I20260709 12:30:47.281977 18668 raft_consensus.cc:493] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.285262 18633 ts_tablet_manager.cc:1434] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:30:47.285084 18668 raft_consensus.cc:515] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.288643 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.289391 18441 raft_consensus.cc:2468] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 0.
I20260709 12:30:47.289664 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.290421 18631 raft_consensus.cc:359] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.290762 18479 leader_election.cc:304] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.291182 18631 raft_consensus.cc:385] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.291496 18631 raft_consensus.cc:740] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.291648 18668 leader_election.cc:290] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.284154 18630 consensus_queue.cc:260] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.292078 18679 raft_consensus.cc:2804] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.292285 18631 consensus_queue.cc:260] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.292646 18589 raft_consensus.cc:2468] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 0.
I20260709 12:30:47.294306 18630 ts_tablet_manager.cc:1434] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:30:47.292829 18679 raft_consensus.cc:493] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.294929 18679 raft_consensus.cc:3060] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.295454 18630 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f: Bootstrap starting.
I20260709 12:30:47.296967 18631 ts_tablet_manager.cc:1434] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:30:47.297981 18631 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:47.302476 18630 tablet_bootstrap.cc:654] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.304026 18631 tablet_bootstrap.cc:654] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.306635 18679 raft_consensus.cc:515] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.309654 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.310256 18441 raft_consensus.cc:3060] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.320348 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:47.321082 18589 raft_consensus.cc:3060] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.323275 18338 catalog_manager.cc:5697] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 1 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:47.329162 18589 raft_consensus.cc:2468] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:47.331689 18479 leader_election.cc:304] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:47.329911 18441 raft_consensus.cc:2468] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:47.330375 18679 leader_election.cc:290] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.332602 18668 raft_consensus.cc:2804] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.333477 18668 raft_consensus.cc:697] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:47.334784 18668 consensus_queue.cc:237] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.339228 18631 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:47.339857 18631 ts_tablet_manager.cc:1403] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.027s
I20260709 12:30:47.340656 18630 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f: No bootstrap required, opened a new log
I20260709 12:30:47.341240 18630 ts_tablet_manager.cc:1403] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.017s
I20260709 12:30:47.342734 18664 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:47.343721 18631 raft_consensus.cc:359] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.344538 18631 raft_consensus.cc:385] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.344386 18630 raft_consensus.cc:359] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.345119 18631 raft_consensus.cc:740] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.345237 18630 raft_consensus.cc:385] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.345818 18630 raft_consensus.cc:740] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.346570 18630 consensus_queue.cc:260] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.346428 18631 consensus_queue.cc:260] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.349084 18630 ts_tablet_manager.cc:1434] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:30:47.349097 18631 ts_tablet_manager.cc:1434] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:30:47.350637 18631 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:47.357061 18631 tablet_bootstrap.cc:654] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:47.362809 18337 catalog_manager.cc:5697] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.368993 18631 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: No bootstrap required, opened a new log
I20260709 12:30:47.369606 18631 ts_tablet_manager.cc:1403] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:30:47.372617 18631 raft_consensus.cc:359] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.373363 18631 raft_consensus.cc:385] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:47.373659 18631 raft_consensus.cc:740] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:47.374392 18631 consensus_queue.cc:260] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.376624 18631 ts_tablet_manager.cc:1434] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.001s
I20260709 12:30:47.377969 18662 raft_consensus.cc:493] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.378590 18662 raft_consensus.cc:515] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.380882 18662 leader_election.cc:290] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:47.382718 18653 raft_consensus.cc:493] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.383478 18653 raft_consensus.cc:515] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.384842 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:47.386605 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:47.387418 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.387939 18636 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.388275 18441 raft_consensus.cc:2393] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9972f1b44e7742beb5b7386e736ead4f in current term 1: Already voted for candidate 1396e5332bb14f96bb184162cf048329 in this term.
I20260709 12:30:47.386911 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.389670 18553 leader_election.cc:304] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f; no voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a
I20260709 12:30:47.390599 18589 raft_consensus.cc:2393] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dede9b011d384c4896611658022c757a in current term 1: Already voted for candidate 1396e5332bb14f96bb184162cf048329 in this term.
I20260709 12:30:47.391048 18636 raft_consensus.cc:3060] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.391667 18653 leader_election.cc:290] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.395270 18404 leader_election.cc:304] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dede9b011d384c4896611658022c757a; no voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:47.396423 18653 raft_consensus.cc:3060] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.400699 18636 raft_consensus.cc:2749] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:47.404304 18653 raft_consensus.cc:2749] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:47.416932 18512 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.423312 18653 consensus_queue.cc:1048] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.435330 18700 consensus_queue.cc:1048] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:47.448431 18668 raft_consensus.cc:493] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.448935 18668 raft_consensus.cc:515] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:47.450907 18668 leader_election.cc:290] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539), dede9b011d384c4896611658022c757a (127.5.4.65:46627)
I20260709 12:30:47.451788 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f87a4a1efe584b8e9c9a9b74fb367c40" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.451926 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f87a4a1efe584b8e9c9a9b74fb367c40" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a" is_pre_election: true
I20260709 12:30:47.452939 18441 raft_consensus.cc:2468] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 0.
I20260709 12:30:47.453609 18589 raft_consensus.cc:2468] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 0.
I20260709 12:30:47.454322 18479 leader_election.cc:304] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.455149 18668 raft_consensus.cc:2804] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.455566 18668 raft_consensus.cc:493] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.455952 18668 raft_consensus.cc:3060] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.460767 18668 raft_consensus.cc:515] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:47.462396 18668 leader_election.cc:290] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Requested vote from peers 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539), dede9b011d384c4896611658022c757a (127.5.4.65:46627)
I20260709 12:30:47.463338 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f87a4a1efe584b8e9c9a9b74fb367c40" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:47.463488 18441 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f87a4a1efe584b8e9c9a9b74fb367c40" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:47.464049 18589 raft_consensus.cc:3060] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.464085 18441 raft_consensus.cc:3060] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.471071 18589 raft_consensus.cc:2468] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:47.471073 18441 raft_consensus.cc:2468] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:47.472447 18479 leader_election.cc:304] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:47.473083 18668 raft_consensus.cc:2804] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.473556 18668 raft_consensus.cc:697] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:47.474372 18668 consensus_queue.cc:237] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:47.482143 18653 raft_consensus.cc:493] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.482106 18337 catalog_manager.cc:5697] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.482699 18653 raft_consensus.cc:515] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.485564 18653 leader_election.cc:290] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.486214 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.486014 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:47.486922 18589 raft_consensus.cc:2468] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dede9b011d384c4896611658022c757a in term 0.
I20260709 12:30:47.486971 18512 raft_consensus.cc:2468] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dede9b011d384c4896611658022c757a in term 0.
I20260709 12:30:47.488257 18405 leader_election.cc:304] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.489104 18653 raft_consensus.cc:2804] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:47.489645 18653 raft_consensus.cc:493] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:47.489986 18653 raft_consensus.cc:3060] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.494827 18441 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.497048 18653 raft_consensus.cc:515] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.499063 18653 leader_election.cc:290] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 election: Requested vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.499991 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:47.500607 18512 raft_consensus.cc:3060] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.500377 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:47.501225 18589 raft_consensus.cc:3060] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.502185 18636 consensus_queue.cc:1048] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.507879 18512 raft_consensus.cc:2468] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dede9b011d384c4896611658022c757a in term 1.
I20260709 12:30:47.509315 18405 leader_election.cc:304] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:47.509363 18589 raft_consensus.cc:2468] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dede9b011d384c4896611658022c757a in term 1.
I20260709 12:30:47.510360 18653 raft_consensus.cc:2804] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:47.510926 18653 raft_consensus.cc:697] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 LEADER]: Becoming Leader. State: Replica: dede9b011d384c4896611658022c757a, State: Running, Role: LEADER
I20260709 12:30:47.511896 18653 consensus_queue.cc:237] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.516474 18664 consensus_queue.cc:1048] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.523353 18336 catalog_manager.cc:5697] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a reported cstate change: term changed from 0 to 1, leader changed from <none> to dede9b011d384c4896611658022c757a (127.5.4.65). New cstate: current_term: 1 leader_uuid: "dede9b011d384c4896611658022c757a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:47.573593 18668 consensus_queue.cc:1048] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.587386 18668 consensus_queue.cc:1048] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:47.605616 18668 consensus_queue.cc:1048] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.621376 18639 consensus_queue.cc:1048] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.623970 18653 raft_consensus.cc:493] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:47.624536 18653 raft_consensus.cc:515] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:47.627838 18653 leader_election.cc:290] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:47.628147 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:47.628315 18590 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "dede9b011d384c4896611658022c757a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:47.629125 18512 raft_consensus.cc:2393] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate dede9b011d384c4896611658022c757a in current term 1: Already voted for candidate 9972f1b44e7742beb5b7386e736ead4f in this term.
I20260709 12:30:47.630196 18405 leader_election.cc:304] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dede9b011d384c4896611658022c757a; no voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:47.630956 18653 raft_consensus.cc:3060] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:47.640818 18653 raft_consensus.cc:2749] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:47.661278 18664 consensus_queue.cc:1048] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.676380 18662 consensus_queue.cc:1048] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:47.731194  5137 ts_manager.cc:295] Set tserver state for dede9b011d384c4896611658022c757a to MAINTENANCE_MODE
I20260709 12:30:47.732016  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
I20260709 12:30:47.756062  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:47.756924  5137 tablet_replica.cc:333] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.757548  5137 raft_consensus.cc:2243] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:47.758199  5137 pending_rounds.cc:70] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Trying to abort 1 pending ops.
I20260709 12:30:47.758527  5137 pending_rounds.cc:77] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626613827301376 op_type: NO_OP noop_request { }
I20260709 12:30:47.758860  5137 raft_consensus.cc:2889] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:30:47.759140  5137 raft_consensus.cc:2272] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.761662  5137 tablet_replica.cc:333] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.762253  5137 raft_consensus.cc:2243] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.762732  5137 raft_consensus.cc:2272] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.765278  5137 tablet_replica.cc:333] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.765789  5137 raft_consensus.cc:2243] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.766142  5137 raft_consensus.cc:2272] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.768575  5137 tablet_replica.cc:333] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.769063  5137 raft_consensus.cc:2243] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.769470  5137 raft_consensus.cc:2272] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.771875  5137 tablet_replica.cc:333] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.772459  5137 raft_consensus.cc:2243] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.772876  5137 raft_consensus.cc:2272] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.775127  5137 tablet_replica.cc:333] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.775803  5137 raft_consensus.cc:2243] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.776247  5137 raft_consensus.cc:2272] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.778676  5137 tablet_replica.cc:333] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.779213  5137 raft_consensus.cc:2243] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.779600  5137 raft_consensus.cc:2272] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.781317  5137 tablet_replica.cc:333] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.781828  5137 raft_consensus.cc:2243] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.782222  5137 raft_consensus.cc:2272] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.784138  5137 tablet_replica.cc:333] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.784572  5137 raft_consensus.cc:2243] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:47.784881  5137 raft_consensus.cc:2272] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:47.786860  5137 tablet_replica.cc:333] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: stopping tablet replica
I20260709 12:30:47.787798  5137 raft_consensus.cc:2243] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:47.788542  5137 raft_consensus.cc:2272] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:47.798780 18553 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 167 similar messages]
W20260709 12:30:47.802470 18553 consensus_peers.cc:597] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:47.820588  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
I20260709 12:30:47.864027 18639 consensus_queue.cc:1048] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:47.864027 18662 consensus_queue.cc:1048] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:30:47.885820 18479 consensus_peers.cc:597] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:47.955432 18553 consensus_peers.cc:597] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:48.053982 18479 consensus_peers.cc:597] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:48.080739 18639 consensus_queue.cc:1048] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:48.083813 18553 consensus_peers.cc:597] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:48.098866 18479 consensus_peers.cc:597] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:48.099014 18553 consensus_peers.cc:597] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:48.176467 18479 consensus_peers.cc:597] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:48.614428 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:48.935756 18722 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dede9b011d384c4896611658022c757a)
I20260709 12:30:48.936331 18722 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:48.938122 18722 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:48.938901 18711 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dede9b011d384c4896611658022c757a)
I20260709 12:30:48.939474 18711 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:48.940479 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:48.941283 18589 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:48.941692 18711 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:48.943303 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329" is_pre_election: true
I20260709 12:30:48.943945 18512 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 1.
W20260709 12:30:48.944339 18479 leader_election.cc:336] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:48.945151 18554 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:48.945297 18479 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:48.946003 18711 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:48.946435 18711 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting leader election (detected failure of leader dede9b011d384c4896611658022c757a)
I20260709 12:30:48.946456 18722 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:48.946967 18711 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:48.947060 18722 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting leader election (detected failure of leader dede9b011d384c4896611658022c757a)
W20260709 12:30:48.947314 18553 leader_election.cc:336] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:48.947754 18722 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:48.953174 18722 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:48.953855 18711 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:48.954813 18722 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:48.956070 18711 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:48.957121 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:48.957207 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:48.957973 18512 raft_consensus.cc:2393] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9972f1b44e7742beb5b7386e736ead4f in current term 2: Already voted for candidate 1396e5332bb14f96bb184162cf048329 in this term.
I20260709 12:30:48.958287 18589 raft_consensus.cc:2393] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1396e5332bb14f96bb184162cf048329 in current term 2: Already voted for candidate 9972f1b44e7742beb5b7386e736ead4f in this term.
W20260709 12:30:48.960417 18479 leader_election.cc:336] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:48.960806 18479 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1396e5332bb14f96bb184162cf048329; no voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a
I20260709 12:30:48.961483 18722 raft_consensus.cc:2749] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:30:48.962083 18553 leader_election.cc:336] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:48.962558 18553 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f; no voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a
I20260709 12:30:48.963256 18711 raft_consensus.cc:2749] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:30:49.541306 18722 raft_consensus.cc:493] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:49.541914 18722 raft_consensus.cc:515] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:49.543911 18722 leader_election.cc:290] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:49.545920 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:49.546877 18589 raft_consensus.cc:2468] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 1.
I20260709 12:30:49.547950 18479 leader_election.cc:304] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:49.548566 18722 raft_consensus.cc:2804] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:30:49.549016 18722 raft_consensus.cc:493] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:30:49.548652 18479 leader_election.cc:336] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:49.549365 18722 raft_consensus.cc:3060] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:49.555189 18722 raft_consensus.cc:515] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:49.557006 18722 leader_election.cc:290] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:49.558826 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cb246f39e3c04375bb0364916c6c82cb" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:49.559540 18589 raft_consensus.cc:3060] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:30:49.560926 18479 leader_election.cc:336] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:49.565130 18589 raft_consensus.cc:2468] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:49.566226 18479 leader_election.cc:304] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:49.567040 18722 raft_consensus.cc:2804] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:49.568040 18722 raft_consensus.cc:697] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:49.568825 18722 consensus_queue.cc:237] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:49.577104 18337 catalog_manager.cc:5697] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 1 to 2, leader changed from dede9b011d384c4896611658022c757a (127.5.4.65) to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:49.615646 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:50.086115 18589 raft_consensus.cc:1275] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:30:50.087666 18722 consensus_queue.cc:1048] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:30:50.094147 18479 consensus_peers.cc:597] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:50.168099 18479 consensus_peers.cc:597] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:50.200096 18553 consensus_peers.cc:597] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:50.265969 18553 consensus_peers.cc:597] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:50.496594 18479 consensus_peers.cc:597] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:50.518388 18553 consensus_peers.cc:597] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:50.593523 18553 consensus_peers.cc:597] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:50.616729 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:50.657140 18722 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:50.657769 18722 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:50.659677 18722 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:50.661350 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f" is_pre_election: true
I20260709 12:30:50.662001 18589 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:50.663169 18479 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
W20260709 12:30:50.663862 18479 leader_election.cc:336] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:50.663858 18722 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:30:50.664358 18722 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:50.664654 18722 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:50.670288 18722 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:50.671986 18722 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 3 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:50.673512 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:50.674088 18589 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:30:50.675552 18479 leader_election.cc:336] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111)
I20260709 12:30:50.681164 18589 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 3.
I20260709 12:30:50.682231 18479 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: dede9b011d384c4896611658022c757a
I20260709 12:30:50.683064 18722 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:50.683537 18722 raft_consensus.cc:697] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:50.684353 18722 consensus_queue.cc:237] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:50.692662 18336 catalog_manager.cc:5697] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 1 to 3, leader changed from dede9b011d384c4896611658022c757a (127.5.4.65) to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 3 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:50.746203 18479 consensus_peers.cc:597] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:30:51.001271 18479 consensus_peers.cc:597] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:51.288630 18589 raft_consensus.cc:1275] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:30:51.290033 18722 consensus_queue.cc:1048] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:30:51.305408 18479 consensus_peers.cc:597] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:51.617910 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:30:52.519836 18553 consensus_peers.cc:597] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:30:52.619635 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:30:52.623106 18479 consensus_peers.cc:597] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:30:52.843024  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:52.851300 18747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:52.851320 18748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:52.856031 18750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:52.858310  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:52.859854  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:52.860145  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:52.860366  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600252860344 us; error 0 us; skew 500 ppm
I20260709 12:30:52.861069  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:52.864317  5137 webserver.cc:533] Webserver started at http://127.5.4.65:43115/ using document root <none> and password file <none>
I20260709 12:30:52.864931  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:52.865154  5137 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:30:52.870143  5137 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.002s	sys 0.001s
W20260709 12:30:52.871124 18479 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111) [suppressed 96 similar messages]
I20260709 12:30:52.875867 18755 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:52.876995  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:30:52.877391  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "dede9b011d384c4896611658022c757a"
format_stamp: "Formatted at 2026-07-09 12:30:45 on dist-test-slave-lg8f"
I20260709 12:30:52.877903  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:52.898816  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:52.900501  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:52.902268  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:52.907361 18762 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:30:52.959026 18479 consensus_peers.cc:597] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:52.968627  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:30:52.969038  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.063s	user 0.000s	sys 0.004s
I20260709 12:30:52.969321  5137 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:30:52.976243 18762 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Bootstrap starting.
W20260709 12:30:52.980741 18553 consensus_peers.cc:597] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:52.996318 18762 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:52.997443 18762 tablet_bootstrap.cc:492] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:52.998229 18762 ts_tablet_manager.cc:1403] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:30:53.001825 18762 raft_consensus.cc:359] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.002966 18762 raft_consensus.cc:740] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.003825 18762 consensus_queue.cc:260] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.007157 18762 ts_tablet_manager.cc:1434] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:30:53.008102 18762 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.026636 18762 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.027714 18762 tablet_bootstrap.cc:492] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.028450 18762 ts_tablet_manager.cc:1403] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 12:30:53.031457 18762 raft_consensus.cc:359] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.032186 18762 raft_consensus.cc:740] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.033052 18762 consensus_queue.cc:260] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.035338 18762 ts_tablet_manager.cc:1434] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:53.036218 18762 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.037755  5137 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:30:53.038164  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.069s	user 0.058s	sys 0.008s
I20260709 12:30:53.056836 18762 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.058118 18762 tablet_bootstrap.cc:492] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.059003 18762 ts_tablet_manager.cc:1403] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 12:30:53.062223 18762 raft_consensus.cc:359] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.063059 18762 raft_consensus.cc:740] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.064016 18762 consensus_queue.cc:260] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.066027 18762 ts_tablet_manager.cc:1434] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:30:53.066886 18762 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Bootstrap starting.
W20260709 12:30:53.072206 18553 consensus_peers.cc:597] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:30:53.085239 18553 consensus_peers.cc:597] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:30:53.086234 18762 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.087574 18762 tablet_bootstrap.cc:492] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.088474 18762 ts_tablet_manager.cc:1403] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.004s
I20260709 12:30:53.092437 18762 raft_consensus.cc:359] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:53.093436 18762 raft_consensus.cc:740] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.094358 18762 consensus_queue.cc:260] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } }
I20260709 12:30:53.096875 18762 ts_tablet_manager.cc:1434] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:53.097787 18762 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.111471  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:46627
I20260709 12:30:53.111565 18823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:46627 every 8 connection(s)
I20260709 12:30:53.117892  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:53.118182 18762 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.119992 18762 tablet_bootstrap.cc:492] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.120668 18762 ts_tablet_manager.cc:1403] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.004s
I20260709 12:30:53.123267 18762 raft_consensus.cc:359] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.124008 18762 raft_consensus.cc:740] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.124799 18762 consensus_queue.cc:260] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.127205 18762 ts_tablet_manager.cc:1434] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:30:53.128103 18762 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.151589 18762 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.152668 18762 tablet_bootstrap.cc:492] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.152688 18825 heartbeater.cc:344] Connected to a master server at 127.5.4.126:44499
I20260709 12:30:53.153250 18825 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:53.153475 18762 ts_tablet_manager.cc:1403] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.006s
I20260709 12:30:53.154361 18825 heartbeater.cc:507] Master 127.5.4.126:44499 requested a full tablet report, sending...
I20260709 12:30:53.156852 18762 raft_consensus.cc:359] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.157637 18762 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.158413 18762 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.160610 18762 ts_tablet_manager.cc:1434] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:30:53.161437 18762 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.162405 18512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
mode: GRACEFUL
new_leader_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:53154
I20260709 12:30:53.162889 18336 ts_manager.cc:194] Re-registered known tserver with Master: dede9b011d384c4896611658022c757a (127.5.4.65:46627)
I20260709 12:30:53.163542 18512 raft_consensus.cc:606] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Received request to transfer leadership to TS 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:53.166352 18512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
mode: GRACEFUL
new_leader_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:53154
I20260709 12:30:53.166898 18512 raft_consensus.cc:606] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Received request to transfer leadership to TS 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:53.168017  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:53.168550  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:53.169230  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:53.176903 18336 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47084
I20260709 12:30:53.178357 18762 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:30:53.179522 18762 tablet_bootstrap.cc:492] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.180230 18762 ts_tablet_manager.cc:1403] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 12:30:53.181941 18825 heartbeater.cc:499] Master 127.5.4.126:44499 was elected leader, sending a full tablet report...
I20260709 12:30:53.183570 18762 raft_consensus.cc:359] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.184476 18762 raft_consensus.cc:740] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.185268 18762 consensus_queue.cc:260] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.187584 18762 ts_tablet_manager.cc:1434] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:30:53.188424 18762 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.203704 18762 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.204583 18762 tablet_bootstrap.cc:492] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.205292 18762 ts_tablet_manager.cc:1403] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.017s	user 0.013s	sys 0.000s
I20260709 12:30:53.208076 18762 raft_consensus.cc:359] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.208618 18762 raft_consensus.cc:740] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.209147 18762 consensus_queue.cc:260] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.211200 18762 ts_tablet_manager.cc:1434] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:30:53.212064 18762 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.227034 18762 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.227802 18762 tablet_bootstrap.cc:492] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.228369 18762 ts_tablet_manager.cc:1403] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:30:53.230324 18762 raft_consensus.cc:359] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.230859 18762 raft_consensus.cc:740] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.231460 18762 consensus_queue.cc:260] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.233433 18762 ts_tablet_manager.cc:1434] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:30:53.234292 18762 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Bootstrap starting.
I20260709 12:30:53.248668 18762 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:30:53.249419 18762 tablet_bootstrap.cc:492] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Bootstrap complete.
I20260709 12:30:53.250034 18741 raft_consensus.cc:993] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: : Instructing follower 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:53.250180 18762 ts_tablet_manager.cc:1403] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.003s
I20260709 12:30:53.250965 18836 raft_consensus.cc:1081] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Signalling peer 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:53.252871 18589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:35884
I20260709 12:30:53.252930 18762 raft_consensus.cc:359] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.253513 18589 raft_consensus.cc:493] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:53.253603 18762 raft_consensus.cc:740] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dede9b011d384c4896611658022c757a, State: Initialized, Role: FOLLOWER
I20260709 12:30:53.254009 18589 raft_consensus.cc:3060] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.254616 18762 consensus_queue.cc:260] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.256902 18762 ts_tablet_manager.cc:1434] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 12:30:53.258616 18741 raft_consensus.cc:993] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: : Instructing follower 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:53.259081 18836 raft_consensus.cc:1081] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Signalling peer 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:53.260762 18590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:35884
I20260709 12:30:53.261175 18590 raft_consensus.cc:493] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:53.261420 18590 raft_consensus.cc:3060] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.264251 18589 raft_consensus.cc:515] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.268795 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:53.269771 18512 raft_consensus.cc:3055] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:53.270232 18512 raft_consensus.cc:740] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:53.270008 18590 raft_consensus.cc:515] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.274267 18590 leader_election.cc:290] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:53.276028 18511 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:53.277052 18511 raft_consensus.cc:3055] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:53.277436 18511 raft_consensus.cc:740] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:53.279745 18589 leader_election.cc:290] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:53.279670 18512 consensus_queue.cc:260] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.279031 18511 consensus_queue.cc:260] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.281426 18512 raft_consensus.cc:3060] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.282791 18511 raft_consensus.cc:3060] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.287704 18799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "feb8927db5a34050961c0c096ca6a03c" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:53.288544 18799 raft_consensus.cc:3060] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.292487 18511 raft_consensus.cc:2468] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 2.
I20260709 12:30:53.294152 18554 leader_election.cc:304] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:53.295429 18746 raft_consensus.cc:2804] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:53.295679 18798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0bf4422aa7554e08855d77908f99763c" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:53.296000 18746 raft_consensus.cc:697] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 2 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:53.296417 18798 raft_consensus.cc:3060] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.296895 18512 raft_consensus.cc:2468] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 2.
I20260709 12:30:53.297307 18746 consensus_queue.cc:237] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.299100 18796 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:30:53.302598 18554 leader_election.cc:304] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:53.306986 18636 raft_consensus.cc:2804] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:53.320727 18799 raft_consensus.cc:2468] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 2.
I20260709 12:30:53.321317 18636 raft_consensus.cc:697] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 2 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:53.323414 18336 catalog_manager.cc:5697] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 1 to 2, leader changed from 1396e5332bb14f96bb184162cf048329 (127.5.4.66) to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 2 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:53.329530 18798 raft_consensus.cc:2468] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 2.
I20260709 12:30:53.338776 18636 consensus_queue.cc:237] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:53.359963 18337 catalog_manager.cc:5697] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 1 to 2, leader changed from 1396e5332bb14f96bb184162cf048329 (127.5.4.66) to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 2 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:53.452376 18796 raft_consensus.cc:3060] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:53.459038 18796 pending_rounds.cc:85] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Aborting all ops after (but not including) 0
I20260709 12:30:53.459263 18796 pending_rounds.cc:107] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:30:53.459568 18796 raft_consensus.cc:2889] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:30:53.621033 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:53.779559 18512 raft_consensus.cc:1275] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:53.781072 18636 consensus_queue.cc:1048] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:53.792754 18796 raft_consensus.cc:1275] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:53.794550 18636 consensus_queue.cc:1048] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:53.805080 18512 raft_consensus.cc:1275] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:53.806514 18746 consensus_queue.cc:1048] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:53.816365 18796 raft_consensus.cc:1275] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:53.819988 18746 consensus_queue.cc:1048] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:54.176975  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:54.197940 18589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
mode: GRACEFUL
new_leader_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:35886
I20260709 12:30:54.198520 18589 raft_consensus.cc:606] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Received request to transfer leadership to TS 1396e5332bb14f96bb184162cf048329
I20260709 12:30:54.201171 18589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
mode: GRACEFUL
new_leader_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:35886
I20260709 12:30:54.201603 18589 raft_consensus.cc:606] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Received request to transfer leadership to TS 1396e5332bb14f96bb184162cf048329
I20260709 12:30:54.202884  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:54.203567  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:54.204453  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:54.384327 18841 raft_consensus.cc:993] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: : Instructing follower 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:54.384949 18636 raft_consensus.cc:1081] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Signalling peer 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:54.386817 18512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:30:54.387413 18512 raft_consensus.cc:493] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:54.387768 18512 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:54.394173 18512 raft_consensus.cc:515] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:54.395678 18512 leader_election.cc:290] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:54.396938 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:54.397552 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:54.397632 18589 raft_consensus.cc:3055] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:54.398221 18796 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:54.398248 18589 raft_consensus.cc:740] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:54.399459 18589 consensus_queue.cc:260] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:54.400604 18589 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:54.403488 18796 raft_consensus.cc:2468] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:54.404829 18479 leader_election.cc:304] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:54.405649 18854 raft_consensus.cc:2804] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:54.406191 18854 raft_consensus.cc:697] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:54.407279 18854 consensus_queue.cc:237] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:54.407908 18589 raft_consensus.cc:2468] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:54.416147 18337 catalog_manager.cc:5697] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 1 to 2, leader changed from 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67) to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:54.440832 18841 raft_consensus.cc:993] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: : Instructing follower 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:54.441632 18636 raft_consensus.cc:1081] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Signalling peer 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:54.444605 18512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:30:54.445448 18512 raft_consensus.cc:493] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:54.445911 18512 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:54.453105 18512 raft_consensus.cc:515] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:54.454694 18512 leader_election.cc:290] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:54.456089 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:54.455978 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:54.456858 18589 raft_consensus.cc:3055] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:54.456956 18796 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:54.457180 18589 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:54.458066 18589 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:54.459088 18589 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:54.463665 18796 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:54.464375 18589 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:54.466126 18479 leader_election.cc:304] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, 9972f1b44e7742beb5b7386e736ead4f; no voters: 
I20260709 12:30:54.467075 18854 raft_consensus.cc:2804] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:54.467931 18854 raft_consensus.cc:697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:54.468904 18854 consensus_queue.cc:237] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:54.476516 18337 catalog_manager.cc:5697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 1 to 2, leader changed from 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67) to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:54.622418 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:54.874923 18796 raft_consensus.cc:1275] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:54.876883 18854 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:54.889143 18589 raft_consensus.cc:1275] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:54.890756 18854 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:30:55.005417 18796 raft_consensus.cc:1275] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:55.007050 18854 consensus_queue.cc:1048] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:55.019081 18589 raft_consensus.cc:1275] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:55.020387 18853 consensus_queue.cc:1048] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:55.212970  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:55.220885 18512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
mode: GRACEFUL
new_leader_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:53154
I20260709 12:30:55.221474 18512 raft_consensus.cc:606] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Received request to transfer leadership to TS 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:55.224030 18512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
mode: GRACEFUL
new_leader_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:53154
I20260709 12:30:55.224474 18512 raft_consensus.cc:606] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Received request to transfer leadership to TS 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:55.225540  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:55.226131  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:55.226825  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:55.447604 18836 raft_consensus.cc:993] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: : Instructing follower 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:55.448240 18853 raft_consensus.cc:1081] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Signalling peer 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:55.449944 18589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:35884
I20260709 12:30:55.450435 18589 raft_consensus.cc:493] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:55.450754 18589 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:55.455474 18589 raft_consensus.cc:515] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:55.457005 18589 leader_election.cc:290] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 3 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:55.458032 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:55.458212 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:55.458953 18796 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:55.459015 18512 raft_consensus.cc:3055] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:55.459468 18512 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:55.460053 18512 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:55.460973 18512 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:55.465015 18796 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 3.
I20260709 12:30:55.466092 18512 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 3.
I20260709 12:30:55.466298 18553 leader_election.cc:304] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:55.467744 18841 raft_consensus.cc:2804] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:55.468665 18841 raft_consensus.cc:697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:55.469517 18841 consensus_queue.cc:237] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:55.476094 18337 catalog_manager.cc:5697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 2 to 3, leader changed from 1396e5332bb14f96bb184162cf048329 (127.5.4.66) to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 3 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:55.623756 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:55.800707 18836 raft_consensus.cc:993] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: : Instructing follower 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:55.801591 18853 raft_consensus.cc:1081] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Signalling peer 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:55.803359 18589 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:35884
I20260709 12:30:55.803833 18589 raft_consensus.cc:493] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:55.804106 18589 raft_consensus.cc:3060] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:55.809103 18589 raft_consensus.cc:515] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:55.811017 18589 leader_election.cc:290] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:55.811930 18798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:55.812700 18798 raft_consensus.cc:3060] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:55.813103 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "08ff9c1df5704377bf35c7804397f431" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:55.813695 18512 raft_consensus.cc:3055] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:55.814034 18512 raft_consensus.cc:740] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:55.814857 18512 consensus_queue.cc:260] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:55.816213 18512 raft_consensus.cc:3060] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:55.820935 18798 raft_consensus.cc:2468] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 2.
I20260709 12:30:55.822268 18553 leader_election.cc:304] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:55.823153 18841 raft_consensus.cc:2804] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:55.823670 18841 raft_consensus.cc:697] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 2 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:55.823705 18512 raft_consensus.cc:2468] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 2.
I20260709 12:30:55.824721 18841 consensus_queue.cc:237] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:55.832254 18337 catalog_manager.cc:5697] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 1 to 2, leader changed from 1396e5332bb14f96bb184162cf048329 (127.5.4.66) to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 2 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:55.946135 18798 raft_consensus.cc:1275] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:55.947913 18841 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:55.961488 18512 raft_consensus.cc:1275] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:55.963018 18841 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:56.233443  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:56.240839 18589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
mode: GRACEFUL
new_leader_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:35886
I20260709 12:30:56.241374 18589 raft_consensus.cc:606] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Received request to transfer leadership to TS 1396e5332bb14f96bb184162cf048329
I20260709 12:30:56.245169 18589 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
mode: GRACEFUL
new_leader_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:35886
I20260709 12:30:56.245951 18589 raft_consensus.cc:606] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Received request to transfer leadership to TS 1396e5332bb14f96bb184162cf048329
I20260709 12:30:56.247386  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:56.248034  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:56.248742  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:56.254853 18512 raft_consensus.cc:1275] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:56.256982 18841 consensus_queue.cc:1048] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:56.267552 18798 raft_consensus.cc:1275] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:56.269594 18877 consensus_queue.cc:1048] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:56.437947 18885 raft_consensus.cc:993] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: : Instructing follower 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:56.438558 18877 raft_consensus.cc:1081] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Signalling peer 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:56.440272 18512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:30:56.440877 18512 raft_consensus.cc:493] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:56.441171 18512 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:56.446465 18512 raft_consensus.cc:515] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:56.448028 18512 leader_election.cc:290] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 4 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:56.449098 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:56.449371 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a8ae5b9817d423fb60455f04d1bcc8d" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:56.449863 18589 raft_consensus.cc:3055] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:30:56.449863 18796 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:56.450331 18589 raft_consensus.cc:740] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:56.451185 18589 consensus_queue.cc:260] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:56.452353 18589 raft_consensus.cc:3060] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:56.457455 18796 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 4.
I20260709 12:30:56.458793 18479 leader_election.cc:304] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:56.458904 18589 raft_consensus.cc:2468] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 4.
I20260709 12:30:56.459638 18853 raft_consensus.cc:2804] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:56.460804 18853 raft_consensus.cc:697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 4 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:56.461573 18853 consensus_queue.cc:237] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:56.469449 18336 catalog_manager.cc:5697] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 3 to 4, leader changed from 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67) to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 4 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:56.565212 18885 raft_consensus.cc:993] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: : Instructing follower 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:56.565728 18877 raft_consensus.cc:1081] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Signalling peer 1396e5332bb14f96bb184162cf048329 to start an election
I20260709 12:30:56.567430 18512 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
 from {username='slave'} at 127.0.0.1:53138
I20260709 12:30:56.568004 18512 raft_consensus.cc:493] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:56.568337 18512 raft_consensus.cc:3060] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:56.574105 18512 raft_consensus.cc:515] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:56.575672 18512 leader_election.cc:290] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67:40539)
I20260709 12:30:56.576699 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:56.576855 18589 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f58e58d872a43ba84789c32783e4a74" candidate_uuid: "1396e5332bb14f96bb184162cf048329" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
I20260709 12:30:56.577375 18796 raft_consensus.cc:3060] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:56.577514 18589 raft_consensus.cc:3055] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:56.577977 18589 raft_consensus.cc:740] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:56.578629 18589 consensus_queue.cc:260] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:56.579571 18589 raft_consensus.cc:3060] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:56.583331 18796 raft_consensus.cc:2468] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:56.584420 18589 raft_consensus.cc:2468] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1396e5332bb14f96bb184162cf048329 in term 2.
I20260709 12:30:56.584565 18479 leader_election.cc:304] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1396e5332bb14f96bb184162cf048329, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:56.585467 18853 raft_consensus.cc:2804] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:56.586112 18853 raft_consensus.cc:697] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Becoming Leader. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:56.587379 18853 consensus_queue.cc:237] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:56.595676 18336 catalog_manager.cc:5697] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 reported cstate change: term changed from 1 to 2, leader changed from 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67) to 1396e5332bb14f96bb184162cf048329 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "1396e5332bb14f96bb184162cf048329" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: UNKNOWN } } }
W20260709 12:30:56.624748 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:56.911252 18590 raft_consensus.cc:1275] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:30:56.912775 18892 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:30:56.921623 18796 raft_consensus.cc:1275] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:30:56.923718 18853 consensus_queue.cc:1048] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:30:57.075512 18796 raft_consensus.cc:1275] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:57.076936 18892 consensus_queue.cc:1048] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:57.088191 18590 raft_consensus.cc:1275] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Refusing update from remote peer 1396e5332bb14f96bb184162cf048329: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:57.090296 18853 consensus_queue.cc:1048] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:57.256567  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:57.265702 18512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
mode: GRACEFUL
new_leader_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:53154
I20260709 12:30:57.266165 18512 raft_consensus.cc:606] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 LEADER]: Received request to transfer leadership to TS 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:57.268647 18512 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a"
dest_uuid: "1396e5332bb14f96bb184162cf048329"
mode: GRACEFUL
new_leader_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:53154
I20260709 12:30:57.269091 18512 raft_consensus.cc:606] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Received request to transfer leadership to TS 9972f1b44e7742beb5b7386e736ead4f
I20260709 12:30:57.270150  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:57.270645  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:57.271286  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:57.532217 18903 raft_consensus.cc:993] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: : Instructing follower 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:57.532837 18853 raft_consensus.cc:1081] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Signalling peer 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:57.534301 18590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:35884
I20260709 12:30:57.534894 18590 raft_consensus.cc:493] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:57.535198 18590 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:57.541786 18590 raft_consensus.cc:515] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:57.543696 18590 leader_election.cc:290] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 3 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:57.544556 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:57.544838 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbeedc48c0f04eb4b10f9df20f06695a" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:57.545243 18796 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:57.545503 18512 raft_consensus.cc:3055] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:57.545897 18512 raft_consensus.cc:740] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:57.546695 18512 consensus_queue.cc:260] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:57.547712 18512 raft_consensus.cc:3060] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:57.550276 18796 raft_consensus.cc:2468] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 3.
I20260709 12:30:57.551349 18553 leader_election.cc:304] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:57.552017 18877 raft_consensus.cc:2804] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:57.552987 18877 raft_consensus.cc:697] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:57.553869 18512 raft_consensus.cc:2468] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 3.
I20260709 12:30:57.553777 18877 consensus_queue.cc:237] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:57.560757 18336 catalog_manager.cc:5697] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 2 to 3, leader changed from 1396e5332bb14f96bb184162cf048329 (127.5.4.66) to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 3 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
W20260709 12:30:57.625823 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:57.837321 18903 raft_consensus.cc:993] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: : Instructing follower 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:57.837929 18903 raft_consensus.cc:1081] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 LEADER]: Signalling peer 9972f1b44e7742beb5b7386e736ead4f to start an election
I20260709 12:30:57.840130 18590 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4"
dest_uuid: "9972f1b44e7742beb5b7386e736ead4f"
 from {username='slave'} at 127.0.0.1:35884
I20260709 12:30:57.840875 18590 raft_consensus.cc:493] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:57.841341 18590 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:57.849490 18590 raft_consensus.cc:515] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:57.851336 18590 leader_election.cc:290] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 4 election: Requested vote from peers dede9b011d384c4896611658022c757a (127.5.4.65:46627), 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163)
I20260709 12:30:57.852142 18796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "dede9b011d384c4896611658022c757a"
I20260709 12:30:57.852597 18512 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c9000dd337d14f7ba6de5fc47003bac4" candidate_uuid: "9972f1b44e7742beb5b7386e736ead4f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1396e5332bb14f96bb184162cf048329"
I20260709 12:30:57.852818 18796 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:57.853299 18512 raft_consensus.cc:3055] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:30:57.853575 18512 raft_consensus.cc:740] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1396e5332bb14f96bb184162cf048329, State: Running, Role: LEADER
I20260709 12:30:57.854293 18512 consensus_queue.cc:260] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:57.855401 18512 raft_consensus.cc:3060] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:30:57.858443 18796 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 4.
I20260709 12:30:57.859776 18553 leader_election.cc:304] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9972f1b44e7742beb5b7386e736ead4f, dede9b011d384c4896611658022c757a; no voters: 
I20260709 12:30:57.860669 18877 raft_consensus.cc:2804] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:30:57.861126 18512 raft_consensus.cc:2468] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9972f1b44e7742beb5b7386e736ead4f in term 4.
I20260709 12:30:57.861191 18877 raft_consensus.cc:697] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 4 LEADER]: Becoming Leader. State: Replica: 9972f1b44e7742beb5b7386e736ead4f, State: Running, Role: LEADER
I20260709 12:30:57.862605 18877 consensus_queue.cc:237] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } }
I20260709 12:30:57.871157 18336 catalog_manager.cc:5697] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f reported cstate change: term changed from 3 to 4, leader changed from 1396e5332bb14f96bb184162cf048329 (127.5.4.66) to 9972f1b44e7742beb5b7386e736ead4f (127.5.4.67). New cstate: current_term: 4 leader_uuid: "9972f1b44e7742beb5b7386e736ead4f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9972f1b44e7742beb5b7386e736ead4f" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 40539 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:58.089062 18796 raft_consensus.cc:1275] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:58.090332 18877 consensus_queue.cc:1048] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dede9b011d384c4896611658022c757a" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 46627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:58.101070 18512 raft_consensus.cc:1275] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:58.103245 18877 consensus_queue.cc:1048] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:58.278079  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:58.307960  5137 tablet_server.cc:179] TabletServer@127.5.4.65:46627 shutting down...
I20260709 12:30:58.329108  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:58.329939  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.330734  5137 raft_consensus.cc:2243] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.331287  5137 raft_consensus.cc:2272] T 08ff9c1df5704377bf35c7804397f431 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.333654  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.334268  5137 raft_consensus.cc:2243] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.334743  5137 raft_consensus.cc:2272] T cb246f39e3c04375bb0364916c6c82cb P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.336690  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.337255  5137 raft_consensus.cc:2243] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.337702  5137 raft_consensus.cc:2272] T dbeedc48c0f04eb4b10f9df20f06695a P dede9b011d384c4896611658022c757a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.340432  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.341035  5137 raft_consensus.cc:2243] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.341465  5137 raft_consensus.cc:2272] T 3a8ae5b9817d423fb60455f04d1bcc8d P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.343479  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.344041  5137 raft_consensus.cc:2243] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.344475  5137 raft_consensus.cc:2272] T 0bf4422aa7554e08855d77908f99763c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.346400  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.346992  5137 raft_consensus.cc:2243] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.347414  5137 raft_consensus.cc:2272] T 9f58e58d872a43ba84789c32783e4a74 P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.349300  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.349877  5137 raft_consensus.cc:2243] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.350299  5137 raft_consensus.cc:2272] T f87a4a1efe584b8e9c9a9b74fb367c40 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.352226  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.352789  5137 raft_consensus.cc:2243] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.353202  5137 raft_consensus.cc:2272] T feb8927db5a34050961c0c096ca6a03c P dede9b011d384c4896611658022c757a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.355155  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.355676  5137 raft_consensus.cc:2243] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.356325  5137 raft_consensus.cc:2272] T b4f15bd8832b4b39abfaec4c976b8ec1 P dede9b011d384c4896611658022c757a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.358428  5137 tablet_replica.cc:333] stopping tablet replica
I20260709 12:30:58.358994  5137 raft_consensus.cc:2243] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.359416  5137 raft_consensus.cc:2272] T c9000dd337d14f7ba6de5fc47003bac4 P dede9b011d384c4896611658022c757a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.363960 18512 raft_consensus.cc:1275] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Refusing update from remote peer 9972f1b44e7742beb5b7386e736ead4f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:30:58.365413 18905 consensus_queue.cc:1048] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1396e5332bb14f96bb184162cf048329" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:30:58.377998 18553 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 12:30:58.384902 18553 consensus_peers.cc:597] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:58.399024  5137 tablet_server.cc:196] TabletServer@127.5.4.65:46627 shutdown complete.
W20260709 12:30:58.420200 18479 consensus_peers.cc:597] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:58.424829  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
W20260709 12:30:58.453037 18479 consensus_peers.cc:597] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:58.459394  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:58.460275  5137 tablet_replica.cc:333] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.461092  5137 raft_consensus.cc:2243] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.461977  5137 raft_consensus.cc:2272] T cb246f39e3c04375bb0364916c6c82cb P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.464818  5137 tablet_replica.cc:333] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.465431  5137 raft_consensus.cc:2243] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.465821  5137 raft_consensus.cc:2272] T 08ff9c1df5704377bf35c7804397f431 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.468212  5137 tablet_replica.cc:333] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.468724  5137 raft_consensus.cc:2243] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.469544  5137 raft_consensus.cc:2272] T f87a4a1efe584b8e9c9a9b74fb367c40 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.471669  5137 tablet_replica.cc:333] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.472155  5137 raft_consensus.cc:2243] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.472538  5137 raft_consensus.cc:2272] T feb8927db5a34050961c0c096ca6a03c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.474759  5137 tablet_replica.cc:333] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.475304  5137 raft_consensus.cc:2243] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.475693  5137 raft_consensus.cc:2272] T c9000dd337d14f7ba6de5fc47003bac4 P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.479249  5137 tablet_replica.cc:333] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.480098  5137 raft_consensus.cc:2243] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.480688  5137 raft_consensus.cc:2272] T b4f15bd8832b4b39abfaec4c976b8ec1 P 1396e5332bb14f96bb184162cf048329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.483253  5137 tablet_replica.cc:333] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.483887  5137 raft_consensus.cc:2243] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.484267  5137 raft_consensus.cc:2272] T dbeedc48c0f04eb4b10f9df20f06695a P 1396e5332bb14f96bb184162cf048329 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.487140  5137 tablet_replica.cc:333] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.487718  5137 raft_consensus.cc:2243] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.488560  5137 raft_consensus.cc:2272] T 3a8ae5b9817d423fb60455f04d1bcc8d P 1396e5332bb14f96bb184162cf048329 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.491004  5137 tablet_replica.cc:333] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
I20260709 12:30:58.491642  5137 raft_consensus.cc:2243] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.492089  5137 raft_consensus.cc:2272] T 0bf4422aa7554e08855d77908f99763c P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.495054  5137 tablet_replica.cc:333] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329: stopping tablet replica
W20260709 12:30:58.495575 18553 consensus_peers.cc:597] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:58.495580  5137 raft_consensus.cc:2243] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.496935  5137 raft_consensus.cc:2272] T 9f58e58d872a43ba84789c32783e4a74 P 1396e5332bb14f96bb184162cf048329 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:58.519778 18554 consensus_peers.cc:597] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f -> Peer 1396e5332bb14f96bb184162cf048329 (127.5.4.66:41163): Couldn't send request to peer 1396e5332bb14f96bb184162cf048329. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:58.528692  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
I20260709 12:30:58.558591  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
W20260709 12:30:58.568264 18553 consensus_peers.cc:597] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f -> Peer dede9b011d384c4896611658022c757a (127.5.4.65:46627): Couldn't send request to peer dede9b011d384c4896611658022c757a. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:46627: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:58.592813  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:58.593750  5137 tablet_replica.cc:333] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.594543  5137 raft_consensus.cc:2243] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.595319  5137 raft_consensus.cc:2272] T 08ff9c1df5704377bf35c7804397f431 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.597512  5137 tablet_replica.cc:333] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.598100  5137 raft_consensus.cc:2243] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.598965  5137 raft_consensus.cc:2272] T feb8927db5a34050961c0c096ca6a03c P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.600975  5137 tablet_replica.cc:333] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.601531  5137 raft_consensus.cc:2243] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.601974  5137 raft_consensus.cc:2272] T f87a4a1efe584b8e9c9a9b74fb367c40 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.603983  5137 tablet_replica.cc:333] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.604549  5137 raft_consensus.cc:2243] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.604950  5137 raft_consensus.cc:2272] T 9f58e58d872a43ba84789c32783e4a74 P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.606905  5137 tablet_replica.cc:333] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.607503  5137 raft_consensus.cc:2243] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.608696  5137 raft_consensus.cc:2272] T 0bf4422aa7554e08855d77908f99763c P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.611618  5137 tablet_replica.cc:333] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.612215  5137 raft_consensus.cc:2243] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.613080  5137 raft_consensus.cc:2272] T dbeedc48c0f04eb4b10f9df20f06695a P 9972f1b44e7742beb5b7386e736ead4f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.616122  5137 tablet_replica.cc:333] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.616737  5137 raft_consensus.cc:2243] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.617097  5137 raft_consensus.cc:2272] T 3a8ae5b9817d423fb60455f04d1bcc8d P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.618983  5137 tablet_replica.cc:333] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.619544  5137 raft_consensus.cc:2243] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:58.619921  5137 raft_consensus.cc:2272] T cb246f39e3c04375bb0364916c6c82cb P 9972f1b44e7742beb5b7386e736ead4f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.621922  5137 tablet_replica.cc:333] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.622345  5137 raft_consensus.cc:2243] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.623126  5137 raft_consensus.cc:2272] T b4f15bd8832b4b39abfaec4c976b8ec1 P 9972f1b44e7742beb5b7386e736ead4f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.625129  5137 tablet_replica.cc:333] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f: stopping tablet replica
I20260709 12:30:58.625770  5137 raft_consensus.cc:2243] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.626751  5137 raft_consensus.cc:2272] T c9000dd337d14f7ba6de5fc47003bac4 P 9972f1b44e7742beb5b7386e736ead4f [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:58.626822 18387 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:30:58.652624  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:30:58.682372  5137 master.cc:562] Master@127.5.4.126:44499 shutting down...
I20260709 12:30:58.720464  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:58.721092  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:58.721450  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P 071688f3f2324b91baab385d069c2681: stopping tablet replica
I20260709 12:30:58.743445  5137 master.cc:584] Master@127.5.4.126:44499 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13367 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:30:58.790333  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:42927
I20260709 12:30:58.791599  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:58.797232 18925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:58.797391 18926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:58.799391 18928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:58.799849  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:58.800931  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:58.801117  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:58.801239  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600258801226 us; error 0 us; skew 500 ppm
I20260709 12:30:58.801741  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:58.804479  5137 webserver.cc:533] Webserver started at http://127.5.4.126:41951/ using document root <none> and password file <none>
I20260709 12:30:58.804993  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:58.805161  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:58.805400  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:58.806654  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "9f5445a57c764969a7be646f6f3f4704"
format_stamp: "Formatted at 2026-07-09 12:30:58 on dist-test-slave-lg8f"
I20260709 12:30:58.811214  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:30:58.814746 18933 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:58.815641  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:30:58.816046  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "9f5445a57c764969a7be646f6f3f4704"
format_stamp: "Formatted at 2026-07-09 12:30:58 on dist-test-slave-lg8f"
I20260709 12:30:58.816385  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:58.834465  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:58.835821  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:58.880431  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:42927
I20260709 12:30:58.880539 18984 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:42927 every 8 connection(s)
I20260709 12:30:58.884610 18985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:58.895367 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704: Bootstrap starting.
I20260709 12:30:58.900175 18985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:58.904719 18985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704: No bootstrap required, opened a new log
I20260709 12:30:58.906937 18985 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5445a57c764969a7be646f6f3f4704" member_type: VOTER }
I20260709 12:30:58.907334 18985 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:58.907611 18985 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5445a57c764969a7be646f6f3f4704, State: Initialized, Role: FOLLOWER
I20260709 12:30:58.908277 18985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5445a57c764969a7be646f6f3f4704" member_type: VOTER }
I20260709 12:30:58.908768 18985 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:58.908998 18985 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:58.909282 18985 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:58.913934 18985 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5445a57c764969a7be646f6f3f4704" member_type: VOTER }
I20260709 12:30:58.914448 18985 leader_election.cc:304] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f5445a57c764969a7be646f6f3f4704; no voters: 
I20260709 12:30:58.915755 18985 leader_election.cc:290] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:58.916399 18988 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:58.918015 18988 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 1 LEADER]: Becoming Leader. State: Replica: 9f5445a57c764969a7be646f6f3f4704, State: Running, Role: LEADER
I20260709 12:30:58.918802 18988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5445a57c764969a7be646f6f3f4704" member_type: VOTER }
I20260709 12:30:58.919095 18985 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:58.924926 18989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f5445a57c764969a7be646f6f3f4704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5445a57c764969a7be646f6f3f4704" member_type: VOTER } }
I20260709 12:30:58.925654 18989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:58.929363 18990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f5445a57c764969a7be646f6f3f4704. Latest consensus state: current_term: 1 leader_uuid: "9f5445a57c764969a7be646f6f3f4704" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f5445a57c764969a7be646f6f3f4704" member_type: VOTER } }
I20260709 12:30:58.929960 18990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:58.931785 18998 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:58.937551 18998 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:58.938824  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:58.947479 18998 catalog_manager.cc:1383] Generated new cluster ID: 52c7b4e2c783475cb93a514a632a0df3
I20260709 12:30:58.947926 18998 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:58.967489 18998 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:58.968851 18998 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:58.981719 18998 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704: Generated new TSK 0
I20260709 12:30:58.982345 18998 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:59.006789  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:59.013875 19006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:59.015111 19007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:59.016922 19009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:59.017457  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:59.018465  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:59.018735  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:59.018864  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600259018853 us; error 0 us; skew 500 ppm
I20260709 12:30:59.019368  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:59.021919  5137 webserver.cc:533] Webserver started at http://127.5.4.65:45667/ using document root <none> and password file <none>
I20260709 12:30:59.022362  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:59.022614  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:59.022841  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:59.023887  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "68f4013bd92b48bf8ff45b72dc0af087"
format_stamp: "Formatted at 2026-07-09 12:30:59 on dist-test-slave-lg8f"
I20260709 12:30:59.028715  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:30:59.032205 19014 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.033083  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:30:59.033355  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "68f4013bd92b48bf8ff45b72dc0af087"
format_stamp: "Formatted at 2026-07-09 12:30:59 on dist-test-slave-lg8f"
I20260709 12:30:59.033620  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:59.048503  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:59.049722  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:59.051368  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:59.053742  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:59.053920  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.054219  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:59.054411  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.097622  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:34177
I20260709 12:30:59.097802 19076 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:34177 every 8 connection(s)
I20260709 12:30:59.102723  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:59.110225 19081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:59.112239 19082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:59.115797 19084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:59.116087  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:59.116732 19077 heartbeater.cc:344] Connected to a master server at 127.5.4.126:42927
I20260709 12:30:59.117175 19077 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:59.117504  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:59.117802  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:59.118000  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600259117982 us; error 0 us; skew 500 ppm
I20260709 12:30:59.118162 19077 heartbeater.cc:507] Master 127.5.4.126:42927 requested a full tablet report, sending...
I20260709 12:30:59.118710  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:59.121069 18950 ts_manager.cc:194] Registered new tserver with Master: 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177)
I20260709 12:30:59.121845  5137 webserver.cc:533] Webserver started at http://127.5.4.66:34629/ using document root <none> and password file <none>
I20260709 12:30:59.122557  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:59.122808  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:59.123113  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:59.123643 18950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53642
I20260709 12:30:59.124503  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "38958d7b020c472ca77e3192fab9d37e"
format_stamp: "Formatted at 2026-07-09 12:30:59 on dist-test-slave-lg8f"
I20260709 12:30:59.129413  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 12:30:59.132925 19089 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.133898  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:30:59.134263  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "38958d7b020c472ca77e3192fab9d37e"
format_stamp: "Formatted at 2026-07-09 12:30:59 on dist-test-slave-lg8f"
I20260709 12:30:59.134605  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:59.151343  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:59.152532  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:59.154122  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:59.156392  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:59.156639  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.156886  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:59.157059  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.200668  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:38689
I20260709 12:30:59.200814 19151 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:38689 every 8 connection(s)
I20260709 12:30:59.205765  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:59.213795 19155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:59.215792 19156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:59.218019 19158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:59.218077  5137 server_base.cc:1061] running on GCE node
I20260709 12:30:59.219671  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:59.220371  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:59.220695  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600259220586 us; error 0 us; skew 500 ppm
I20260709 12:30:59.221374 19152 heartbeater.cc:344] Connected to a master server at 127.5.4.126:42927
I20260709 12:30:59.221467  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:59.221899 19152 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:59.222810 19152 heartbeater.cc:507] Master 127.5.4.126:42927 requested a full tablet report, sending...
I20260709 12:30:59.224633  5137 webserver.cc:533] Webserver started at http://127.5.4.67:46411/ using document root <none> and password file <none>
I20260709 12:30:59.225445  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:59.225327 18950 ts_manager.cc:194] Registered new tserver with Master: 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689)
I20260709 12:30:59.225903  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:59.226269  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:59.227632 18950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53654
I20260709 12:30:59.228125  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "324846bcca064a8a9042f16b4c675165"
format_stamp: "Formatted at 2026-07-09 12:30:59 on dist-test-slave-lg8f"
I20260709 12:30:59.233659  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:30:59.237635 19163 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.238477  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:30:59.238857  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "324846bcca064a8a9042f16b4c675165"
format_stamp: "Formatted at 2026-07-09 12:30:59 on dist-test-slave-lg8f"
I20260709 12:30:59.239125  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:59.269074  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:59.270525  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:59.272234  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:59.274701  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:59.274943  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.275200  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:59.275389  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:59.329609  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:34875
I20260709 12:30:59.329643 19225 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:34875 every 8 connection(s)
I20260709 12:30:59.348963 19226 heartbeater.cc:344] Connected to a master server at 127.5.4.126:42927
I20260709 12:30:59.349437 19226 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:59.350535 19226 heartbeater.cc:507] Master 127.5.4.126:42927 requested a full tablet report, sending...
I20260709 12:30:59.353178 18950 ts_manager.cc:194] Registered new tserver with Master: 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:30:59.353555  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018352125s
I20260709 12:30:59.355165 18950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53658
I20260709 12:30:59.388685 18950 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53674:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:59.390925 18950 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:59.456163 19041 tablet_service.cc:1511] Processing CreateTablet for tablet c40b8b071ab8413287c3725bba20f64f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=8b49ecfa3eb44b4ab1e03035dde696f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:30:59.458091 19041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40b8b071ab8413287c3725bba20f64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.459185 19116 tablet_service.cc:1511] Processing CreateTablet for tablet c40b8b071ab8413287c3725bba20f64f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=8b49ecfa3eb44b4ab1e03035dde696f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:30:59.460518 19116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40b8b071ab8413287c3725bba20f64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.460879 19042 tablet_service.cc:1511] Processing CreateTablet for tablet 2967db3c2e7c408f8b8c9ea4d720b05c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=8b49ecfa3eb44b4ab1e03035dde696f3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:30:59.459005 19117 tablet_service.cc:1511] Processing CreateTablet for tablet 2967db3c2e7c408f8b8c9ea4d720b05c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=8b49ecfa3eb44b4ab1e03035dde696f3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:30:59.462435 19042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2967db3c2e7c408f8b8c9ea4d720b05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.463371 19117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2967db3c2e7c408f8b8c9ea4d720b05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.473939 19191 tablet_service.cc:1511] Processing CreateTablet for tablet 2967db3c2e7c408f8b8c9ea4d720b05c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=8b49ecfa3eb44b4ab1e03035dde696f3]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:30:59.475433 19191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2967db3c2e7c408f8b8c9ea4d720b05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.477998 19190 tablet_service.cc:1511] Processing CreateTablet for tablet c40b8b071ab8413287c3725bba20f64f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=8b49ecfa3eb44b4ab1e03035dde696f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:30:59.479445 19190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40b8b071ab8413287c3725bba20f64f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.487490 19241 tablet_bootstrap.cc:492] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087: Bootstrap starting.
I20260709 12:30:59.491736 19242 tablet_bootstrap.cc:492] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: Bootstrap starting.
I20260709 12:30:59.498375 19242 tablet_bootstrap.cc:654] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.500530 19241 tablet_bootstrap.cc:654] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.510021 19244 tablet_bootstrap.cc:492] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: Bootstrap starting.
I20260709 12:30:59.513392 19242 tablet_bootstrap.cc:492] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: No bootstrap required, opened a new log
I20260709 12:30:59.514338 19242 ts_tablet_manager.cc:1403] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.005s
I20260709 12:30:59.517199 19241 tablet_bootstrap.cc:492] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087: No bootstrap required, opened a new log
I20260709 12:30:59.517796 19241 ts_tablet_manager.cc:1403] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087: Time spent bootstrapping tablet: real 0.031s	user 0.018s	sys 0.000s
I20260709 12:30:59.518126 19242 raft_consensus.cc:359] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.519008 19242 raft_consensus.cc:385] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.519243 19244 tablet_bootstrap.cc:654] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.519327 19242 raft_consensus.cc:740] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.520403 19242 consensus_queue.cc:260] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.521166 19241 raft_consensus.cc:359] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.522974 19241 raft_consensus.cc:385] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.523943 19241 raft_consensus.cc:740] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.525156 19241 consensus_queue.cc:260] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.528069 19152 heartbeater.cc:499] Master 127.5.4.126:42927 was elected leader, sending a full tablet report...
I20260709 12:30:59.530164 19244 tablet_bootstrap.cc:492] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: No bootstrap required, opened a new log
I20260709 12:30:59.530715 19244 ts_tablet_manager.cc:1403] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 12:30:59.531973 19242 ts_tablet_manager.cc:1434] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.003s
I20260709 12:30:59.533342 19242 tablet_bootstrap.cc:492] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e: Bootstrap starting.
I20260709 12:30:59.534883 19244 raft_consensus.cc:359] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.535595 19241 ts_tablet_manager.cc:1434] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.008s
I20260709 12:30:59.535714 19244 raft_consensus.cc:385] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.536432 19244 raft_consensus.cc:740] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.537374 19241 tablet_bootstrap.cc:492] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087: Bootstrap starting.
I20260709 12:30:59.537410 19244 consensus_queue.cc:260] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.543879 19242 tablet_bootstrap.cc:654] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.535633 19077 heartbeater.cc:499] Master 127.5.4.126:42927 was elected leader, sending a full tablet report...
I20260709 12:30:59.547621 19241 tablet_bootstrap.cc:654] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.565919 19244 ts_tablet_manager.cc:1434] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: Time spent starting tablet: real 0.034s	user 0.012s	sys 0.018s
I20260709 12:30:59.567577 19226 heartbeater.cc:499] Master 127.5.4.126:42927 was elected leader, sending a full tablet report...
I20260709 12:30:59.568131 19244 tablet_bootstrap.cc:492] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165: Bootstrap starting.
I20260709 12:30:59.568778 19242 tablet_bootstrap.cc:492] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e: No bootstrap required, opened a new log
I20260709 12:30:59.569555 19242 ts_tablet_manager.cc:1403] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e: Time spent bootstrapping tablet: real 0.036s	user 0.024s	sys 0.009s
I20260709 12:30:59.569671 19241 tablet_bootstrap.cc:492] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087: No bootstrap required, opened a new log
I20260709 12:30:59.570400 19241 ts_tablet_manager.cc:1403] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087: Time spent bootstrapping tablet: real 0.033s	user 0.004s	sys 0.026s
I20260709 12:30:59.580305 19241 raft_consensus.cc:359] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.580725 19242 raft_consensus.cc:359] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.581354 19241 raft_consensus.cc:385] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.581784 19242 raft_consensus.cc:385] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.581809 19241 raft_consensus.cc:740] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.582325 19242 raft_consensus.cc:740] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.583069 19241 consensus_queue.cc:260] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.583633 19242 consensus_queue.cc:260] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.584513 19244 tablet_bootstrap.cc:654] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.585695 19241 ts_tablet_manager.cc:1434] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 12:30:59.588413 19242 ts_tablet_manager.cc:1434] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.003s
I20260709 12:30:59.591300 19244 tablet_bootstrap.cc:492] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165: No bootstrap required, opened a new log
I20260709 12:30:59.591938 19244 ts_tablet_manager.cc:1403] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.004s
I20260709 12:30:59.595495 19244 raft_consensus.cc:359] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.596306 19244 raft_consensus.cc:385] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.596563 19244 raft_consensus.cc:740] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.597229 19244 consensus_queue.cc:260] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.599248 19244 ts_tablet_manager.cc:1434] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:59.606194 19246 raft_consensus.cc:493] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.606951 19246 raft_consensus.cc:515] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.609952 19246 leader_election.cc:290] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:30:59.611826 19249 raft_consensus.cc:493] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.612396 19249 raft_consensus.cc:515] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.615262 19249 leader_election.cc:290] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689)
I20260709 12:30:59.621395 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087" is_pre_election: true
I20260709 12:30:59.622228 19052 raft_consensus.cc:2468] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 0.
I20260709 12:30:59.623776 19091 leader_election.cc:304] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38958d7b020c472ca77e3192fab9d37e, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.624619 19246 raft_consensus.cc:2804] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.624980 19246 raft_consensus.cc:493] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.625345 19246 raft_consensus.cc:3060] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.629452 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324846bcca064a8a9042f16b4c675165" is_pre_election: true
I20260709 12:30:59.630250 19201 raft_consensus.cc:2468] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 0.
I20260709 12:30:59.633872 19246 raft_consensus.cc:515] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.634443 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c" candidate_uuid: "324846bcca064a8a9042f16b4c675165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087" is_pre_election: true
I20260709 12:30:59.635210 19052 raft_consensus.cc:2468] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 324846bcca064a8a9042f16b4c675165 in term 0.
I20260709 12:30:59.636740 19165 leader_election.cc:304] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.637006 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
I20260709 12:30:59.637490 19246 leader_election.cc:290] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 election: Requested vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:30:59.637738 19249 raft_consensus.cc:2804] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.637590 19052 raft_consensus.cc:3060] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.638232 19249 raft_consensus.cc:493] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.638935 19249 raft_consensus.cc:3060] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.639353 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c" candidate_uuid: "324846bcca064a8a9042f16b4c675165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38958d7b020c472ca77e3192fab9d37e" is_pre_election: true
I20260709 12:30:59.639631 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:30:59.640055 19127 raft_consensus.cc:2468] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 324846bcca064a8a9042f16b4c675165 in term 0.
I20260709 12:30:59.640349 19201 raft_consensus.cc:3060] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.648917 19052 raft_consensus.cc:2468] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 1.
I20260709 12:30:59.649189 19201 raft_consensus.cc:2468] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 1.
I20260709 12:30:59.649286 19249 raft_consensus.cc:515] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.650334 19091 leader_election.cc:304] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38958d7b020c472ca77e3192fab9d37e, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.651233 19246 raft_consensus.cc:2804] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.651862 19249 leader_election.cc:290] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [CANDIDATE]: Term 1 election: Requested vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689)
I20260709 12:30:59.652979 19246 raft_consensus.cc:697] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Becoming Leader. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:30:59.653187 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c" candidate_uuid: "324846bcca064a8a9042f16b4c675165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
I20260709 12:30:59.653865 19127 raft_consensus.cc:3060] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.654124 19246 consensus_queue.cc:237] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.654659 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c" candidate_uuid: "324846bcca064a8a9042f16b4c675165" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
I20260709 12:30:59.655269 19052 raft_consensus.cc:3060] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.661996 19052 raft_consensus.cc:2468] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 324846bcca064a8a9042f16b4c675165 in term 1.
I20260709 12:30:59.662365 19127 raft_consensus.cc:2468] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 324846bcca064a8a9042f16b4c675165 in term 1.
I20260709 12:30:59.663491 19165 leader_election.cc:304] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.664319 19249 raft_consensus.cc:2804] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.663921 18950 catalog_manager.cc:5697] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38958d7b020c472ca77e3192fab9d37e (127.5.4.66). New cstate: current_term: 1 leader_uuid: "38958d7b020c472ca77e3192fab9d37e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.665957 19249 raft_consensus.cc:697] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 LEADER]: Becoming Leader. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Running, Role: LEADER
I20260709 12:30:59.667171 19249 consensus_queue.cc:237] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:30:59.675433 18948 catalog_manager.cc:5697] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 reported cstate change: term changed from 0 to 1, leader changed from <none> to 324846bcca064a8a9042f16b4c675165 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "324846bcca064a8a9042f16b4c675165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.720070 18948 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53688:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:59.722311 18948 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:59.747164 19041 tablet_service.cc:1511] Processing CreateTablet for tablet 6b00ddb53b9a4c21a3f4aa748e5e4226 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=452fc2ca644f47eeac40e02f19a02857]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:30:59.748433 19041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b00ddb53b9a4c21a3f4aa748e5e4226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.748412 19117 tablet_service.cc:1511] Processing CreateTablet for tablet 6b00ddb53b9a4c21a3f4aa748e5e4226 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=452fc2ca644f47eeac40e02f19a02857]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:30:59.749970 19117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b00ddb53b9a4c21a3f4aa748e5e4226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.749742 19190 tablet_service.cc:1511] Processing CreateTablet for tablet 6b00ddb53b9a4c21a3f4aa748e5e4226 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=452fc2ca644f47eeac40e02f19a02857]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:30:59.751029 19190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b00ddb53b9a4c21a3f4aa748e5e4226. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.752025 19042 tablet_service.cc:1511] Processing CreateTablet for tablet 7d31d523efa047cabc3e39438ded71da (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=452fc2ca644f47eeac40e02f19a02857]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:30:59.753456 19042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d31d523efa047cabc3e39438ded71da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.755637 19191 tablet_service.cc:1511] Processing CreateTablet for tablet 7d31d523efa047cabc3e39438ded71da (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=452fc2ca644f47eeac40e02f19a02857]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:30:59.756978 19191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d31d523efa047cabc3e39438ded71da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.759701 19116 tablet_service.cc:1511] Processing CreateTablet for tablet 7d31d523efa047cabc3e39438ded71da (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=452fc2ca644f47eeac40e02f19a02857]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:30:59.761011 19116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d31d523efa047cabc3e39438ded71da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:59.764935 19241 tablet_bootstrap.cc:492] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087: Bootstrap starting.
I20260709 12:30:59.766966 19244 tablet_bootstrap.cc:492] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165: Bootstrap starting.
I20260709 12:30:59.772352 19241 tablet_bootstrap.cc:654] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.773602 19244 tablet_bootstrap.cc:654] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.780351 19244 tablet_bootstrap.cc:492] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165: No bootstrap required, opened a new log
I20260709 12:30:59.781034 19244 ts_tablet_manager.cc:1403] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165: Time spent bootstrapping tablet: real 0.014s	user 0.006s	sys 0.005s
I20260709 12:30:59.784489 19244 raft_consensus.cc:359] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.785385 19244 raft_consensus.cc:385] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.785723 19244 raft_consensus.cc:740] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.785008 19241 tablet_bootstrap.cc:492] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087: No bootstrap required, opened a new log
I20260709 12:30:59.786871 19241 ts_tablet_manager.cc:1403] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.001s
I20260709 12:30:59.790398 19241 raft_consensus.cc:359] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.791158 19241 raft_consensus.cc:385] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.791448 19241 raft_consensus.cc:740] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.791661 19242 tablet_bootstrap.cc:492] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Bootstrap starting.
I20260709 12:30:59.792362 19244 consensus_queue.cc:260] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.794912 19244 ts_tablet_manager.cc:1434] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.002s
I20260709 12:30:59.794704 19241 consensus_queue.cc:260] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.797490 19241 ts_tablet_manager.cc:1434] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:30:59.798563 19244 tablet_bootstrap.cc:492] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165: Bootstrap starting.
I20260709 12:30:59.798628 19241 tablet_bootstrap.cc:492] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: Bootstrap starting.
I20260709 12:30:59.805079 19242 tablet_bootstrap.cc:654] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.806599 19241 tablet_bootstrap.cc:654] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.806977 19244 tablet_bootstrap.cc:654] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.824311 19244 tablet_bootstrap.cc:492] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165: No bootstrap required, opened a new log
I20260709 12:30:59.824962 19244 ts_tablet_manager.cc:1403] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165: Time spent bootstrapping tablet: real 0.027s	user 0.017s	sys 0.008s
I20260709 12:30:59.825713 19241 tablet_bootstrap.cc:492] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: No bootstrap required, opened a new log
I20260709 12:30:59.826315 19241 ts_tablet_manager.cc:1403] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.011s
I20260709 12:30:59.824324 19242 tablet_bootstrap.cc:492] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: No bootstrap required, opened a new log
I20260709 12:30:59.827107 19242 ts_tablet_manager.cc:1403] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.009s
I20260709 12:30:59.829157 19244 raft_consensus.cc:359] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.829998 19244 raft_consensus.cc:385] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.829677 19241 raft_consensus.cc:359] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.830358 19244 raft_consensus.cc:740] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.830534 19241 raft_consensus.cc:385] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.830423 19242 raft_consensus.cc:359] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.831285 19241 raft_consensus.cc:740] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.831524 19242 raft_consensus.cc:385] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.832034 19242 raft_consensus.cc:740] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.831578 19244 consensus_queue.cc:260] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.832297 19241 consensus_queue.cc:260] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.832973 19242 consensus_queue.cc:260] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.835523 19242 ts_tablet_manager.cc:1434] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:59.836647 19242 tablet_bootstrap.cc:492] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: Bootstrap starting.
I20260709 12:30:59.837188 19241 ts_tablet_manager.cc:1434] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 12:30:59.844957 19242 tablet_bootstrap.cc:654] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:59.846873 19244 ts_tablet_manager.cc:1434] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:30:59.864764 19242 tablet_bootstrap.cc:492] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: No bootstrap required, opened a new log
I20260709 12:30:59.865479 19242 ts_tablet_manager.cc:1403] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.013s
I20260709 12:30:59.869148 19242 raft_consensus.cc:359] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.869995 19242 raft_consensus.cc:385] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:59.870347 19242 raft_consensus.cc:740] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Initialized, Role: FOLLOWER
I20260709 12:30:59.871356 19242 consensus_queue.cc:260] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.873977 19242 ts_tablet_manager.cc:1434] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:30:59.925782 19246 raft_consensus.cc:493] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.926379 19246 raft_consensus.cc:515] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.928599 19246 leader_election.cc:290] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
W20260709 12:30:59.928828 18997 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:59.929665 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087" is_pre_election: true
I20260709 12:30:59.930264 19052 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 0.
I20260709 12:30:59.930164 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324846bcca064a8a9042f16b4c675165" is_pre_election: true
I20260709 12:30:59.931313 19201 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 0.
I20260709 12:30:59.931753 19091 leader_election.cc:304] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38958d7b020c472ca77e3192fab9d37e, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.932664 19246 raft_consensus.cc:2804] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.933092 19246 raft_consensus.cc:493] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.933379 19246 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.938452 19246 raft_consensus.cc:515] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.940047 19246 leader_election.cc:290] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 election: Requested vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:30:59.940963 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
I20260709 12:30:59.941330 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:30:59.941570 19052 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.941977 19201 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.946969 19201 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 1.
I20260709 12:30:59.947886 19052 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 1.
I20260709 12:30:59.948064 19091 leader_election.cc:304] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 38958d7b020c472ca77e3192fab9d37e; no voters: 
I20260709 12:30:59.949003 19246 raft_consensus.cc:2804] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.949532 19246 raft_consensus.cc:697] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Becoming Leader. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:30:59.950296 19246 consensus_queue.cc:237] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.960146 18948 catalog_manager.cc:5697] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38958d7b020c472ca77e3192fab9d37e (127.5.4.66). New cstate: current_term: 1 leader_uuid: "38958d7b020c472ca77e3192fab9d37e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:59.965392 19246 raft_consensus.cc:493] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:59.966120 19246 raft_consensus.cc:515] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.968310 19246 leader_election.cc:290] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:30:59.969022 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087" is_pre_election: true
I20260709 12:30:59.969482 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324846bcca064a8a9042f16b4c675165" is_pre_election: true
I20260709 12:30:59.969728 19052 raft_consensus.cc:2468] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 0.
I20260709 12:30:59.970268 19201 raft_consensus.cc:2468] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 0.
I20260709 12:30:59.970803 19091 leader_election.cc:304] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38958d7b020c472ca77e3192fab9d37e, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.971698 19246 raft_consensus.cc:2804] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:59.972117 19246 raft_consensus.cc:493] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:59.972412 19246 raft_consensus.cc:3060] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.978602 19246 raft_consensus.cc:515] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.980341 19246 leader_election.cc:290] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 election: Requested vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:30:59.981061 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
I20260709 12:30:59.981559 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:30:59.981696 19052 raft_consensus.cc:3060] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.982218 19201 raft_consensus.cc:3060] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:59.987208 19052 raft_consensus.cc:2468] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 1.
I20260709 12:30:59.988107 19201 raft_consensus.cc:2468] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 1.
I20260709 12:30:59.988405 19091 leader_election.cc:304] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38958d7b020c472ca77e3192fab9d37e, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:30:59.989307 19246 raft_consensus.cc:2804] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:59.989760 19246 raft_consensus.cc:697] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Becoming Leader. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:30:59.990648 19246 consensus_queue.cc:237] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:30:59.998348 18948 catalog_manager.cc:5697] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e reported cstate change: term changed from 0 to 1, leader changed from <none> to 38958d7b020c472ca77e3192fab9d37e (127.5.4.66). New cstate: current_term: 1 leader_uuid: "38958d7b020c472ca77e3192fab9d37e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:00.067297 19246 consensus_queue.cc:1048] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Connected to new peer: Peer: permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:00.068683 19201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c"
dest_uuid: "324846bcca064a8a9042f16b4c675165"
mode: GRACEFUL
new_leader_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
 from {username='slave'} at 127.0.0.1:34928
I20260709 12:31:00.069283 19201 raft_consensus.cc:606] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 LEADER]: Received request to transfer leadership to TS 68f4013bd92b48bf8ff45b72dc0af087
I20260709 12:31:00.081889 19256 consensus_queue.cc:1048] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:00.088114 19127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f"
dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
mode: GRACEFUL
new_leader_uuid: "324846bcca064a8a9042f16b4c675165"
 from {username='slave'} at 127.0.0.1:42420
I20260709 12:31:00.088932 19127 raft_consensus.cc:606] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Received request to transfer leadership to TS 324846bcca064a8a9042f16b4c675165
I20260709 12:31:00.095490 19246 raft_consensus.cc:993] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: : Instructing follower 324846bcca064a8a9042f16b4c675165 to start an election
I20260709 12:31:00.095959 19246 raft_consensus.cc:1081] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Signalling peer 324846bcca064a8a9042f16b4c675165 to start an election
I20260709 12:31:00.100112 19200 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f"
dest_uuid: "324846bcca064a8a9042f16b4c675165"
 from {username='slave'} at 127.0.0.1:34918
I20260709 12:31:00.100972 19200 raft_consensus.cc:493] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:00.101425 19200 raft_consensus.cc:3060] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.101632 19127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226"
dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
mode: GRACEFUL
new_leader_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
 from {username='slave'} at 127.0.0.1:42420
I20260709 12:31:00.102330 19127 raft_consensus.cc:606] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Received request to transfer leadership to TS 68f4013bd92b48bf8ff45b72dc0af087
I20260709 12:31:00.109099 19127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da"
dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
mode: GRACEFUL
new_leader_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
 from {username='slave'} at 127.0.0.1:42420
I20260709 12:31:00.109710 19127 raft_consensus.cc:606] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Received request to transfer leadership to TS 68f4013bd92b48bf8ff45b72dc0af087
I20260709 12:31:00.110034 19200 raft_consensus.cc:515] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.112637 19200 leader_election.cc:290] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [CANDIDATE]: Term 2 election: Requested vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689)
I20260709 12:31:00.113299 19052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f" candidate_uuid: "324846bcca064a8a9042f16b4c675165" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
I20260709 12:31:00.113665 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c40b8b071ab8413287c3725bba20f64f" candidate_uuid: "324846bcca064a8a9042f16b4c675165" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
I20260709 12:31:00.114863 19052 raft_consensus.cc:3060] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.115243 19127 raft_consensus.cc:3055] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:00.115636 19127 raft_consensus.cc:740] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:31:00.116482 19127 consensus_queue.cc:260] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.117743 19127 raft_consensus.cc:3060] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.122143 19052 raft_consensus.cc:2468] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 324846bcca064a8a9042f16b4c675165 in term 2.
I20260709 12:31:00.123265 19165 leader_election.cc:304] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:31:00.124217 19249 raft_consensus.cc:2804] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:00.124790 19249 raft_consensus.cc:697] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 2 LEADER]: Becoming Leader. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Running, Role: LEADER
I20260709 12:31:00.126098 19127 raft_consensus.cc:2468] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 324846bcca064a8a9042f16b4c675165 in term 2.
I20260709 12:31:00.125900 19249 consensus_queue.cc:237] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.135115 18949 catalog_manager.cc:5697] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 reported cstate change: term changed from 1 to 2, leader changed from 38958d7b020c472ca77e3192fab9d37e (127.5.4.66) to 324846bcca064a8a9042f16b4c675165 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "324846bcca064a8a9042f16b4c675165" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:00.159476 19249 consensus_queue.cc:1048] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:00.166622 19258 raft_consensus.cc:993] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: : Instructing follower 68f4013bd92b48bf8ff45b72dc0af087 to start an election
I20260709 12:31:00.167292 19249 raft_consensus.cc:1081] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 LEADER]: Signalling peer 68f4013bd92b48bf8ff45b72dc0af087 to start an election
I20260709 12:31:00.170826 19051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c"
dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
 from {username='slave'} at 127.0.0.1:59096
I20260709 12:31:00.172830 19051 raft_consensus.cc:493] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:00.173230 19051 raft_consensus.cc:3060] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.180055 19280 consensus_queue.cc:1048] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:00.180709 19051 raft_consensus.cc:515] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:31:00.191464 19051 leader_election.cc:290] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [CANDIDATE]: Term 2 election: Requested vote from peers 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875), 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689)
I20260709 12:31:00.206727 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c" candidate_uuid: "68f4013bd92b48bf8ff45b72dc0af087" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:31:00.207276 19200 raft_consensus.cc:3055] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:00.207561 19200 raft_consensus.cc:740] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 324846bcca064a8a9042f16b4c675165, State: Running, Role: LEADER
I20260709 12:31:00.208386 19200 consensus_queue.cc:260] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:31:00.209686 19200 raft_consensus.cc:3060] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.212648 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2967db3c2e7c408f8b8c9ea4d720b05c" candidate_uuid: "68f4013bd92b48bf8ff45b72dc0af087" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
I20260709 12:31:00.213418 19127 raft_consensus.cc:3060] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.216750 19200 raft_consensus.cc:2468] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f4013bd92b48bf8ff45b72dc0af087 in term 2.
I20260709 12:31:00.218058 19016 leader_election.cc:304] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:31:00.219120 19248 raft_consensus.cc:2804] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:00.220423 19248 raft_consensus.cc:697] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Becoming Leader. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Running, Role: LEADER
I20260709 12:31:00.221370 19248 consensus_queue.cc:237] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } }
I20260709 12:31:00.222604 19127 raft_consensus.cc:2468] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f4013bd92b48bf8ff45b72dc0af087 in term 2.
I20260709 12:31:00.228439 18949 catalog_manager.cc:5697] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 reported cstate change: term changed from 1 to 2, leader changed from 324846bcca064a8a9042f16b4c675165 (127.5.4.67) to 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "68f4013bd92b48bf8ff45b72dc0af087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:00.384488 19256 consensus_queue.cc:1048] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Connected to new peer: Peer: permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:00.397214 19286 consensus_queue.cc:1048] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:00.407181 19256 raft_consensus.cc:993] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: : Instructing follower 68f4013bd92b48bf8ff45b72dc0af087 to start an election
I20260709 12:31:00.407763 19286 raft_consensus.cc:1081] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Signalling peer 68f4013bd92b48bf8ff45b72dc0af087 to start an election
I20260709 12:31:00.409595 19051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226"
dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
 from {username='slave'} at 127.0.0.1:59084
I20260709 12:31:00.410178 19051 raft_consensus.cc:493] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:00.410604 19051 raft_consensus.cc:3060] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.418449 19051 raft_consensus.cc:515] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.420810 19051 leader_election.cc:290] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [CANDIDATE]: Term 2 election: Requested vote from peers 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:31:00.421681 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226" candidate_uuid: "68f4013bd92b48bf8ff45b72dc0af087" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
I20260709 12:31:00.422307 19200 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b00ddb53b9a4c21a3f4aa748e5e4226" candidate_uuid: "68f4013bd92b48bf8ff45b72dc0af087" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:31:00.422675 19127 raft_consensus.cc:3055] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:00.423149 19200 raft_consensus.cc:3060] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.423173 19127 raft_consensus.cc:740] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:31:00.424495 19127 consensus_queue.cc:260] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.425875 19127 raft_consensus.cc:3060] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.426959 19286 consensus_queue.cc:1048] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Connected to new peer: Peer: permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:00.431798 19200 raft_consensus.cc:2468] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f4013bd92b48bf8ff45b72dc0af087 in term 2.
I20260709 12:31:00.433378 19016 leader_election.cc:304] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:31:00.434468 19248 raft_consensus.cc:2804] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:00.434916 19248 raft_consensus.cc:697] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Becoming Leader. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Running, Role: LEADER
I20260709 12:31:00.435779 19248 consensus_queue.cc:237] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.441124 19127 raft_consensus.cc:2468] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f4013bd92b48bf8ff45b72dc0af087 in term 2.
I20260709 12:31:00.447440 19256 consensus_queue.cc:1048] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:00.448695 18948 catalog_manager.cc:5697] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 reported cstate change: term changed from 1 to 2, leader changed from 38958d7b020c472ca77e3192fab9d37e (127.5.4.66) to 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "68f4013bd92b48bf8ff45b72dc0af087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:00.456315 19246 raft_consensus.cc:993] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: : Instructing follower 68f4013bd92b48bf8ff45b72dc0af087 to start an election
I20260709 12:31:00.456851 19246 raft_consensus.cc:1081] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Signalling peer 68f4013bd92b48bf8ff45b72dc0af087 to start an election
I20260709 12:31:00.458647 19051 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da"
dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
 from {username='slave'} at 127.0.0.1:59084
I20260709 12:31:00.459297 19051 raft_consensus.cc:493] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:00.459682 19051 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.465339 19051 raft_consensus.cc:515] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.467396 19051 leader_election.cc:290] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [CANDIDATE]: Term 2 election: Requested vote from peers 38958d7b020c472ca77e3192fab9d37e (127.5.4.66:38689), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:31:00.468091 19127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "68f4013bd92b48bf8ff45b72dc0af087" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
I20260709 12:31:00.468609 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "68f4013bd92b48bf8ff45b72dc0af087" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:31:00.468832 19127 raft_consensus.cc:3055] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:00.469302 19127 raft_consensus.cc:740] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:31:00.469302 19201 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.470192 19127 consensus_queue.cc:260] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.472807 19127 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:00.475991 19201 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f4013bd92b48bf8ff45b72dc0af087 in term 2.
I20260709 12:31:00.477113 19016 leader_election.cc:304] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 68f4013bd92b48bf8ff45b72dc0af087; no voters: 
I20260709 12:31:00.477886 19248 raft_consensus.cc:2804] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:00.478428 19248 raft_consensus.cc:697] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Becoming Leader. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Running, Role: LEADER
I20260709 12:31:00.478565 19127 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68f4013bd92b48bf8ff45b72dc0af087 in term 2.
I20260709 12:31:00.479343 19248 consensus_queue.cc:237] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.486102 18948 catalog_manager.cc:5697] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 reported cstate change: term changed from 1 to 2, leader changed from 38958d7b020c472ca77e3192fab9d37e (127.5.4.66) to 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "68f4013bd92b48bf8ff45b72dc0af087" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:00.599066  5137 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:31:00.600847  5137 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:31:00.601263  5137 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:31:00.614964 19051 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da"
dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
mode: GRACEFUL
new_leader_uuid: "38958d7b020c472ca77e3192fab9d37e"
 from {username='slave'} at 127.0.0.1:59110
I20260709 12:31:00.615694 19051 raft_consensus.cc:606] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Received request to transfer leadership to TS 38958d7b020c472ca77e3192fab9d37e
I20260709 12:31:00.616966  5137 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:31:00.671214 19127 raft_consensus.cc:1275] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Refusing update from remote peer 324846bcca064a8a9042f16b4c675165: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.672567 19280 consensus_queue.cc:1048] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.677095 19126 raft_consensus.cc:1275] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Refusing update from remote peer 68f4013bd92b48bf8ff45b72dc0af087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.679116 19248 consensus_queue.cc:1048] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.694279 19051 raft_consensus.cc:1275] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Refusing update from remote peer 324846bcca064a8a9042f16b4c675165: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.692335 19201 raft_consensus.cc:1275] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Refusing update from remote peer 68f4013bd92b48bf8ff45b72dc0af087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.698959 19284 consensus_queue.cc:1048] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.699337 19293 consensus_queue.cc:1048] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:00.847667 19201 raft_consensus.cc:1275] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Refusing update from remote peer 68f4013bd92b48bf8ff45b72dc0af087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.849088 19284 consensus_queue.cc:1048] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.857508 19126 raft_consensus.cc:1275] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Refusing update from remote peer 68f4013bd92b48bf8ff45b72dc0af087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.859910 19284 consensus_queue.cc:1048] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.932091 19126 raft_consensus.cc:1275] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Refusing update from remote peer 68f4013bd92b48bf8ff45b72dc0af087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.933856 19284 consensus_queue.cc:1048] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.940331 19248 raft_consensus.cc:993] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: : Instructing follower 38958d7b020c472ca77e3192fab9d37e to start an election
I20260709 12:31:00.941036 19294 raft_consensus.cc:1081] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Signalling peer 38958d7b020c472ca77e3192fab9d37e to start an election
I20260709 12:31:00.943487 19127 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da"
dest_uuid: "38958d7b020c472ca77e3192fab9d37e"
 from {username='slave'} at 127.0.0.1:42426
I20260709 12:31:00.944087 19127 raft_consensus.cc:493] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:00.944495 19127 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:00.944855 19201 raft_consensus.cc:1275] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Refusing update from remote peer 68f4013bd92b48bf8ff45b72dc0af087: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:00.946568 19294 consensus_queue.cc:1048] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [LEADER]: Connected to new peer: Peer: permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.952685 19127 raft_consensus.cc:515] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.953888 19126 raft_consensus.cc:1240] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 FOLLOWER]: Rejecting Update request from peer 68f4013bd92b48bf8ff45b72dc0af087 for earlier term 2. Current term is 3. Ops: []
I20260709 12:31:00.955169 19127 leader_election.cc:290] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 3 election: Requested vote from peers 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65:34177), 324846bcca064a8a9042f16b4c675165 (127.5.4.67:34875)
I20260709 12:31:00.955781 19051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "68f4013bd92b48bf8ff45b72dc0af087"
I20260709 12:31:00.956440 19051 raft_consensus.cc:3055] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:00.956791 19051 raft_consensus.cc:740] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 68f4013bd92b48bf8ff45b72dc0af087, State: Running, Role: LEADER
I20260709 12:31:00.956813 19201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d31d523efa047cabc3e39438ded71da" candidate_uuid: "38958d7b020c472ca77e3192fab9d37e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "324846bcca064a8a9042f16b4c675165"
I20260709 12:31:00.957647 19201 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:00.957847 19051 consensus_queue.cc:260] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.958801 19294 consensus_queue.cc:1059] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:00.962729 19051 raft_consensus.cc:3060] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:00.964874 19201 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 3.
I20260709 12:31:00.966292 19091 leader_election.cc:304] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 324846bcca064a8a9042f16b4c675165, 38958d7b020c472ca77e3192fab9d37e; no voters: 
I20260709 12:31:00.967141 19246 raft_consensus.cc:2804] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:00.968495 19246 raft_consensus.cc:697] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 LEADER]: Becoming Leader. State: Replica: 38958d7b020c472ca77e3192fab9d37e, State: Running, Role: LEADER
I20260709 12:31:00.969355 19246 consensus_queue.cc:237] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } }
I20260709 12:31:00.970022 19051 raft_consensus.cc:2468] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38958d7b020c472ca77e3192fab9d37e in term 3.
I20260709 12:31:00.976922 18949 catalog_manager.cc:5697] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e reported cstate change: term changed from 2 to 3, leader changed from 68f4013bd92b48bf8ff45b72dc0af087 (127.5.4.65) to 38958d7b020c472ca77e3192fab9d37e (127.5.4.66). New cstate: current_term: 3 leader_uuid: "38958d7b020c472ca77e3192fab9d37e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68f4013bd92b48bf8ff45b72dc0af087" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 34177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38958d7b020c472ca77e3192fab9d37e" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 38689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "324846bcca064a8a9042f16b4c675165" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 34875 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:01.255263  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
I20260709 12:31:01.280622  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:01.281308  5137 tablet_replica.cc:333] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087: stopping tablet replica
I20260709 12:31:01.281898  5137 raft_consensus.cc:2243] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:01.282805  5137 raft_consensus.cc:2272] T 2967db3c2e7c408f8b8c9ea4d720b05c P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.285424  5137 tablet_replica.cc:333] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087: stopping tablet replica
I20260709 12:31:01.285966  5137 raft_consensus.cc:2243] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.286389  5137 raft_consensus.cc:2272] T 7d31d523efa047cabc3e39438ded71da P 68f4013bd92b48bf8ff45b72dc0af087 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.288690  5137 tablet_replica.cc:333] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087: stopping tablet replica
I20260709 12:31:01.289204  5137 raft_consensus.cc:2243] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:01.289987  5137 raft_consensus.cc:2272] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.292320  5137 tablet_replica.cc:333] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087: stopping tablet replica
I20260709 12:31:01.292812  5137 raft_consensus.cc:2243] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.293231  5137 raft_consensus.cc:2272] T c40b8b071ab8413287c3725bba20f64f P 68f4013bd92b48bf8ff45b72dc0af087 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.307070  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
I20260709 12:31:01.322911  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
I20260709 12:31:01.346712  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:01.347486  5137 tablet_replica.cc:333] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e: stopping tablet replica
I20260709 12:31:01.348161  5137 raft_consensus.cc:2243] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.348661  5137 raft_consensus.cc:2272] T 2967db3c2e7c408f8b8c9ea4d720b05c P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.351751  5137 tablet_replica.cc:333] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e: stopping tablet replica
I20260709 12:31:01.352352  5137 raft_consensus.cc:2243] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.352862  5137 raft_consensus.cc:2272] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.355365  5137 tablet_replica.cc:333] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: stopping tablet replica
I20260709 12:31:01.355921  5137 raft_consensus.cc:2243] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:01.356803  5137 pending_rounds.cc:70] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Trying to abort 2 pending ops.
I20260709 12:31:01.357033  5137 pending_rounds.cc:77] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626668939362304 op_type: NO_OP noop_request { }
I20260709 12:31:01.357358  5137 raft_consensus.cc:2889] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:31:01.357604  5137 pending_rounds.cc:77] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626666932006912 op_type: NO_OP noop_request { }
I20260709 12:31:01.357878  5137 raft_consensus.cc:2889] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:31:01.358187  5137 raft_consensus.cc:2272] T 7d31d523efa047cabc3e39438ded71da P 38958d7b020c472ca77e3192fab9d37e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.360639  5137 tablet_replica.cc:333] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e: stopping tablet replica
I20260709 12:31:01.361167  5137 raft_consensus.cc:2243] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.361629  5137 raft_consensus.cc:2272] T c40b8b071ab8413287c3725bba20f64f P 38958d7b020c472ca77e3192fab9d37e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.387516  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
I20260709 12:31:01.402884  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
I20260709 12:31:01.424206  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:01.424808  5137 tablet_replica.cc:333] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165: stopping tablet replica
I20260709 12:31:01.425423  5137 raft_consensus.cc:2243] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.425940  5137 raft_consensus.cc:2272] T 7d31d523efa047cabc3e39438ded71da P 324846bcca064a8a9042f16b4c675165 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.428809  5137 tablet_replica.cc:333] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165: stopping tablet replica
I20260709 12:31:01.429361  5137 raft_consensus.cc:2243] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.429808  5137 raft_consensus.cc:2272] T 6b00ddb53b9a4c21a3f4aa748e5e4226 P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.432683  5137 tablet_replica.cc:333] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165: stopping tablet replica
I20260709 12:31:01.433332  5137 raft_consensus.cc:2243] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:01.434324  5137 raft_consensus.cc:2272] T c40b8b071ab8413287c3725bba20f64f P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.437516  5137 tablet_replica.cc:333] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165: stopping tablet replica
I20260709 12:31:01.438099  5137 raft_consensus.cc:2243] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:01.438678  5137 raft_consensus.cc:2272] T 2967db3c2e7c408f8b8c9ea4d720b05c P 324846bcca064a8a9042f16b4c675165 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.464972  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:31:01.480562  5137 master.cc:562] Master@127.5.4.126:42927 shutting down...
I20260709 12:31:01.501663  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:01.502233  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:01.502595  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9f5445a57c764969a7be646f6f3f4704: stopping tablet replica
I20260709 12:31:01.523547  5137 master.cc:584] Master@127.5.4.126:42927 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2764 ms)
[----------] 6 tests from LeaderRebalancerTest (455781 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:31:01.554761  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:34849
I20260709 12:31:01.555888  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:01.561028 19298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.562530 19299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.562906 19301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:01.563514  5137 server_base.cc:1061] running on GCE node
I20260709 12:31:01.564327  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:01.564533  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:01.564702  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600261564665 us; error 0 us; skew 500 ppm
I20260709 12:31:01.565260  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:01.567813  5137 webserver.cc:533] Webserver started at http://127.5.4.126:33047/ using document root <none> and password file <none>
I20260709 12:31:01.568284  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:01.568475  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:01.568750  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:01.569796  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "c2357f374958434d988d6958ee7a75ab"
format_stamp: "Formatted at 2026-07-09 12:31:01 on dist-test-slave-lg8f"
I20260709 12:31:01.574250  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:31:01.578444 19306 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.579423  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:01.579782  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "c2357f374958434d988d6958ee7a75ab"
format_stamp: "Formatted at 2026-07-09 12:31:01 on dist-test-slave-lg8f"
I20260709 12:31:01.580066  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:01.606582  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:01.607692  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:01.647425  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:34849
I20260709 12:31:01.647581 19357 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:34849 every 8 connection(s)
I20260709 12:31:01.652107 19358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:01.664665 19358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab: Bootstrap starting.
I20260709 12:31:01.669620 19358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:01.674098 19358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab: No bootstrap required, opened a new log
I20260709 12:31:01.676306 19358 raft_consensus.cc:359] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2357f374958434d988d6958ee7a75ab" member_type: VOTER }
I20260709 12:31:01.676687 19358 raft_consensus.cc:385] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:01.676889 19358 raft_consensus.cc:740] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2357f374958434d988d6958ee7a75ab, State: Initialized, Role: FOLLOWER
I20260709 12:31:01.677426 19358 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2357f374958434d988d6958ee7a75ab" member_type: VOTER }
I20260709 12:31:01.677873 19358 raft_consensus.cc:399] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:31:01.678074 19358 raft_consensus.cc:493] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:31:01.678299 19358 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:01.683611 19358 raft_consensus.cc:515] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2357f374958434d988d6958ee7a75ab" member_type: VOTER }
I20260709 12:31:01.684120 19358 leader_election.cc:304] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c2357f374958434d988d6958ee7a75ab; no voters: 
I20260709 12:31:01.685408 19358 leader_election.cc:290] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:31:01.686058 19361 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:01.687773 19361 raft_consensus.cc:697] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 1 LEADER]: Becoming Leader. State: Replica: c2357f374958434d988d6958ee7a75ab, State: Running, Role: LEADER
I20260709 12:31:01.688532 19361 consensus_queue.cc:237] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2357f374958434d988d6958ee7a75ab" member_type: VOTER }
I20260709 12:31:01.688657 19358 sys_catalog.cc:565] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:31:01.692215 19362 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c2357f374958434d988d6958ee7a75ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2357f374958434d988d6958ee7a75ab" member_type: VOTER } }
I20260709 12:31:01.692267 19363 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader c2357f374958434d988d6958ee7a75ab. Latest consensus state: current_term: 1 leader_uuid: "c2357f374958434d988d6958ee7a75ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c2357f374958434d988d6958ee7a75ab" member_type: VOTER } }
I20260709 12:31:01.693081 19362 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:01.693163 19363 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:01.699126 19367 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:31:01.704986 19367 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:31:01.709492  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:31:01.715735 19367 catalog_manager.cc:1383] Generated new cluster ID: 658258cbdfaf416585af726aab36527b
I20260709 12:31:01.716064 19367 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:31:01.753120 19367 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:31:01.754428 19367 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:31:01.770352 19367 catalog_manager.cc:6070] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab: Generated new TSK 0
I20260709 12:31:01.771003 19367 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:31:01.777062  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:01.784303 19379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.785781 19380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:01.787663  5137 server_base.cc:1061] running on GCE node
W20260709 12:31:01.787698 19382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:01.789202  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:01.789455  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:01.789640  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600261789624 us; error 0 us; skew 500 ppm
I20260709 12:31:01.790279  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:01.793000  5137 webserver.cc:533] Webserver started at http://127.5.4.65:45427/ using document root <none> and password file <none>
I20260709 12:31:01.793524  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:01.793704  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:01.793962  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:01.795274  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
format_stamp: "Formatted at 2026-07-09 12:31:01 on dist-test-slave-lg8f"
I20260709 12:31:01.800489  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:31:01.804601 19387 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.805522  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:31:01.805933  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
format_stamp: "Formatted at 2026-07-09 12:31:01 on dist-test-slave-lg8f"
I20260709 12:31:01.806250  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:01.819322  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:01.820693  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:01.822358  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:01.825114  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:01.825366  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.825645  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:01.825853  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.876784  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:40457
I20260709 12:31:01.876937 19449 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:40457 every 8 connection(s)
I20260709 12:31:01.881933  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:01.890687 19454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.891284 19455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.895900 19457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:01.896750  5137 server_base.cc:1061] running on GCE node
I20260709 12:31:01.897603 19450 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34849
I20260709 12:31:01.897958  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:31:01.898093 19450 heartbeater.cc:461] Registering TS with master...
W20260709 12:31:01.898281  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:01.898548  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600261898525 us; error 0 us; skew 500 ppm
I20260709 12:31:01.899173 19450 heartbeater.cc:507] Master 127.5.4.126:34849 requested a full tablet report, sending...
I20260709 12:31:01.899436  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:01.901845 19323 ts_manager.cc:194] Registered new tserver with Master: d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457)
I20260709 12:31:01.902570  5137 webserver.cc:533] Webserver started at http://127.5.4.66:45697/ using document root <none> and password file <none>
I20260709 12:31:01.903195  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:01.903467  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:01.903757  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:01.904083 19323 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36078
I20260709 12:31:01.905107  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "051c8a7691c2489bb513f0e150a6089b"
format_stamp: "Formatted at 2026-07-09 12:31:01 on dist-test-slave-lg8f"
I20260709 12:31:01.910111  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20260709 12:31:01.913829 19462 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.914865  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:01.915158  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "051c8a7691c2489bb513f0e150a6089b"
format_stamp: "Formatted at 2026-07-09 12:31:01 on dist-test-slave-lg8f"
I20260709 12:31:01.915424  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:01.935236  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:01.936262  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:01.937670  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:01.940035  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:01.940235  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.940542  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:01.940712  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:01.982106  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:44361
I20260709 12:31:01.982250 19524 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:44361 every 8 connection(s)
I20260709 12:31:01.987037  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:01.994705 19528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.994987 19529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:01.998668 19531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:01.999756  5137 server_base.cc:1061] running on GCE node
I20260709 12:31:02.000887  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:31:02.000923 19525 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34849
W20260709 12:31:02.001154  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:02.001590  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600262001565 us; error 0 us; skew 500 ppm
I20260709 12:31:02.001619 19525 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:02.002255  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:02.002746 19525 heartbeater.cc:507] Master 127.5.4.126:34849 requested a full tablet report, sending...
I20260709 12:31:02.005306  5137 webserver.cc:533] Webserver started at http://127.5.4.67:39465/ using document root <none> and password file <none>
I20260709 12:31:02.005520 19323 ts_manager.cc:194] Registered new tserver with Master: 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:02.005988  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:02.006265  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:02.006592  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:02.007439 19323 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36084
I20260709 12:31:02.007767  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
format_stamp: "Formatted at 2026-07-09 12:31:02 on dist-test-slave-lg8f"
I20260709 12:31:02.012682  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:31:02.016402 19536 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:02.017369  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 12:31:02.017724  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
format_stamp: "Formatted at 2026-07-09 12:31:02 on dist-test-slave-lg8f"
I20260709 12:31:02.018023  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:02.043124  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:02.044188  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:02.045957  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:02.048242  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:02.048470  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:02.048713  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:02.048887  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:02.089645  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:44555
I20260709 12:31:02.089787 19598 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:44555 every 8 connection(s)
I20260709 12:31:02.104748 19599 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34849
I20260709 12:31:02.105324 19599 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:02.106079 19599 heartbeater.cc:507] Master 127.5.4.126:34849 requested a full tablet report, sending...
I20260709 12:31:02.108242 19323 ts_manager.cc:194] Registered new tserver with Master: a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.109056  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015814226s
I20260709 12:31:02.109953 19323 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36098
I20260709 12:31:02.135965 19323 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36112:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:02.138850 19323 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:02.266011 19415 tablet_service.cc:1511] Processing CreateTablet for tablet 9e077c0962e947c6bd92d06245f244fd (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:02.268476 19415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e077c0962e947c6bd92d06245f244fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.271842 19414 tablet_service.cc:1511] Processing CreateTablet for tablet 9372849a880b49e49e1753cc6f57c87e (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:02.273334 19414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9372849a880b49e49e1753cc6f57c87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.276171 19413 tablet_service.cc:1511] Processing CreateTablet for tablet 1df2d8c357c44bc08e7478ac539f508a (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:02.277962 19413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df2d8c357c44bc08e7478ac539f508a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.279686 19490 tablet_service.cc:1511] Processing CreateTablet for tablet 9e077c0962e947c6bd92d06245f244fd (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:02.281606 19490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e077c0962e947c6bd92d06245f244fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.281956 19412 tablet_service.cc:1511] Processing CreateTablet for tablet 013fb25300cb44e6ac3d9a1ece427992 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:02.283541 19412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013fb25300cb44e6ac3d9a1ece427992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.283912 19489 tablet_service.cc:1511] Processing CreateTablet for tablet 9372849a880b49e49e1753cc6f57c87e (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:02.285431 19489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9372849a880b49e49e1753cc6f57c87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.287920 19411 tablet_service.cc:1511] Processing CreateTablet for tablet 3c642d6fa088414e84482a66916fe5cf (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:02.289570 19411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c642d6fa088414e84482a66916fe5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.289321 19487 tablet_service.cc:1511] Processing CreateTablet for tablet 013fb25300cb44e6ac3d9a1ece427992 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:02.291899 19486 tablet_service.cc:1511] Processing CreateTablet for tablet 3c642d6fa088414e84482a66916fe5cf (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:02.293555 19486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c642d6fa088414e84482a66916fe5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.295293 19410 tablet_service.cc:1511] Processing CreateTablet for tablet 922473629e704fbabcee362b44ed7409 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:02.296919 19410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 922473629e704fbabcee362b44ed7409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.287912 19488 tablet_service.cc:1511] Processing CreateTablet for tablet 1df2d8c357c44bc08e7478ac539f508a (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:02.298130 19488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df2d8c357c44bc08e7478ac539f508a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.298353 19407 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2f618923fa4bb9bde5f0acd9a47fb2 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:02.299851 19409 tablet_service.cc:1511] Processing CreateTablet for tablet 9f21d676c01f48bab23326a671a9c6b1 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:02.302331 19487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013fb25300cb44e6ac3d9a1ece427992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.302246 19408 tablet_service.cc:1511] Processing CreateTablet for tablet 9b0aeda8cdb248f2833025fda78b4f9c (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:02.299901 19407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2f618923fa4bb9bde5f0acd9a47fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.310915 19408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0aeda8cdb248f2833025fda78b4f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.313793 19409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f21d676c01f48bab23326a671a9c6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.321764 19485 tablet_service.cc:1511] Processing CreateTablet for tablet 922473629e704fbabcee362b44ed7409 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:02.323563 19485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 922473629e704fbabcee362b44ed7409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.328531 19484 tablet_service.cc:1511] Processing CreateTablet for tablet 9f21d676c01f48bab23326a671a9c6b1 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:02.330174 19484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f21d676c01f48bab23326a671a9c6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.334748 19483 tablet_service.cc:1511] Processing CreateTablet for tablet 9b0aeda8cdb248f2833025fda78b4f9c (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:02.336297 19483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0aeda8cdb248f2833025fda78b4f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.341630 19482 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2f618923fa4bb9bde5f0acd9a47fb2 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:02.344134 19482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2f618923fa4bb9bde5f0acd9a47fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.348277 19615 tablet_bootstrap.cc:492] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:02.363895 19564 tablet_service.cc:1511] Processing CreateTablet for tablet 9e077c0962e947c6bd92d06245f244fd (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:02.366791 19557 tablet_service.cc:1511] Processing CreateTablet for tablet 9b0aeda8cdb248f2833025fda78b4f9c (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:02.368688 19557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b0aeda8cdb248f2833025fda78b4f9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.371905 19562 tablet_service.cc:1511] Processing CreateTablet for tablet 1df2d8c357c44bc08e7478ac539f508a (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:02.373442 19562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df2d8c357c44bc08e7478ac539f508a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.367069 19556 tablet_service.cc:1511] Processing CreateTablet for tablet 8b2f618923fa4bb9bde5f0acd9a47fb2 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:02.376336 19561 tablet_service.cc:1511] Processing CreateTablet for tablet 013fb25300cb44e6ac3d9a1ece427992 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:02.378433 19560 tablet_service.cc:1511] Processing CreateTablet for tablet 3c642d6fa088414e84482a66916fe5cf (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:02.379853 19558 tablet_service.cc:1511] Processing CreateTablet for tablet 9f21d676c01f48bab23326a671a9c6b1 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:02.381255 19563 tablet_service.cc:1511] Processing CreateTablet for tablet 9372849a880b49e49e1753cc6f57c87e (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:02.382596 19559 tablet_service.cc:1511] Processing CreateTablet for tablet 922473629e704fbabcee362b44ed7409 (DEFAULT_TABLE table=test-workload [id=af92e3f341764fb29199647b0ed936cc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:02.385818 19615 tablet_bootstrap.cc:654] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.385900 19556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b2f618923fa4bb9bde5f0acd9a47fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.401353 19561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 013fb25300cb44e6ac3d9a1ece427992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.403005 19558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f21d676c01f48bab23326a671a9c6b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.403919 19563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9372849a880b49e49e1753cc6f57c87e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.406121 19560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c642d6fa088414e84482a66916fe5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.413228 19559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 922473629e704fbabcee362b44ed7409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.447029 19564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e077c0962e947c6bd92d06245f244fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:02.482299 19616 tablet_bootstrap.cc:492] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:02.560302 19615 tablet_bootstrap.cc:492] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:02.561021 19615 ts_tablet_manager.cc:1403] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.213s	user 0.033s	sys 0.078s
I20260709 12:31:02.564836 19615 raft_consensus.cc:359] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.565965 19615 raft_consensus.cc:385] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.566286 19615 raft_consensus.cc:740] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.567353 19615 consensus_queue.cc:260] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.570868 19616 tablet_bootstrap.cc:654] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.584648 19620 tablet_bootstrap.cc:492] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:02.595136 19620 tablet_bootstrap.cc:654] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.597568 19616 tablet_bootstrap.cc:492] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.598687 19616 ts_tablet_manager.cc:1403] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.117s	user 0.011s	sys 0.019s
I20260709 12:31:02.604647 19616 raft_consensus.cc:359] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.605360 19616 raft_consensus.cc:385] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.605643 19616 raft_consensus.cc:740] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.606550 19616 consensus_queue.cc:260] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.616320 19525 heartbeater.cc:499] Master 127.5.4.126:34849 was elected leader, sending a full tablet report...
I20260709 12:31:02.617772 19616 ts_tablet_manager.cc:1434] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.004s
I20260709 12:31:02.618970 19616 tablet_bootstrap.cc:492] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:02.625429 19616 tablet_bootstrap.cc:654] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.636792 19616 tablet_bootstrap.cc:492] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.637288 19616 ts_tablet_manager.cc:1403] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.019s	user 0.003s	sys 0.013s
I20260709 12:31:02.640246 19616 raft_consensus.cc:359] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.640985 19616 raft_consensus.cc:385] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.641319 19616 raft_consensus.cc:740] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.640409 19615 ts_tablet_manager.cc:1434] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.079s	user 0.027s	sys 0.024s
I20260709 12:31:02.642031 19450 heartbeater.cc:499] Master 127.5.4.126:34849 was elected leader, sending a full tablet report...
I20260709 12:31:02.642143 19616 consensus_queue.cc:260] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.643471 19615 tablet_bootstrap.cc:492] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:02.644996 19616 ts_tablet_manager.cc:1434] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:31:02.646126 19616 tablet_bootstrap.cc:492] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:02.653158 19616 tablet_bootstrap.cc:654] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.653380 19615 tablet_bootstrap.cc:654] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.663610 19616 tablet_bootstrap.cc:492] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.664218 19616 ts_tablet_manager.cc:1403] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 12:31:02.665011 19620 tablet_bootstrap.cc:492] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:02.665635 19620 ts_tablet_manager.cc:1403] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.081s	user 0.034s	sys 0.034s
I20260709 12:31:02.667609 19616 raft_consensus.cc:359] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.668581 19616 raft_consensus.cc:385] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.668915 19616 raft_consensus.cc:740] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.669166 19620 raft_consensus.cc:359] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.670063 19620 raft_consensus.cc:385] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.670696 19620 raft_consensus.cc:740] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.669974 19616 consensus_queue.cc:260] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.671566 19620 consensus_queue.cc:260] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.673216 19616 ts_tablet_manager.cc:1434] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:31:02.674559 19616 tablet_bootstrap.cc:492] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:02.675189 19615 tablet_bootstrap.cc:492] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:02.675825 19615 ts_tablet_manager.cc:1403] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.004s
I20260709 12:31:02.675880 19599 heartbeater.cc:499] Master 127.5.4.126:34849 was elected leader, sending a full tablet report...
I20260709 12:31:02.679453 19615 raft_consensus.cc:359] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.680229 19615 raft_consensus.cc:385] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.680533 19615 raft_consensus.cc:740] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.682785 19620 ts_tablet_manager.cc:1434] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.004s
I20260709 12:31:02.683940 19620 tablet_bootstrap.cc:492] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:02.684036 19616 tablet_bootstrap.cc:654] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.692071 19616 tablet_bootstrap.cc:492] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.692508 19620 tablet_bootstrap.cc:654] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.692567 19616 ts_tablet_manager.cc:1403] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.018s	user 0.003s	sys 0.011s
I20260709 12:31:02.695574 19616 raft_consensus.cc:359] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.696177 19616 raft_consensus.cc:385] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.696502 19616 raft_consensus.cc:740] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.696570 19615 consensus_queue.cc:260] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.697474 19616 consensus_queue.cc:260] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.701782 19615 ts_tablet_manager.cc:1434] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.001s
I20260709 12:31:02.702445 19616 ts_tablet_manager.cc:1434] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 12:31:02.703128 19615 tablet_bootstrap.cc:492] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:02.703752 19616 tablet_bootstrap.cc:492] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
W20260709 12:31:02.704694 19371 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:02.708559 19620 tablet_bootstrap.cc:492] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:02.709215 19620 ts_tablet_manager.cc:1403] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.005s
I20260709 12:31:02.711946 19616 tablet_bootstrap.cc:654] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.712639 19620 raft_consensus.cc:359] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.713443 19620 raft_consensus.cc:385] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.713950 19620 raft_consensus.cc:740] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.711946 19615 tablet_bootstrap.cc:654] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.714954 19620 consensus_queue.cc:260] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.717367 19620 ts_tablet_manager.cc:1434] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:31:02.718650 19620 tablet_bootstrap.cc:492] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:02.726451 19620 tablet_bootstrap.cc:654] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.728606 19615 tablet_bootstrap.cc:492] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:02.729303 19615 ts_tablet_manager.cc:1403] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.004s
I20260709 12:31:02.730779 19616 tablet_bootstrap.cc:492] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.732251 19616 ts_tablet_manager.cc:1403] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.009s
I20260709 12:31:02.732928 19615 raft_consensus.cc:359] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.733672 19615 raft_consensus.cc:385] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.734017 19615 raft_consensus.cc:740] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.734845 19620 tablet_bootstrap.cc:492] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:02.734995 19615 consensus_queue.cc:260] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.735476 19620 ts_tablet_manager.cc:1403] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.017s	user 0.004s	sys 0.011s
I20260709 12:31:02.735721 19616 raft_consensus.cc:359] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.736544 19616 raft_consensus.cc:385] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.736847 19616 raft_consensus.cc:740] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.737882 19616 consensus_queue.cc:260] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.738950 19620 raft_consensus.cc:359] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.739380 19615 ts_tablet_manager.cc:1434] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:31:02.739717 19620 raft_consensus.cc:385] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.740065 19620 raft_consensus.cc:740] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.740394 19616 ts_tablet_manager.cc:1434] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:31:02.740523 19615 tablet_bootstrap.cc:492] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:02.741017 19620 consensus_queue.cc:260] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.743866 19620 ts_tablet_manager.cc:1434] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:31:02.744124 19616 tablet_bootstrap.cc:492] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:02.744948 19620 tablet_bootstrap.cc:492] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:02.748607 19615 tablet_bootstrap.cc:654] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.752381 19616 tablet_bootstrap.cc:654] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.752633 19620 tablet_bootstrap.cc:654] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.758384 19622 raft_consensus.cc:493] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.759246 19622 raft_consensus.cc:515] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.766476 19616 tablet_bootstrap.cc:492] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.767161 19616 ts_tablet_manager.cc:1403] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 12:31:02.769811 19616 raft_consensus.cc:359] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.766476 19620 tablet_bootstrap.cc:492] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:02.770393 19616 raft_consensus.cc:385] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.771060 19616 raft_consensus.cc:740] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.772477 19616 consensus_queue.cc:260] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.773131 19622 leader_election.cc:290] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.771328 19620 ts_tablet_manager.cc:1403] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.012s
I20260709 12:31:02.777261 19620 raft_consensus.cc:359] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.778059 19620 raft_consensus.cc:385] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.778367 19620 raft_consensus.cc:740] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.779189 19620 consensus_queue.cc:260] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.780442 19615 tablet_bootstrap.cc:492] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:02.781070 19615 ts_tablet_manager.cc:1403] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.041s	user 0.022s	sys 0.014s
I20260709 12:31:02.781605 19620 ts_tablet_manager.cc:1434] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:31:02.784641 19620 tablet_bootstrap.cc:492] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:02.784857 19615 raft_consensus.cc:359] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.785806 19615 raft_consensus.cc:385] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.786180 19615 raft_consensus.cc:740] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.787132 19615 consensus_queue.cc:260] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.789752 19615 ts_tablet_manager.cc:1434] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:31:02.791277 19615 tablet_bootstrap.cc:492] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:02.793510 19620 tablet_bootstrap.cc:654] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.800174 19618 raft_consensus.cc:493] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.800794 19618 raft_consensus.cc:515] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.801754 19626 raft_consensus.cc:493] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.802591 19626 raft_consensus.cc:515] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.806517 19626 leader_election.cc:290] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:02.809142 19616 ts_tablet_manager.cc:1434] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.042s	user 0.011s	sys 0.001s
I20260709 12:31:02.810968 19616 tablet_bootstrap.cc:492] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:02.817246 19618 leader_election.cc:290] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.827251 19615 tablet_bootstrap.cc:654] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.838369 19620 tablet_bootstrap.cc:492] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:02.839036 19616 tablet_bootstrap.cc:654] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.839046 19620 ts_tablet_manager.cc:1403] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.036s
I20260709 12:31:02.850627 19622 raft_consensus.cc:493] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.850935 19636 raft_consensus.cc:493] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.851321 19622 raft_consensus.cc:515] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.851686 19636 raft_consensus.cc:515] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.853740 19622 leader_election.cc:290] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.855990 19615 tablet_bootstrap.cc:492] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:02.857396 19620 raft_consensus.cc:359] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.858238 19620 raft_consensus.cc:385] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.858551 19620 raft_consensus.cc:740] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.859434 19620 consensus_queue.cc:260] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.862077 19620 ts_tablet_manager.cc:1434] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.000s
I20260709 12:31:02.854187 19636 leader_election.cc:290] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.862278 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.865010 19425 raft_consensus.cc:2468] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:02.866429 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9372849a880b49e49e1753cc6f57c87e" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:02.867254 19574 raft_consensus.cc:2468] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 0.
I20260709 12:31:02.867262 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.868829 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c642d6fa088414e84482a66916fe5cf" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.870136 19464 leader_election.cc:304] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
W20260709 12:31:02.871692 19464 leader_election.cc:343] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:02.872691 19620 tablet_bootstrap.cc:492] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
W20260709 12:31:02.872648 19464 leader_election.cc:343] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:02.870645 19615 ts_tablet_manager.cc:1403] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.080s	user 0.009s	sys 0.010s
I20260709 12:31:02.877053 19389 leader_election.cc:304] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:02.878022 19618 raft_consensus.cc:2804] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:02.878401 19618 raft_consensus.cc:493] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:02.878749 19618 raft_consensus.cc:3060] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.875831 19636 raft_consensus.cc:2804] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:02.882555 19636 raft_consensus.cc:493] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:02.883157 19636 raft_consensus.cc:3060] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.883711 19622 raft_consensus.cc:493] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.884262 19622 raft_consensus.cc:515] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.886274 19622 leader_election.cc:290] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.890070 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:02.890461 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9372849a880b49e49e1753cc6f57c87e" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.891007 19573 raft_consensus.cc:2468] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:02.891472 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.891662 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:02.892190 19425 raft_consensus.cc:2468] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:02.893119 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c642d6fa088414e84482a66916fe5cf" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:02.893893 19573 raft_consensus.cc:2468] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:02.893986 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:02.896145 19618 raft_consensus.cc:515] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.896571 19464 leader_election.cc:304] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:02.897475 19622 raft_consensus.cc:2804] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:02.897701 19424 raft_consensus.cc:2393] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in current term 1: Already voted for candidate d55397b85c414ed0ad7f5481cd5a6c87 in this term.
I20260709 12:31:02.897894 19622 raft_consensus.cc:493] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:02.898505 19622 raft_consensus.cc:3060] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.901860 19620 tablet_bootstrap.cc:654] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.907897 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9372849a880b49e49e1753cc6f57c87e" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:02.881659 19615 raft_consensus.cc:359] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.909162 19500 raft_consensus.cc:2468] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 0.
I20260709 12:31:02.909266 19615 raft_consensus.cc:385] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.909741 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9372849a880b49e49e1753cc6f57c87e" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:02.902909 19464 leader_election.cc:304] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:02.910609 19499 raft_consensus.cc:2468] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:02.904382 19616 tablet_bootstrap.cc:492] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:02.912165 19616 ts_tablet_manager.cc:1403] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.102s	user 0.037s	sys 0.004s
I20260709 12:31:02.914085 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9372849a880b49e49e1753cc6f57c87e" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:02.914804 19574 raft_consensus.cc:3060] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.916018 19616 raft_consensus.cc:359] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.916857 19616 raft_consensus.cc:385] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.917183 19616 raft_consensus.cc:740] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.919566 19537 leader_election.cc:304] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:02.918076 19616 consensus_queue.cc:260] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.922670 19618 leader_election.cc:290] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.910980 19615 raft_consensus.cc:740] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.927211 19615 consensus_queue.cc:260] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.928517 19620 tablet_bootstrap.cc:492] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:02.929149 19620 ts_tablet_manager.cc:1403] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.017s
I20260709 12:31:02.929545 19574 raft_consensus.cc:2468] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:02.930418 19626 raft_consensus.cc:2764] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:31:02.932677 19620 raft_consensus.cc:359] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.933583 19620 raft_consensus.cc:385] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:02.933992 19620 raft_consensus.cc:740] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:02.929637 19615 ts_tablet_manager.cc:1434] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 12:31:02.935302 19620 consensus_queue.cc:260] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.937212 19389 leader_election.cc:304] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:02.938373 19620 ts_tablet_manager.cc:1434] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:31:02.943639 19649 raft_consensus.cc:2804] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:02.907950 19622 raft_consensus.cc:515] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.945930 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9372849a880b49e49e1753cc6f57c87e" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:02.954186 19620 tablet_bootstrap.cc:492] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:02.947113 19649 raft_consensus.cc:493] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:02.951437 19499 raft_consensus.cc:3060] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.959369 19649 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.959515 19618 raft_consensus.cc:2804] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:02.960179 19618 raft_consensus.cc:697] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:02.961181 19618 consensus_queue.cc:237] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.962358 19626 raft_consensus.cc:493] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.963325 19650 raft_consensus.cc:493] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.963052 19626 raft_consensus.cc:515] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.963867 19650 raft_consensus.cc:515] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.965874 19626 leader_election.cc:290] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:02.966220 19650 leader_election.cc:290] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.947245 19615 tablet_bootstrap.cc:492] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
W20260709 12:31:02.957043 19464 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.4.67:44555, user_credentials={real_user=slave}} blocked reactor thread for 54237.7us
W20260709 12:31:02.971125 19464 leader_election.cc:343] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:02.972178 19464 leader_election.cc:304] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:02.973021 19620 tablet_bootstrap.cc:654] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
W20260709 12:31:02.973474 19464 leader_election.cc:343] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:02.973910 19626 raft_consensus.cc:493] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:02.948644 19616 ts_tablet_manager.cc:1434] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.018s
I20260709 12:31:02.956801 19636 raft_consensus.cc:515] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.952827 19622 leader_election.cc:290] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:02.980787 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.974555 19626 raft_consensus.cc:515] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.969221 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:02.978010 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:02.976013 19649 raft_consensus.cc:515] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:02.980893 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:02.982323 19424 raft_consensus.cc:2468] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:02.983052 19425 raft_consensus.cc:3060] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.982173 19650 raft_consensus.cc:2804] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:02.983212 19573 raft_consensus.cc:2468] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:02.983901 19650 raft_consensus.cc:493] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:02.984303 19650 raft_consensus.cc:3060] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:02.984618 19538 leader_election.cc:304] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:02.982199 19615 tablet_bootstrap.cc:654] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:02.978945 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:02.996462 19499 raft_consensus.cc:2468] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:02.985462 19626 leader_election.cc:290] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:02.999073 19626 raft_consensus.cc:2804] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:02.999639 19626 raft_consensus.cc:493] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:03.000159 19626 raft_consensus.cc:3060] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.008139 19636 leader_election.cc:290] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.009939 19425 raft_consensus.cc:2468] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
W20260709 12:31:02.993281 19464 leader_election.cc:343] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:03.018131 19464 leader_election.cc:304] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.018308 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.019522 19425 raft_consensus.cc:3060] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.020004 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.020681 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.021595 19500 raft_consensus.cc:2393] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.022760 19636 raft_consensus.cc:2804] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.023396 19636 raft_consensus.cc:697] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
W20260709 12:31:03.023969 19537 leader_election.cc:343] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:03.024345 19636 consensus_queue.cc:237] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.027390 19649 leader_election.cc:290] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.028195 19616 tablet_bootstrap.cc:492] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:03.032696 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
W20260709 12:31:03.034704 19464 leader_election.cc:343] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:03.030550 19425 raft_consensus.cc:2468] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:03.042840 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:03.043371 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:03.043622 19574 raft_consensus.cc:3060] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.044273 19424 raft_consensus.cc:2393] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.039213 19464 leader_election.cc:304] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.046813 19649 raft_consensus.cc:2804] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:03.047209 19649 raft_consensus.cc:493] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:03.047569 19649 raft_consensus.cc:3060] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.047657 19320 catalog_manager.cc:5697] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.050549 19616 tablet_bootstrap.cc:654] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.052507 19464 leader_election.cc:304] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.053558 19652 raft_consensus.cc:2804] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.054765 19652 raft_consensus.cc:697] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:03.055873 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:03.055703 19652 consensus_queue.cc:237] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.056551 19573 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.062103 19626 raft_consensus.cc:515] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.065661 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:03.065429 19650 raft_consensus.cc:515] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.066722 19573 raft_consensus.cc:2468] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
W20260709 12:31:03.067426 19537 leader_election.cc:343] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:31:03.053561 19464 leader_election.cc:343] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:03.068223 19538 leader_election.cc:304] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.068523 19650 leader_election.cc:290] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.070708 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c642d6fa088414e84482a66916fe5cf" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.070746 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.071682 19425 raft_consensus.cc:3060] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.072059 19620 tablet_bootstrap.cc:492] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:03.072593 19620 ts_tablet_manager.cc:1403] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.119s	user 0.031s	sys 0.020s
W20260709 12:31:03.072911 19464 leader_election.cc:343] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:03.073395 19574 raft_consensus.cc:2468] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:03.073717 19464 leader_election.cc:304] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.074124 19654 raft_consensus.cc:2749] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.071009 19615 tablet_bootstrap.cc:492] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:03.076717 19626 leader_election.cc:290] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:03.082165 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c642d6fa088414e84482a66916fe5cf" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:03.082417 19620 raft_consensus.cc:359] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.082877 19573 raft_consensus.cc:3060] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.077963 19615 ts_tablet_manager.cc:1403] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.137s	user 0.015s	sys 0.003s
I20260709 12:31:03.076956 19650 raft_consensus.cc:2804] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.083293 19620 raft_consensus.cc:385] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.088187 19620 raft_consensus.cc:740] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.090175 19620 consensus_queue.cc:260] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.092744 19620 ts_tablet_manager.cc:1434] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:31:03.096263 19620 tablet_bootstrap.cc:492] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:03.096282 19323 catalog_manager.cc:5697] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.087136 19425 raft_consensus.cc:2468] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 1.
I20260709 12:31:03.098718 19538 leader_election.cc:304] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:03.100865 19626 raft_consensus.cc:2804] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.101369 19626 raft_consensus.cc:697] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:03.102272 19626 consensus_queue.cc:237] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.093571 19650 raft_consensus.cc:697] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:03.107122 19620 tablet_bootstrap.cc:654] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.110448 19650 consensus_queue.cc:237] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.118637 19649 raft_consensus.cc:515] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.123586 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.110015 19615 raft_consensus.cc:359] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
W20260709 12:31:03.126683 19464 leader_election.cc:343] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:03.126026 19649 leader_election.cc:290] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.133639 19616 tablet_bootstrap.cc:492] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:03.128666 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:03.137156 19616 ts_tablet_manager.cc:1403] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.131s	user 0.007s	sys 0.015s
I20260709 12:31:03.137454 19574 raft_consensus.cc:3060] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.140110 19616 raft_consensus.cc:359] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.140780 19616 raft_consensus.cc:385] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.141067 19616 raft_consensus.cc:740] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.127905 19615 raft_consensus.cc:385] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.142031 19615 raft_consensus.cc:740] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.143303 19615 consensus_queue.cc:260] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.141918 19616 consensus_queue.cc:260] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.148546 19616 ts_tablet_manager.cc:1434] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:31:03.149624 19616 tablet_bootstrap.cc:492] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:03.150090 19574 raft_consensus.cc:2468] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:03.135339 19573 raft_consensus.cc:2468] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:03.152611 19464 leader_election.cc:304] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.153947 19649 raft_consensus.cc:2804] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.154428 19649 raft_consensus.cc:697] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:03.155673 19616 tablet_bootstrap.cc:654] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.155229 19649 consensus_queue.cc:237] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.161492 19620 tablet_bootstrap.cc:492] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:03.162158 19620 ts_tablet_manager.cc:1403] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.066s	user 0.012s	sys 0.005s
I20260709 12:31:03.166348 19620 raft_consensus.cc:359] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.167435 19620 raft_consensus.cc:385] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.167802 19620 raft_consensus.cc:740] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.168758 19620 consensus_queue.cc:260] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.171476 19620 ts_tablet_manager.cc:1434] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:31:03.172806 19620 tablet_bootstrap.cc:492] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:03.146356 19615 ts_tablet_manager.cc:1434] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.001s
I20260709 12:31:03.174959 19626 raft_consensus.cc:493] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.176025 19615 tablet_bootstrap.cc:492] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:03.175873 19626 raft_consensus.cc:515] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.180944 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:03.182051 19425 raft_consensus.cc:2468] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:03.183008 19464 leader_election.cc:304] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.186775 19636 raft_consensus.cc:2804] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.188086 19636 raft_consensus.cc:697] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:03.188557 19538 leader_election.cc:304] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.189751 19654 raft_consensus.cc:2804] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:03.196715 19321 catalog_manager.cc:5697] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:03.198237 19615 tablet_bootstrap.cc:654] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.192147 19654 raft_consensus.cc:493] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:03.199039 19654 raft_consensus.cc:3060] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.199134 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.190099 19649 raft_consensus.cc:493] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.189298 19636 consensus_queue.cc:237] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.201961 19649 raft_consensus.cc:515] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.203469 19500 raft_consensus.cc:2468] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:03.205245 19649 leader_election.cc:290] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.207242 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:03.207990 19425 raft_consensus.cc:2468] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:03.209332 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:03.210386 19620 tablet_bootstrap.cc:654] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.215497 19464 leader_election.cc:304] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.215811 19616 tablet_bootstrap.cc:492] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:03.188040 19626 leader_election.cc:290] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:03.222671 19649 raft_consensus.cc:2804] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:03.223230 19649 raft_consensus.cc:493] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:03.223680 19649 raft_consensus.cc:3060] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.228372 19615 tablet_bootstrap.cc:492] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:03.216636 19616 ts_tablet_manager.cc:1403] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.036s
I20260709 12:31:03.231509 19654 raft_consensus.cc:515] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.232421 19615 ts_tablet_manager.cc:1403] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.057s	user 0.010s	sys 0.007s
I20260709 12:31:03.232769 19574 raft_consensus.cc:2393] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 051c8a7691c2489bb513f0e150a6089b in current term 1: Already voted for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in this term.
I20260709 12:31:03.234750 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.235407 19425 raft_consensus.cc:3060] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.242782 19425 raft_consensus.cc:2468] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 1.
I20260709 12:31:03.242327 19615 raft_consensus.cc:359] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.243582 19615 raft_consensus.cc:385] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.243009 19616 raft_consensus.cc:359] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.244012 19615 raft_consensus.cc:740] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.244192 19538 leader_election.cc:304] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.245545 19616 raft_consensus.cc:385] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.245922 19616 raft_consensus.cc:740] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.245056 19654 leader_election.cc:290] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:03.246381 19626 raft_consensus.cc:2804] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.246996 19626 raft_consensus.cc:697] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:03.246780 19616 consensus_queue.cc:260] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.248725 19616 ts_tablet_manager.cc:1434] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:31:03.248536 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:03.247915 19626 consensus_queue.cc:237] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.251607 19649 raft_consensus.cc:515] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.252113 19620 tablet_bootstrap.cc:492] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:03.253070 19500 raft_consensus.cc:2393] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.254424 19620 ts_tablet_manager.cc:1403] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.082s	user 0.028s	sys 0.013s
I20260709 12:31:03.248438 19615 consensus_queue.cc:260] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.256692 19615 ts_tablet_manager.cc:1434] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.002s
I20260709 12:31:03.257887 19615 tablet_bootstrap.cc:492] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:03.257939 19620 raft_consensus.cc:359] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.258883 19620 raft_consensus.cc:385] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.259174 19620 raft_consensus.cc:740] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.259948 19620 consensus_queue.cc:260] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.261466 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:03.262205 19620 ts_tablet_manager.cc:1434] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:31:03.253871 19649 leader_election.cc:290] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.264045 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.264901 19425 raft_consensus.cc:2393] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 051c8a7691c2489bb513f0e150a6089b in current term 1: Already voted for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in this term.
I20260709 12:31:03.266031 19649 raft_consensus.cc:493] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.266615 19649 raft_consensus.cc:515] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.267216 19615 tablet_bootstrap.cc:654] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.269004 19649 leader_election.cc:290] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.269697 19464 leader_election.cc:304] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.272428 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:03.273370 19425 raft_consensus.cc:2393] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 051c8a7691c2489bb513f0e150a6089b in current term 1: Already voted for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in this term.
I20260709 12:31:03.274893 19649 raft_consensus.cc:2749] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.275355 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:03.277112 19464 leader_election.cc:304] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.278126 19649 raft_consensus.cc:3060] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.279347 19615 tablet_bootstrap.cc:492] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:03.279906 19615 ts_tablet_manager.cc:1403] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.014s
I20260709 12:31:03.275355 19323 catalog_manager.cc:5697] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:03.282917 19615 raft_consensus.cc:359] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.283656 19615 raft_consensus.cc:385] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.283968 19615 raft_consensus.cc:740] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.284751 19615 consensus_queue.cc:260] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.285367 19649 raft_consensus.cc:2749] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.286700 19615 ts_tablet_manager.cc:1434] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:31:03.287487 19615 tablet_bootstrap.cc:492] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:03.289521 19321 catalog_manager.cc:5697] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.291350 19321 catalog_manager.cc:5697] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.292506 19321 catalog_manager.cc:5697] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.293283 19615 tablet_bootstrap.cc:654] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:03.293888 19321 catalog_manager.cc:5697] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.298662 19615 tablet_bootstrap.cc:492] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:03.299224 19615 ts_tablet_manager.cc:1403] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:31:03.301944 19615 raft_consensus.cc:359] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.302426 19615 raft_consensus.cc:385] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:03.302690 19615 raft_consensus.cc:740] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:03.303457 19615 consensus_queue.cc:260] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.305668 19615 ts_tablet_manager.cc:1434] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:31:03.365513 19618 raft_consensus.cc:493] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.366117 19618 raft_consensus.cc:515] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.368994 19618 leader_election.cc:290] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.369454 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.369593 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:03.370453 19574 raft_consensus.cc:2393] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d55397b85c414ed0ad7f5481cd5a6c87 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.371516 19389 leader_election.cc:304] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:03.372350 19618 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.378742 19618 raft_consensus.cc:2749] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.424639 19618 consensus_queue.cc:1048] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.431983 19626 raft_consensus.cc:493] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.432735 19626 raft_consensus.cc:515] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.437659 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:03.438596 19425 raft_consensus.cc:2393] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.439163 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.439657 19649 raft_consensus.cc:493] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.440270 19649 raft_consensus.cc:515] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.441691 19626 leader_election.cc:290] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:03.446406 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b0aeda8cdb248f2833025fda78b4f9c" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:03.446537 19618 raft_consensus.cc:493] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.447080 19538 leader_election.cc:304] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.447280 19425 raft_consensus.cc:2468] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:03.447384 19618 raft_consensus.cc:515] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.448709 19464 leader_election.cc:304] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.449281 19626 raft_consensus.cc:3060] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.449678 19636 raft_consensus.cc:2804] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:03.450150 19636 raft_consensus.cc:493] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:03.450476 19636 raft_consensus.cc:3060] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.451395 19618 leader_election.cc:290] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.448400 19651 consensus_queue.cc:1048] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:31:03.453239 19649 leader_election.cc:290] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.452373 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c642d6fa088414e84482a66916fe5cf" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.455047 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b0aeda8cdb248f2833025fda78b4f9c" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:03.455685 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c642d6fa088414e84482a66916fe5cf" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:03.455751 19574 raft_consensus.cc:2468] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:03.456847 19573 raft_consensus.cc:2393] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d55397b85c414ed0ad7f5481cd5a6c87 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.458024 19636 raft_consensus.cc:515] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.459002 19389 leader_election.cc:304] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:03.460304 19651 raft_consensus.cc:3060] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.460664 19636 leader_election.cc:290] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.461398 19626 raft_consensus.cc:2749] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.462637 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b0aeda8cdb248f2833025fda78b4f9c" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:03.463171 19574 raft_consensus.cc:3060] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.466841 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b0aeda8cdb248f2833025fda78b4f9c" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.467568 19425 raft_consensus.cc:3060] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.469822 19651 raft_consensus.cc:2749] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.470865 19574 raft_consensus.cc:2468] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:03.472133 19464 leader_election.cc:304] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:03.473075 19636 raft_consensus.cc:2804] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:03.473570 19636 raft_consensus.cc:697] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:03.474455 19636 consensus_queue.cc:237] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.477182 19425 raft_consensus.cc:2468] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:03.484174 19321 catalog_manager.cc:5697] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:03.503659 19649 consensus_queue.cc:1048] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.504175 19636 consensus_queue.cc:1048] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.520326 19636 consensus_queue.cc:1048] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.535624 19683 consensus_queue.cc:1048] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.536321 19626 consensus_queue.cc:1048] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.557462 19626 consensus_queue.cc:1048] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.571137 19683 consensus_queue.cc:1048] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.584841 19683 consensus_queue.cc:1048] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.614531 19651 raft_consensus.cc:493] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:03.615159 19651 raft_consensus.cc:515] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.615710 19683 consensus_queue.cc:1048] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.617527 19651 leader_election.cc:290] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:03.618225 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:03.618690 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:03.619695 19574 raft_consensus.cc:2393] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d55397b85c414ed0ad7f5481cd5a6c87 in current term 1: Already voted for candidate 051c8a7691c2489bb513f0e150a6089b in this term.
I20260709 12:31:03.621344 19389 leader_election.cc:304] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:03.622257 19651 raft_consensus.cc:3060] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:03.630266 19651 raft_consensus.cc:2749] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:03.631621 19683 consensus_queue.cc:1048] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.673800 19687 consensus_queue.cc:1048] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.688653 19694 consensus_queue.cc:1048] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.726063 19500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:58136
I20260709 12:31:03.726938 19500 raft_consensus.cc:606] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.744882 19574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:41672
I20260709 12:31:03.745467 19574 raft_consensus.cc:606] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.751762 19500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:58136
I20260709 12:31:03.752808 19500 raft_consensus.cc:606] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:03.764691 19574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
mode: GRACEFUL
new_leader_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:41672
I20260709 12:31:03.765309 19574 raft_consensus.cc:606] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Received request to transfer leadership to TS 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:03.773928 19500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:58136
I20260709 12:31:03.774085 19683 consensus_queue.cc:1048] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.774639 19500 raft_consensus.cc:606] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Received request to transfer leadership to TS a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:03.781641 19649 raft_consensus.cc:993] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:03.782217 19636 raft_consensus.cc:1081] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Signalling peer a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:03.785152 19573 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:41656
I20260709 12:31:03.785907 19573 raft_consensus.cc:493] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:03.785915 19649 consensus_queue.cc:1048] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:03.786427 19573 raft_consensus.cc:3060] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:03.795408 19573 raft_consensus.cc:515] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.796551 19574 raft_consensus.cc:1240] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Rejecting Update request from peer 051c8a7691c2489bb513f0e150a6089b for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:03.798331 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:03.799007 19424 raft_consensus.cc:3060] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:03.801357 19649 consensus_queue.cc:1059] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:03.802162 19636 raft_consensus.cc:3055] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:03.802546 19636 raft_consensus.cc:740] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:03.803213 19573 leader_election.cc:290] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:03.803476 19636 consensus_queue.cc:260] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.804821 19636 raft_consensus.cc:3060] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:03.806488 19424 raft_consensus.cc:2468] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 2.
I20260709 12:31:03.808032 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b2f618923fa4bb9bde5f0acd9a47fb2" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:03.808204 19538 leader_election.cc:304] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:03.809046 19694 raft_consensus.cc:2804] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:03.809530 19694 raft_consensus.cc:697] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:03.810336 19694 consensus_queue.cc:237] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:03.817361 19323 catalog_manager.cc:5697] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 1 to 2, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:03.820719 19500 raft_consensus.cc:2468] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 2.
I20260709 12:31:04.020117 19636 consensus_queue.cc:1048] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:04.034452 19650 raft_consensus.cc:993] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:04.034660 19652 consensus_queue.cc:1048] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:04.035226 19683 raft_consensus.cc:1081] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:04.037395 19423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37120
I20260709 12:31:04.038106 19423 raft_consensus.cc:493] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:04.038616 19423 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.048897 19423 raft_consensus.cc:515] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.051656 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:04.051771 19423 leader_election.cc:290] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:04.051965 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:04.052417 19500 raft_consensus.cc:3055] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:04.052877 19500 raft_consensus.cc:740] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:04.052896 19573 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.054041 19500 consensus_queue.cc:260] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.055572 19500 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.059499 19573 raft_consensus.cc:2468] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:04.060724 19389 leader_election.cc:304] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:04.061551 19701 raft_consensus.cc:2804] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:04.062132 19701 raft_consensus.cc:697] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:04.062861 19500 raft_consensus.cc:2468] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:04.063180 19701 consensus_queue.cc:237] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.070425 19323 catalog_manager.cc:5697] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:04.094776 19652 raft_consensus.cc:993] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:04.095510 19649 raft_consensus.cc:1081] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:04.097421 19423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37120
I20260709 12:31:04.098001 19423 raft_consensus.cc:493] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:04.098392 19423 raft_consensus.cc:3060] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.104499 19423 raft_consensus.cc:515] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.106169 19423 leader_election.cc:290] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:04.107260 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:04.107313 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "013fb25300cb44e6ac3d9a1ece427992" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:04.108392 19500 raft_consensus.cc:3055] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:04.108613 19573 raft_consensus.cc:3060] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.108959 19500 raft_consensus.cc:740] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:04.110077 19500 consensus_queue.cc:260] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.111373 19500 raft_consensus.cc:3060] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.113986 19694 raft_consensus.cc:993] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: : Instructing follower 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:04.114620 19687 raft_consensus.cc:1081] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Signalling peer 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:04.116636 19499 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:58120
I20260709 12:31:04.116978 19573 raft_consensus.cc:2468] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:04.117213 19499 raft_consensus.cc:493] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:04.117745 19499 raft_consensus.cc:3060] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.118201 19389 leader_election.cc:304] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:04.118651 19500 raft_consensus.cc:2468] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:04.119120 19701 raft_consensus.cc:2804] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:04.119675 19701 raft_consensus.cc:697] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:04.120611 19701 consensus_queue.cc:237] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.124680 19499 raft_consensus.cc:515] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.126904 19499 leader_election.cc:290] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:04.128189 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:04.128706 19423 raft_consensus.cc:3060] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.128717 19322 catalog_manager.cc:5697] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:04.130940 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f21d676c01f48bab23326a671a9c6b1" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:04.131549 19573 raft_consensus.cc:3055] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:04.131840 19573 raft_consensus.cc:740] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:04.132741 19573 consensus_queue.cc:260] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.134079 19573 raft_consensus.cc:3060] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.136021 19423 raft_consensus.cc:2468] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:04.137463 19464 leader_election.cc:304] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:04.138227 19649 raft_consensus.cc:2804] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:04.138772 19649 raft_consensus.cc:697] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:04.139642 19649 consensus_queue.cc:237] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.141284 19573 raft_consensus.cc:2468] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:04.147130 19322 catalog_manager.cc:5697] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 1 to 2, leader changed from a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67) to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 2 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:04.161720 19694 raft_consensus.cc:993] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:04.162312 19687 raft_consensus.cc:1081] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:04.164126 19423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37126
I20260709 12:31:04.164753 19423 raft_consensus.cc:493] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:04.165148 19423 raft_consensus.cc:3060] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.170478 19423 raft_consensus.cc:515] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.172180 19423 leader_election.cc:290] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:04.173180 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:04.173421 19573 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df2d8c357c44bc08e7478ac539f508a" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:04.173846 19499 raft_consensus.cc:3060] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.174288 19573 raft_consensus.cc:3055] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:04.174703 19573 raft_consensus.cc:740] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:04.175534 19573 consensus_queue.cc:260] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.176697 19573 raft_consensus.cc:3060] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:04.179183 19499 raft_consensus.cc:2468] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:04.180446 19388 leader_election.cc:304] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:04.181192 19701 raft_consensus.cc:2804] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:04.181664 19701 raft_consensus.cc:697] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:04.182526 19701 consensus_queue.cc:237] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:04.183782 19573 raft_consensus.cc:2468] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:04.189880 19322 catalog_manager.cc:5697] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:04.205848 19423 raft_consensus.cc:1275] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.207154 19687 consensus_queue.cc:1048] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.215975 19499 raft_consensus.cc:1275] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.217938 19694 consensus_queue.cc:1048] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.445209 19573 raft_consensus.cc:1275] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.446710 19701 consensus_queue.cc:1048] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.455468 19499 raft_consensus.cc:1275] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.457424 19701 consensus_queue.cc:1048] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.546797 19499 raft_consensus.cc:1275] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.548187 19701 consensus_queue.cc:1048] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.557488 19573 raft_consensus.cc:1275] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.559759 19618 consensus_queue.cc:1048] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.570674 19572 raft_consensus.cc:1275] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.572048 19649 consensus_queue.cc:1048] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.580600 19423 raft_consensus.cc:1275] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.582805 19652 consensus_queue.cc:1048] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.668540 19572 raft_consensus.cc:1275] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.669950 19618 consensus_queue.cc:1048] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:04.679740 19499 raft_consensus.cc:1275] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:04.682032 19701 consensus_queue.cc:1048] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:06.806095 19323 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35510:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:06.808671 19323 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:06.863729 19411 tablet_service.cc:1511] Processing CreateTablet for tablet 959339bed78841e5b523d05b7cc25273 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:06.865015 19411 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 959339bed78841e5b523d05b7cc25273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.865162 19488 tablet_service.cc:1511] Processing CreateTablet for tablet 959339bed78841e5b523d05b7cc25273 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:06.866389 19488 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 959339bed78841e5b523d05b7cc25273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.869866 19484 tablet_service.cc:1511] Processing CreateTablet for tablet 06b85cf2ec024e79bdb45a772107022b (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:06.870528 19557 tablet_service.cc:1511] Processing CreateTablet for tablet 959339bed78841e5b523d05b7cc25273 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:06.871433 19484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b85cf2ec024e79bdb45a772107022b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.871855 19557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 959339bed78841e5b523d05b7cc25273. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.873893 19558 tablet_service.cc:1511] Processing CreateTablet for tablet 06b85cf2ec024e79bdb45a772107022b (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:06.875106 19558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b85cf2ec024e79bdb45a772107022b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.876494 19415 tablet_service.cc:1511] Processing CreateTablet for tablet 06b85cf2ec024e79bdb45a772107022b (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:06.877897 19415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06b85cf2ec024e79bdb45a772107022b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.877635 19483 tablet_service.cc:1511] Processing CreateTablet for tablet 0a52e3c0cab0418f8795b7514ba56c87 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:06.878990 19483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a52e3c0cab0418f8795b7514ba56c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.879580 19407 tablet_service.cc:1511] Processing CreateTablet for tablet 0a52e3c0cab0418f8795b7514ba56c87 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:06.880944 19407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a52e3c0cab0418f8795b7514ba56c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.881743 19559 tablet_service.cc:1511] Processing CreateTablet for tablet 0a52e3c0cab0418f8795b7514ba56c87 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:06.883117 19559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a52e3c0cab0418f8795b7514ba56c87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.884667 19485 tablet_service.cc:1511] Processing CreateTablet for tablet 21d4bb10d9ce456685da713f02091158 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:06.885946 19485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d4bb10d9ce456685da713f02091158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.887570 19560 tablet_service.cc:1511] Processing CreateTablet for tablet 21d4bb10d9ce456685da713f02091158 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:06.888639 19408 tablet_service.cc:1511] Processing CreateTablet for tablet 21d4bb10d9ce456685da713f02091158 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:06.889300 19560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d4bb10d9ce456685da713f02091158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.890072 19408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d4bb10d9ce456685da713f02091158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.892757 19487 tablet_service.cc:1511] Processing CreateTablet for tablet a5d7d0da151049a08e785e94683b05b3 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:06.894136 19487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d7d0da151049a08e785e94683b05b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.896003 19413 tablet_service.cc:1511] Processing CreateTablet for tablet a5d7d0da151049a08e785e94683b05b3 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:06.897348 19413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d7d0da151049a08e785e94683b05b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.911154 19561 tablet_service.cc:1511] Processing CreateTablet for tablet a5d7d0da151049a08e785e94683b05b3 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:06.911656 19409 tablet_service.cc:1511] Processing CreateTablet for tablet a3720e40313144c897120c99a6308819 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:06.912623 19561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a5d7d0da151049a08e785e94683b05b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.913221 19409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3720e40313144c897120c99a6308819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.915907 19486 tablet_service.cc:1511] Processing CreateTablet for tablet a3720e40313144c897120c99a6308819 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:06.917698 19486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3720e40313144c897120c99a6308819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.921060 19726 tablet_bootstrap.cc:492] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:06.923633 19564 tablet_service.cc:1511] Processing CreateTablet for tablet a3720e40313144c897120c99a6308819 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:06.925120 19564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a3720e40313144c897120c99a6308819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.928273 19490 tablet_service.cc:1511] Processing CreateTablet for tablet ed20fe260b974e08b2aab4294797a6fd (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:06.929571 19726 tablet_bootstrap.cc:654] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:06.929750 19490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed20fe260b974e08b2aab4294797a6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.932229 19413 tablet_service.cc:1511] Processing CreateTablet for tablet ed20fe260b974e08b2aab4294797a6fd (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:06.934077 19413 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed20fe260b974e08b2aab4294797a6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.941493 19556 tablet_service.cc:1511] Processing CreateTablet for tablet ed20fe260b974e08b2aab4294797a6fd (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:06.943133 19556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed20fe260b974e08b2aab4294797a6fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.958395 19726 tablet_bootstrap.cc:492] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:06.959015 19726 ts_tablet_manager.cc:1403] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.038s	user 0.005s	sys 0.007s
I20260709 12:31:06.960906 19414 tablet_service.cc:1511] Processing CreateTablet for tablet efd53179cce647948755bc8fc079cbba (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:06.962230 19414 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efd53179cce647948755bc8fc079cbba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.967868 19489 tablet_service.cc:1511] Processing CreateTablet for tablet efd53179cce647948755bc8fc079cbba (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:06.969514 19489 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efd53179cce647948755bc8fc079cbba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.991374 19562 tablet_service.cc:1511] Processing CreateTablet for tablet efd53179cce647948755bc8fc079cbba (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:06.992794 19562 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efd53179cce647948755bc8fc079cbba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.995710 19487 tablet_service.cc:1511] Processing CreateTablet for tablet 295ce06a4c544374b969e439c410f2b3 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:06.997310 19487 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295ce06a4c544374b969e439c410f2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:07.000012 19412 tablet_service.cc:1511] Processing CreateTablet for tablet 295ce06a4c544374b969e439c410f2b3 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:07.001353 19412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295ce06a4c544374b969e439c410f2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:06.982348 19726 raft_consensus.cc:359] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.009054 19726 raft_consensus.cc:385] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.009389 19726 raft_consensus.cc:740] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.011240 19563 tablet_service.cc:1511] Processing CreateTablet for tablet 295ce06a4c544374b969e439c410f2b3 (DEFAULT_TABLE table=soft_deleted_table [id=86838154c66a450481c98de157ec201f]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:07.012599 19563 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 295ce06a4c544374b969e439c410f2b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:07.028013 19726 consensus_queue.cc:260] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.035864 19726 ts_tablet_manager.cc:1434] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.077s	user 0.005s	sys 0.003s
I20260709 12:31:07.043537 19726 tablet_bootstrap.cc:492] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.080794 19725 tablet_bootstrap.cc:492] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.085899 19726 tablet_bootstrap.cc:654] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.092427 19728 tablet_bootstrap.cc:492] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.096803 19726 tablet_bootstrap.cc:492] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.093667 19725 tablet_bootstrap.cc:654] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.097409 19726 ts_tablet_manager.cc:1403] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.002s
I20260709 12:31:07.099854 19728 tablet_bootstrap.cc:654] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.100674 19726 raft_consensus.cc:359] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.101613 19726 raft_consensus.cc:385] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.101989 19726 raft_consensus.cc:740] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.102742 19726 consensus_queue.cc:260] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.103879 19725 tablet_bootstrap.cc:492] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.105414 19726 ts_tablet_manager.cc:1434] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:31:07.106458 19728 tablet_bootstrap.cc:492] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.107118 19725 ts_tablet_manager.cc:1403] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.003s
I20260709 12:31:07.107420 19728 ts_tablet_manager.cc:1403] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:31:07.107940 19726 tablet_bootstrap.cc:492] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.110831 19725 raft_consensus.cc:359] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.111075 19728 raft_consensus.cc:359] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.111680 19725 raft_consensus.cc:385] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.111863 19728 raft_consensus.cc:385] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.112097 19725 raft_consensus.cc:740] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.112232 19728 raft_consensus.cc:740] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.115060 19728 consensus_queue.cc:260] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.115060 19725 consensus_queue.cc:260] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.117628 19728 ts_tablet_manager.cc:1434] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.003s
I20260709 12:31:07.118821 19728 tablet_bootstrap.cc:492] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.116080 19726 tablet_bootstrap.cc:654] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.125943 19728 tablet_bootstrap.cc:654] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.134182 19725 ts_tablet_manager.cc:1434] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:31:07.134332 19714 raft_consensus.cc:493] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.135038 19714 raft_consensus.cc:515] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.135491 19726 tablet_bootstrap.cc:492] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.135699 19725 tablet_bootstrap.cc:492] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.136087 19726 ts_tablet_manager.cc:1403] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.007s
I20260709 12:31:07.138339 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.139289 19726 raft_consensus.cc:359] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.140090 19726 raft_consensus.cc:385] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:31:07.140200 19464 leader_election.cc:343] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.140439 19726 raft_consensus.cc:740] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.141381 19726 consensus_queue.cc:260] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.144501 19728 tablet_bootstrap.cc:492] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.145040 19728 ts_tablet_manager.cc:1403] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.004s
I20260709 12:31:07.147214 19725 tablet_bootstrap.cc:654] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.148159 19728 raft_consensus.cc:359] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.149048 19728 raft_consensus.cc:385] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.149403 19728 raft_consensus.cc:740] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.150290 19728 consensus_queue.cc:260] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.152320 19726 ts_tablet_manager.cc:1434] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.003s
I20260709 12:31:07.152578 19728 ts_tablet_manager.cc:1434] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:31:07.153414 19726 tablet_bootstrap.cc:492] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.158346 19726 tablet_bootstrap.cc:654] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.159955 19714 leader_election.cc:290] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.161643 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.162159 19728 tablet_bootstrap.cc:492] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.163408 19725 tablet_bootstrap.cc:492] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.163520 19705 raft_consensus.cc:493] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.164130 19725 ts_tablet_manager.cc:1403] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.013s
I20260709 12:31:07.164075 19705 raft_consensus.cc:515] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.167783 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.167706 19725 raft_consensus.cc:359] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.168519 19725 raft_consensus.cc:385] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.168900 19725 raft_consensus.cc:740] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.169782 19725 consensus_queue.cc:260] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
W20260709 12:31:07.170265 19464 leader_election.cc:343] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.170789 19464 leader_election.cc:304] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.171166 19728 tablet_bootstrap.cc:654] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.171605 19714 raft_consensus.cc:2749] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.172633 19499 raft_consensus.cc:2468] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 0.
I20260709 12:31:07.175439 19725 ts_tablet_manager.cc:1434] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:31:07.175508 19388 leader_election.cc:304] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.176469 19713 raft_consensus.cc:2804] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.176851 19713 raft_consensus.cc:493] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.177188 19713 raft_consensus.cc:3060] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.169821 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
W20260709 12:31:07.178925 19389 leader_election.cc:343] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.176494 19705 leader_election.cc:290] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.186980 19728 tablet_bootstrap.cc:492] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.188158 19713 raft_consensus.cc:515] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.189384 19725 tablet_bootstrap.cc:492] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.192924 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.193660 19499 raft_consensus.cc:3060] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.195055 19728 ts_tablet_manager.cc:1403] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.007s
I20260709 12:31:07.198980 19728 raft_consensus.cc:359] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.199741 19728 raft_consensus.cc:385] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.200045 19728 raft_consensus.cc:740] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.200332 19726 tablet_bootstrap.cc:492] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.200935 19726 ts_tablet_manager.cc:1403] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.009s
I20260709 12:31:07.201026 19728 consensus_queue.cc:260] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.202597 19499 raft_consensus.cc:2468] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:07.203047 19713 leader_election.cc:290] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.204308 19388 leader_election.cc:304] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.204380 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:07.205354 19713 raft_consensus.cc:2804] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:31:07.206046 19389 leader_election.cc:343] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.206902 19713 raft_consensus.cc:697] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:07.208524 19725 tablet_bootstrap.cc:654] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.207974 19713 consensus_queue.cc:237] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.205801 19726 raft_consensus.cc:359] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.210973 19726 raft_consensus.cc:385] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.211400 19726 raft_consensus.cc:740] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.212328 19726 consensus_queue.cc:260] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.212687 19705 raft_consensus.cc:493] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.213337 19705 raft_consensus.cc:515] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.215608 19705 leader_election.cc:290] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.215752 19711 raft_consensus.cc:493] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.216336 19711 raft_consensus.cc:515] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.217734 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efd53179cce647948755bc8fc079cbba" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.218309 19726 ts_tablet_manager.cc:1434] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.003s
I20260709 12:31:07.218509 19572 raft_consensus.cc:2468] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 0.
I20260709 12:31:07.219460 19726 tablet_bootstrap.cc:492] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.219374 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efd53179cce647948755bc8fc079cbba" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.219960 19389 leader_election.cc:304] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.220916 19705 raft_consensus.cc:2804] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:31:07.221153 19388 leader_election.cc:343] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.221324 19705 raft_consensus.cc:493] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.221630 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
W20260709 12:31:07.223371 19537 leader_election.cc:343] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.224339 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.225132 19423 raft_consensus.cc:2468] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:07.221807 19705 raft_consensus.cc:3060] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.226374 19538 leader_election.cc:304] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:07.230605 19711 leader_election.cc:290] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:07.231377 19711 raft_consensus.cc:2804] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.231163 19728 ts_tablet_manager.cc:1434] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.036s	user 0.004s	sys 0.004s
I20260709 12:31:07.225606 19321 catalog_manager.cc:5697] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.231923 19711 raft_consensus.cc:493] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.233153 19711 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.234457 19728 tablet_bootstrap.cc:492] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.235685 19726 tablet_bootstrap.cc:654] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.249058 19726 tablet_bootstrap.cc:492] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.258648 19726 ts_tablet_manager.cc:1403] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.003s
I20260709 12:31:07.260571 19728 tablet_bootstrap.cc:654] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.261562 19726 raft_consensus.cc:359] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.262223 19726 raft_consensus.cc:385] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.262531 19726 raft_consensus.cc:740] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.246430 19711 raft_consensus.cc:515] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.263685 19726 consensus_queue.cc:260] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.265563 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:07.266240 19423 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.270412 19726 ts_tablet_manager.cc:1434] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:31:07.271633 19726 tablet_bootstrap.cc:492] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.274338 19423 raft_consensus.cc:2468] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 1.
I20260709 12:31:07.275882 19538 leader_election.cc:304] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.252076 19725 tablet_bootstrap.cc:492] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.276803 19739 raft_consensus.cc:2804] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.279731 19725 ts_tablet_manager.cc:1403] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.091s	user 0.016s	sys 0.010s
I20260709 12:31:07.279793 19739 raft_consensus.cc:697] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:07.280905 19739 consensus_queue.cc:237] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
W20260709 12:31:07.277416 19464 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:40457, user_credentials={real_user=slave}} blocked reactor thread for 62549us
I20260709 12:31:07.257881 19705 raft_consensus.cc:515] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.284014 19726 tablet_bootstrap.cc:654] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.284605 19705 leader_election.cc:290] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.285786 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efd53179cce647948755bc8fc079cbba" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:07.286444 19572 raft_consensus.cc:3060] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.287297 19711 leader_election.cc:290] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:07.287871 19728 tablet_bootstrap.cc:492] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.288445 19728 ts_tablet_manager.cc:1403] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.054s	user 0.012s	sys 0.006s
I20260709 12:31:07.289515 19725 raft_consensus.cc:359] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.290290 19725 raft_consensus.cc:385] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.290670 19725 raft_consensus.cc:740] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.291071 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efd53179cce647948755bc8fc079cbba" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.291527 19725 consensus_queue.cc:260] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.291810 19728 raft_consensus.cc:359] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.292542 19728 raft_consensus.cc:385] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.292903 19728 raft_consensus.cc:740] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.293974 19728 consensus_queue.cc:260] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.294596 19725 ts_tablet_manager.cc:1434] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.002s
I20260709 12:31:07.295379 19500 raft_consensus.cc:3060] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.295590 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.297178 19728 ts_tablet_manager.cc:1434] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:31:07.298311 19728 tablet_bootstrap.cc:492] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.302825 19705 raft_consensus.cc:493] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.303457 19705 raft_consensus.cc:515] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.312780 19728 tablet_bootstrap.cc:654] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.328330 19726 tablet_bootstrap.cc:492] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.312619 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.305639 19705 leader_election.cc:290] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.333753 19726 ts_tablet_manager.cc:1403] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.062s	user 0.014s	sys 0.024s
I20260709 12:31:07.323722 19725 tablet_bootstrap.cc:492] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
W20260709 12:31:07.334197 19388 leader_election.cc:343] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.313170 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.336067 19574 raft_consensus.cc:2468] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 0.
I20260709 12:31:07.337765 19389 leader_election.cc:304] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:07.338671 19705 raft_consensus.cc:2804] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:31:07.311781 19537 leader_election.cc:343] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.339030 19705 raft_consensus.cc:493] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.339594 19705 raft_consensus.cc:3060] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.334195 19572 raft_consensus.cc:2468] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:07.304869 19500 raft_consensus.cc:2468] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:07.344859 19728 tablet_bootstrap.cc:492] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.348767 19725 tablet_bootstrap.cc:654] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.348852 19713 raft_consensus.cc:493] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.349478 19713 raft_consensus.cc:515] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.350374 19389 leader_election.cc:304] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.351722 19713 leader_election.cc:290] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.352288 19713 raft_consensus.cc:2804] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.352720 19713 raft_consensus.cc:697] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:07.352731 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.352525 19323 catalog_manager.cc:5697] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:07.353796 19713 consensus_queue.cc:237] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
W20260709 12:31:07.354391 19388 leader_election.cc:343] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.359335 19726 raft_consensus.cc:359] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.363081 19726 raft_consensus.cc:385] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.363484 19726 raft_consensus.cc:740] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.361719 19725 tablet_bootstrap.cc:492] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.364598 19725 ts_tablet_manager.cc:1403] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.041s	user 0.014s	sys 0.010s
I20260709 12:31:07.362082 19705 raft_consensus.cc:515] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.345577 19728 ts_tablet_manager.cc:1403] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.019s
I20260709 12:31:07.368173 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.368994 19572 raft_consensus.cc:2468] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 0.
I20260709 12:31:07.370117 19728 raft_consensus.cc:359] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.370638 19705 leader_election.cc:290] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.370457 19389 leader_election.cc:304] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:07.374519 19726 consensus_queue.cc:260] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.370985 19728 raft_consensus.cc:385] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.375910 19728 raft_consensus.cc:740] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.377091 19572 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:07.378087 19747 raft_consensus.cc:493] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.372822 19725 raft_consensus.cc:359] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.376902 19728 consensus_queue.cc:260] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.377748 19726 ts_tablet_manager.cc:1434] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.043s	user 0.005s	sys 0.004s
I20260709 12:31:07.380059 19726 tablet_bootstrap.cc:492] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.372534 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.381003 19728 ts_tablet_manager.cc:1434] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:31:07.379055 19725 raft_consensus.cc:385] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.375978 19750 raft_consensus.cc:2804] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.382601 19725 raft_consensus.cc:740] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.382761 19750 raft_consensus.cc:493] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.379544 19321 catalog_manager.cc:5697] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.383564 19725 consensus_queue.cc:260] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
W20260709 12:31:07.381843 19388 leader_election.cc:343] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.378587 19572 raft_consensus.cc:3060] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.378726 19747 raft_consensus.cc:515] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.386634 19725 ts_tablet_manager.cc:1434] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.005s
I20260709 12:31:07.387799 19725 tablet_bootstrap.cc:492] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.389873 19747 leader_election.cc:290] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.383193 19750 raft_consensus.cc:3060] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.382190 19728 tablet_bootstrap.cc:492] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.395138 19725 tablet_bootstrap.cc:654] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.398962 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.399964 19726 tablet_bootstrap.cc:654] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
W20260709 12:31:07.400357 19388 leader_election.cc:343] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.404265 19750 raft_consensus.cc:515] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.405118 19572 raft_consensus.cc:2468] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:07.407423 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.407585 19750 leader_election.cc:290] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.410799 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.410805 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:07.412313 19571 raft_consensus.cc:3060] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:07.412714 19388 leader_election.cc:343] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.413589 19389 leader_election.cc:304] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:07.414371 19705 raft_consensus.cc:2804] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.414414 19725 tablet_bootstrap.cc:492] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.415098 19705 raft_consensus.cc:697] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:07.416720 19705 consensus_queue.cc:237] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.409555 19728 tablet_bootstrap.cc:654] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.411408 19726 tablet_bootstrap.cc:492] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.418982 19726 ts_tablet_manager.cc:1403] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
I20260709 12:31:07.421628 19571 raft_consensus.cc:2468] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 1.
I20260709 12:31:07.422773 19726 raft_consensus.cc:359] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.423624 19726 raft_consensus.cc:385] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.423938 19726 raft_consensus.cc:740] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
W20260709 12:31:07.415356 19389 leader_election.cc:343] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.415375 19725 ts_tablet_manager.cc:1403] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.006s
I20260709 12:31:07.424933 19726 consensus_queue.cc:260] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.427239 19728 tablet_bootstrap.cc:492] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.427871 19728 ts_tablet_manager.cc:1403] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.006s
I20260709 12:31:07.428073 19726 ts_tablet_manager.cc:1434] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:31:07.427666 19725 raft_consensus.cc:359] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.429858 19725 raft_consensus.cc:385] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.430282 19725 raft_consensus.cc:740] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.431221 19725 consensus_queue.cc:260] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.432941 19728 raft_consensus.cc:359] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.433758 19728 raft_consensus.cc:385] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.433411 19322 catalog_manager.cc:5697] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.434093 19728 raft_consensus.cc:740] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.434084 19725 ts_tablet_manager.cc:1434] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:31:07.428589 19389 leader_election.cc:304] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:07.435228 19728 consensus_queue.cc:260] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.437095 19389 leader_election.cc:304] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:07.438182 19728 ts_tablet_manager.cc:1434] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:31:07.439487 19728 tablet_bootstrap.cc:492] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.435249 19726 tablet_bootstrap.cc:492] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.443475 19750 raft_consensus.cc:2804] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.444074 19750 raft_consensus.cc:697] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:07.444909 19750 consensus_queue.cc:237] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.447674 19726 tablet_bootstrap.cc:654] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.450695 19725 tablet_bootstrap.cc:492] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.438223 19705 raft_consensus.cc:2749] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.456529 19728 tablet_bootstrap.cc:654] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.458279 19725 tablet_bootstrap.cc:654] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.464634 19725 tablet_bootstrap.cc:492] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.465181 19725 ts_tablet_manager.cc:1403] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.029s	user 0.002s	sys 0.009s
I20260709 12:31:07.466441 19728 tablet_bootstrap.cc:492] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.467300 19728 ts_tablet_manager.cc:1403] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.028s	user 0.015s	sys 0.000s
I20260709 12:31:07.469914 19725 raft_consensus.cc:359] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.470815 19725 raft_consensus.cc:385] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.470918 19726 tablet_bootstrap.cc:492] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.471416 19725 raft_consensus.cc:740] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.470846 19728 raft_consensus.cc:359] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.471952 19726 ts_tablet_manager.cc:1403] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.008s
I20260709 12:31:07.472123 19728 raft_consensus.cc:385] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.472738 19728 raft_consensus.cc:740] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.472420 19725 consensus_queue.cc:260] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.473974 19728 consensus_queue.cc:260] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.475344 19725 ts_tablet_manager.cc:1434] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 12:31:07.476491 19725 tablet_bootstrap.cc:492] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.477530 19726 raft_consensus.cc:359] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.479522 19728 ts_tablet_manager.cc:1434] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:31:07.479756 19726 raft_consensus.cc:385] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.480269 19726 raft_consensus.cc:740] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.480829 19728 tablet_bootstrap.cc:492] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.481016 19726 consensus_queue.cc:260] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.483382 19726 ts_tablet_manager.cc:1434] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.001s
I20260709 12:31:07.484369 19726 tablet_bootstrap.cc:492] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: Bootstrap starting.
I20260709 12:31:07.486250 19725 tablet_bootstrap.cc:654] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.474431 19320 catalog_manager.cc:5697] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.489341 19728 tablet_bootstrap.cc:654] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.494551 19726 tablet_bootstrap.cc:654] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.499831 19725 tablet_bootstrap.cc:492] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.500532 19725 ts_tablet_manager.cc:1403] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:31:07.502126 19728 tablet_bootstrap.cc:492] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.502810 19728 ts_tablet_manager.cc:1403] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.010s
I20260709 12:31:07.504086 19725 raft_consensus.cc:359] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.504976 19725 raft_consensus.cc:385] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.505312 19725 raft_consensus.cc:740] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.505918 19728 raft_consensus.cc:359] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.506170 19725 consensus_queue.cc:260] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.506852 19728 raft_consensus.cc:385] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.507215 19728 raft_consensus.cc:740] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.508250 19739 raft_consensus.cc:493] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.509706 19725 ts_tablet_manager.cc:1434] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:31:07.509585 19739 raft_consensus.cc:515] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.511035 19725 tablet_bootstrap.cc:492] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.512039 19739 leader_election.cc:290] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:07.514472 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
W20260709 12:31:07.516335 19538 leader_election.cc:343] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.518500 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.508056 19728 consensus_queue.cc:260] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.519291 19500 raft_consensus.cc:2468] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:07.520675 19738 raft_consensus.cc:493] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.521548 19728 ts_tablet_manager.cc:1434] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:31:07.521827 19537 leader_election.cc:304] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.520267 19739 raft_consensus.cc:493] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.521214 19738 raft_consensus.cc:515] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.522845 19711 raft_consensus.cc:2804] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.523351 19739 raft_consensus.cc:515] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.525207 19726 tablet_bootstrap.cc:492] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: No bootstrap required, opened a new log
I20260709 12:31:07.525743 19739 leader_election.cc:290] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:07.523717 19711 raft_consensus.cc:493] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.526556 19711 raft_consensus.cc:3060] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.527694 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.528633 19424 raft_consensus.cc:2468] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:07.529175 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.530042 19571 raft_consensus.cc:2393] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 051c8a7691c2489bb513f0e150a6089b in current term 1: Already voted for candidate d55397b85c414ed0ad7f5481cd5a6c87 in this term.
I20260709 12:31:07.531620 19538 leader_election.cc:304] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.531663 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.532598 19499 raft_consensus.cc:2468] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 0.
I20260709 12:31:07.533208 19739 raft_consensus.cc:2804] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.524956 19738 leader_election.cc:290] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.527793 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.535301 19464 leader_election.cc:304] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.536149 19711 raft_consensus.cc:515] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.536607 19738 raft_consensus.cc:3060] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.519254 19725 tablet_bootstrap.cc:654] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.539808 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:07.540727 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.525768 19726 ts_tablet_manager.cc:1403] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent bootstrapping tablet: real 0.042s	user 0.016s	sys 0.004s
I20260709 12:31:07.541443 19499 raft_consensus.cc:3060] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.545358 19726 raft_consensus.cc:359] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.546180 19726 raft_consensus.cc:385] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:31:07.541401 19538 leader_election.cc:343] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.546540 19726 raft_consensus.cc:740] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.546958 19711 leader_election.cc:290] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:07.547091 19739 raft_consensus.cc:493] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.546664 19725 tablet_bootstrap.cc:492] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.547991 19725 ts_tablet_manager.cc:1403] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.001s
I20260709 12:31:07.547631 19739 raft_consensus.cc:3060] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.549402 19738 raft_consensus.cc:2749] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.522778 19728 tablet_bootstrap.cc:492] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: Bootstrap starting.
I20260709 12:31:07.551433 19499 raft_consensus.cc:2468] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 1.
I20260709 12:31:07.551513 19725 raft_consensus.cc:359] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.552459 19725 raft_consensus.cc:385] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.552812 19725 raft_consensus.cc:740] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.553071 19537 leader_election.cc:304] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.547657 19726 consensus_queue.cc:260] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.553741 19725 consensus_queue.cc:260] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.554469 19711 raft_consensus.cc:2804] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.554713 19750 raft_consensus.cc:493] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.555253 19711 raft_consensus.cc:697] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:07.555331 19750 raft_consensus.cc:515] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.556231 19711 consensus_queue.cc:237] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.556870 19728 tablet_bootstrap.cc:654] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.558933 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.559819 19500 raft_consensus.cc:2393] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d55397b85c414ed0ad7f5481cd5a6c87 in current term 1: Already voted for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in this term.
I20260709 12:31:07.560830 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.562947 19389 leader_election.cc:304] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:07.564951 19705 raft_consensus.cc:3060] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.569785 19750 leader_election.cc:290] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.573062 19725 ts_tablet_manager.cc:1434] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:31:07.575213 19705 raft_consensus.cc:2749] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.577615 19714 raft_consensus.cc:493] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.578189 19714 raft_consensus.cc:515] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.580549 19714 leader_election.cc:290] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.581903 19726 ts_tablet_manager.cc:1434] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.006s
I20260709 12:31:07.582262 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.584014 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.588898 19423 raft_consensus.cc:2468] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:07.591194 19739 raft_consensus.cc:515] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.593456 19739 leader_election.cc:290] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
W20260709 12:31:07.595220 19464 leader_election.cc:343] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:07.596439 19464 leader_election.cc:304] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:07.597811 19725 tablet_bootstrap.cc:492] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: Bootstrap starting.
I20260709 12:31:07.597983 19714 raft_consensus.cc:2804] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.598671 19714 raft_consensus.cc:493] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.599133 19714 raft_consensus.cc:3060] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.599270 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:07.597824 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:07.600930 19423 raft_consensus.cc:3060] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.603806 19728 tablet_bootstrap.cc:492] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: No bootstrap required, opened a new log
I20260709 12:31:07.604347 19728 ts_tablet_manager.cc:1403] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent bootstrapping tablet: real 0.082s	user 0.031s	sys 0.016s
I20260709 12:31:07.606009 19711 raft_consensus.cc:493] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.606134 19725 tablet_bootstrap.cc:654] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:07.600415 19499 raft_consensus.cc:3060] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.606631 19711 raft_consensus.cc:515] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.609464 19728 raft_consensus.cc:359] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.610204 19728 raft_consensus.cc:385] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.609197 19320 catalog_manager.cc:5697] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:07.612251 19424 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.610450 19728 raft_consensus.cc:740] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.614338 19728 consensus_queue.cc:260] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.615312 19711 leader_election.cc:290] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:07.617432 19728 ts_tablet_manager.cc:1434] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:31:07.617440 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051c8a7691c2489bb513f0e150a6089b" is_pre_election: true
I20260709 12:31:07.618734 19500 raft_consensus.cc:2393] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in current term 1: Already voted for candidate d55397b85c414ed0ad7f5481cd5a6c87 in this term.
I20260709 12:31:07.620967 19537 leader_election.cc:304] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.621598 19725 tablet_bootstrap.cc:492] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: No bootstrap required, opened a new log
I20260709 12:31:07.622157 19725 ts_tablet_manager.cc:1403] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.008s
I20260709 12:31:07.622627 19499 raft_consensus.cc:2468] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 1.
I20260709 12:31:07.622141 19739 raft_consensus.cc:3060] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.624013 19537 leader_election.cc:304] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:07.625373 19725 raft_consensus.cc:359] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.626034 19725 raft_consensus.cc:385] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:07.626366 19725 raft_consensus.cc:740] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Initialized, Role: FOLLOWER
I20260709 12:31:07.627241 19725 consensus_queue.cc:260] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.628309 19714 raft_consensus.cc:515] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.629817 19423 raft_consensus.cc:2468] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 1.
I20260709 12:31:07.628232 19711 raft_consensus.cc:2804] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.631019 19711 raft_consensus.cc:697] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:07.631934 19711 consensus_queue.cc:237] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.633153 19725 ts_tablet_manager.cc:1434] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:31:07.635002 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:07.635695 19571 raft_consensus.cc:3060] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.643800 19571 raft_consensus.cc:2468] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:07.645049 19464 leader_election.cc:304] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:07.645404 19320 catalog_manager.cc:5697] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:07.648797 19739 raft_consensus.cc:2749] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.648986 19738 raft_consensus.cc:2804] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.652709 19738 raft_consensus.cc:697] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:07.653599 19738 consensus_queue.cc:237] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.654661 19714 leader_election.cc:290] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.655807 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:07.656481 19423 raft_consensus.cc:3060] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.662370 19423 raft_consensus.cc:2468] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:07.662976 19320 catalog_manager.cc:5697] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.679455 19705 consensus_queue.cc:1048] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:07.695040 19705 consensus_queue.cc:1048] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:07.709880 19738 raft_consensus.cc:493] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.710520 19738 raft_consensus.cc:515] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.713099 19738 leader_election.cc:290] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.713272 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.714332 19423 raft_consensus.cc:2393] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 051c8a7691c2489bb513f0e150a6089b in current term 1: Already voted for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in this term.
I20260709 12:31:07.715013 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.717983 19464 leader_election.cc:304] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.718747 19739 consensus_queue.cc:1048] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:07.719131 19738 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.727921 19738 raft_consensus.cc:2749] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.733645 19739 consensus_queue.cc:1048] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:07.746516 19705 consensus_queue.cc:1048] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:07.753334 19738 raft_consensus.cc:493] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.753765 19738 raft_consensus.cc:515] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.755882 19738 leader_election.cc:290] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.757695 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.758421 19423 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:07.759042 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.759678 19571 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 0.
I20260709 12:31:07.759874 19464 leader_election.cc:304] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.760064 19747 consensus_queue.cc:1048] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:31:07.761461 19371 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:07.761658 19738 raft_consensus.cc:2804] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:07.762106 19738 raft_consensus.cc:493] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:07.762518 19738 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.775694 19738 raft_consensus.cc:515] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.780169 19738 leader_election.cc:290] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.780939 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:07.781464 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:07.781780 19423 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.782250 19571 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.788066 19423 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:07.789273 19464 leader_election.cc:304] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:07.789369 19571 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 1.
I20260709 12:31:07.790175 19738 raft_consensus.cc:2804] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:07.790742 19738 raft_consensus.cc:697] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:07.791520 19738 consensus_queue.cc:237] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.800479 19321 catalog_manager.cc:5697] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 0 to 1, leader changed from <none> to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 1 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:07.914366 19738 raft_consensus.cc:493] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:07.915023 19738 raft_consensus.cc:515] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:07.916997 19738 leader_election.cc:290] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:07.918040 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" is_pre_election: true
I20260709 12:31:07.918262 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" is_pre_election: true
I20260709 12:31:07.919337 19571 raft_consensus.cc:2393] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 051c8a7691c2489bb513f0e150a6089b in current term 1: Already voted for candidate d55397b85c414ed0ad7f5481cd5a6c87 in this term.
I20260709 12:31:07.920388 19464 leader_election.cc:304] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b; no voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:07.921209 19738 raft_consensus.cc:3060] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:07.927421 19738 raft_consensus.cc:2749] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:07.968809 19739 consensus_queue.cc:1048] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:07.968937 19747 consensus_queue.cc:1048] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:07.985347 19705 consensus_queue.cc:1048] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:07.986851 19705 consensus_queue.cc:1048] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:07.987763 19741 consensus_queue.cc:1048] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:08.001200 19705 consensus_queue.cc:1048] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:08.021958 19574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "959339bed78841e5b523d05b7cc25273"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:41672
I20260709 12:31:08.022661 19574 raft_consensus.cc:606] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:08.032603 19500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:58136
I20260709 12:31:08.033383 19500 raft_consensus.cc:606] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:08.040766 19500 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:58136
I20260709 12:31:08.041491 19500 raft_consensus.cc:606] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:08.047709 19574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21d4bb10d9ce456685da713f02091158"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
mode: GRACEFUL
new_leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:41672
I20260709 12:31:08.048344 19574 raft_consensus.cc:606] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Received request to transfer leadership to TS d55397b85c414ed0ad7f5481cd5a6c87
I20260709 12:31:08.064327 19423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
mode: GRACEFUL
new_leader_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:34088
I20260709 12:31:08.064992 19423 raft_consensus.cc:606] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Received request to transfer leadership to TS 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:08.070781 19423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a3720e40313144c897120c99a6308819"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
mode: GRACEFUL
new_leader_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:34088
I20260709 12:31:08.071465 19423 raft_consensus.cc:606] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Received request to transfer leadership to TS 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:08.076813 19574 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
mode: GRACEFUL
new_leader_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:41672
I20260709 12:31:08.077452 19574 raft_consensus.cc:606] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Received request to transfer leadership to TS 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:08.084714 19423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "efd53179cce647948755bc8fc079cbba"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
mode: GRACEFUL
new_leader_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:34088
I20260709 12:31:08.085263 19741 consensus_queue.cc:1048] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:08.085523 19423 raft_consensus.cc:606] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Received request to transfer leadership to TS 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:08.094058 19739 raft_consensus.cc:993] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.094631 19711 raft_consensus.cc:1081] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.097321 19425 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "959339bed78841e5b523d05b7cc25273"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37126
I20260709 12:31:08.101114 19425 raft_consensus.cc:493] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.101616 19425 raft_consensus.cc:3060] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.102619 19423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
mode: GRACEFUL
new_leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:34088
I20260709 12:31:08.103459 19423 raft_consensus.cc:606] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Received request to transfer leadership to TS a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:08.105281 19741 consensus_queue.cc:1048] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:08.111485 19425 raft_consensus.cc:515] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.118672 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.118702 19498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "959339bed78841e5b523d05b7cc25273" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:08.119451 19574 raft_consensus.cc:3055] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.119858 19574 raft_consensus.cc:740] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:08.120509 19425 leader_election.cc:290] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.120852 19574 consensus_queue.cc:260] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.121999 19574 raft_consensus.cc:3060] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.129760 19574 raft_consensus.cc:2468] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.131090 19389 leader_election.cc:304] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 051c8a7691c2489bb513f0e150a6089b
I20260709 12:31:08.132046 19705 raft_consensus.cc:2804] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.132586 19705 raft_consensus.cc:697] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.133469 19705 consensus_queue.cc:237] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.140569 19320 catalog_manager.cc:5697] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.144428 19714 consensus_queue.cc:1048] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:08.151852 19738 raft_consensus.cc:993] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.152424 19767 raft_consensus.cc:1081] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.155228 19423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37120
I20260709 12:31:08.156021 19423 raft_consensus.cc:493] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.156554 19423 raft_consensus.cc:3060] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.160138 19714 consensus_queue.cc:1048] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:08.170087 19423 raft_consensus.cc:515] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.171909 19423 leader_election.cc:290] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.172983 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.173707 19574 raft_consensus.cc:3060] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.174688 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "06b85cf2ec024e79bdb45a772107022b" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:08.175335 19500 raft_consensus.cc:3055] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.175691 19500 raft_consensus.cc:740] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:08.176570 19500 consensus_queue.cc:260] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.178088 19500 raft_consensus.cc:3060] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.182973 19574 raft_consensus.cc:2468] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.184552 19389 leader_election.cc:304] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:08.185616 19705 raft_consensus.cc:2804] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.186276 19705 raft_consensus.cc:697] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.187151 19500 raft_consensus.cc:2468] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.187276 19705 consensus_queue.cc:237] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.195412 19322 catalog_manager.cc:5697] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.217382 19781 raft_consensus.cc:993] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: : Instructing follower 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.218019 19705 raft_consensus.cc:1081] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Signalling peer 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.219847 19500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:58108
I20260709 12:31:08.220511 19500 raft_consensus.cc:493] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.220978 19500 raft_consensus.cc:3060] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.228426 19500 raft_consensus.cc:515] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.230055 19500 leader_election.cc:290] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.230742 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:08.231191 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a5d7d0da151049a08e785e94683b05b3" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.231520 19423 raft_consensus.cc:3055] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.231887 19574 raft_consensus.cc:3060] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.231891 19423 raft_consensus.cc:740] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.232893 19423 consensus_queue.cc:260] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.233865 19423 raft_consensus.cc:3060] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.237012 19574 raft_consensus.cc:2468] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.238166 19464 leader_election.cc:304] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:08.239048 19767 raft_consensus.cc:2804] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.239108 19423 raft_consensus.cc:2468] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.239639 19767 raft_consensus.cc:697] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:08.240573 19767 consensus_queue.cc:237] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.248314 19322 catalog_manager.cc:5697] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 1 to 2, leader changed from d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65) to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 2 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.252702 19767 consensus_queue.cc:1048] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:08.267267 19714 consensus_queue.cc:1048] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:08.275255 19741 raft_consensus.cc:993] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: : Instructing follower 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.275897 19741 raft_consensus.cc:1081] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Signalling peer 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.278898 19500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:58120
I20260709 12:31:08.280055 19500 raft_consensus.cc:493] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.280486 19500 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.283041 19738 raft_consensus.cc:993] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.283636 19738 raft_consensus.cc:1081] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.287684 19423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37120
I20260709 12:31:08.288364 19423 raft_consensus.cc:493] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.288367 19500 raft_consensus.cc:515] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.289074 19423 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.290860 19500 leader_election.cc:290] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.293066 19574 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.293673 19574 raft_consensus.cc:3055] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.293529 19425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:08.294224 19574 raft_consensus.cc:740] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:08.294445 19425 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.295321 19574 consensus_queue.cc:260] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.296808 19574 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.298118 19423 raft_consensus.cc:515] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.301680 19425 raft_consensus.cc:2468] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.302922 19498 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:08.303596 19498 raft_consensus.cc:3055] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.303920 19498 raft_consensus.cc:740] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:08.304396 19464 leader_election.cc:304] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:08.304944 19423 leader_election.cc:290] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.304782 19498 consensus_queue.cc:260] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.305372 19714 raft_consensus.cc:2804] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.306025 19714 raft_consensus.cc:697] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:08.306460 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.306182 19498 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.307163 19571 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.307072 19714 consensus_queue.cc:237] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.312434 19574 raft_consensus.cc:2468] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.315771 19571 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.317278 19389 leader_election.cc:304] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:08.316974 19322 catalog_manager.cc:5697] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 1 to 2, leader changed from a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67) to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 2 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.318176 19747 raft_consensus.cc:2804] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.318718 19747 raft_consensus.cc:697] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.319358 19498 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.319543 19747 consensus_queue.cc:237] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.328855 19320 catalog_manager.cc:5697] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.383284 19705 raft_consensus.cc:993] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: : Instructing follower 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.383872 19747 raft_consensus.cc:1081] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Signalling peer 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.385427 19500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "efd53179cce647948755bc8fc079cbba"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:58108
I20260709 12:31:08.386020 19500 raft_consensus.cc:493] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.386350 19500 raft_consensus.cc:3060] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.392194 19500 raft_consensus.cc:515] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.393923 19500 leader_election.cc:290] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.394897 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efd53179cce647948755bc8fc079cbba" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:08.395143 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efd53179cce647948755bc8fc079cbba" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.395659 19423 raft_consensus.cc:3055] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.395819 19571 raft_consensus.cc:3060] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.396018 19423 raft_consensus.cc:740] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.396883 19423 consensus_queue.cc:260] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.397802 19423 raft_consensus.cc:3060] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.402021 19571 raft_consensus.cc:2468] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.402882 19423 raft_consensus.cc:2468] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.403463 19464 leader_election.cc:304] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:08.404466 19714 raft_consensus.cc:2804] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.405004 19714 raft_consensus.cc:697] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:08.405855 19714 consensus_queue.cc:237] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.412591 19320 catalog_manager.cc:5697] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 1 to 2, leader changed from d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65) to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 2 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.445412 19739 raft_consensus.cc:993] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: : Instructing follower d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.445948 19711 raft_consensus.cc:1081] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Signalling peer d55397b85c414ed0ad7f5481cd5a6c87 to start an election
I20260709 12:31:08.447785 19423 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21d4bb10d9ce456685da713f02091158"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
 from {username='slave'} at 127.0.0.1:37126
I20260709 12:31:08.448328 19423 raft_consensus.cc:493] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.448690 19423 raft_consensus.cc:3060] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.455554 19423 raft_consensus.cc:515] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.457319 19423 leader_election.cc:290] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Requested vote from peers 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.458316 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:08.458457 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d4bb10d9ce456685da713f02091158" candidate_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.459405 19500 raft_consensus.cc:3060] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.459487 19571 raft_consensus.cc:3055] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.459996 19571 raft_consensus.cc:740] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:08.461005 19571 consensus_queue.cc:260] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.463032 19571 raft_consensus.cc:3060] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.468717 19500 raft_consensus.cc:2468] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.470316 19388 leader_election.cc:304] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:08.471314 19747 raft_consensus.cc:2804] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.471607 19571 raft_consensus.cc:2468] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d55397b85c414ed0ad7f5481cd5a6c87 in term 2.
I20260709 12:31:08.471886 19747 raft_consensus.cc:697] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Leader. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.472805 19747 consensus_queue.cc:237] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.480962 19320 catalog_manager.cc:5697] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 reported cstate change: term changed from 1 to 2, leader changed from a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67) to d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.490128 19705 raft_consensus.cc:993] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: : Instructing follower a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:08.490772 19747 raft_consensus.cc:1081] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Signalling peer a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:08.492228 19571 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:41662
I20260709 12:31:08.492663 19571 raft_consensus.cc:493] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.492918 19571 raft_consensus.cc:3060] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.498624 19571 raft_consensus.cc:515] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.500650 19571 leader_election.cc:290] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:08.501487 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:08.501802 19500 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "295ce06a4c544374b969e439c410f2b3" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:08.502197 19423 raft_consensus.cc:3055] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.502529 19500 raft_consensus.cc:3060] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.502655 19423 raft_consensus.cc:740] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.503746 19423 consensus_queue.cc:260] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.504776 19423 raft_consensus.cc:3060] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.507187 19500 raft_consensus.cc:2468] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 2.
I20260709 12:31:08.508435 19537 leader_election.cc:304] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:08.509182 19711 raft_consensus.cc:2804] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.509744 19711 raft_consensus.cc:697] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:08.510701 19711 consensus_queue.cc:237] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.511355 19423 raft_consensus.cc:2468] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 2.
I20260709 12:31:08.518112 19320 catalog_manager.cc:5697] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 1 to 2, leader changed from d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65) to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:08.563881 19705 raft_consensus.cc:993] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: : Instructing follower 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.564674 19747 raft_consensus.cc:1081] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Signalling peer 051c8a7691c2489bb513f0e150a6089b to start an election
I20260709 12:31:08.567174 19500 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a3720e40313144c897120c99a6308819"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
 from {username='slave'} at 127.0.0.1:58108
I20260709 12:31:08.567907 19500 raft_consensus.cc:493] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:08.568265 19500 raft_consensus.cc:3060] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.574079 19500 raft_consensus.cc:515] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.575982 19500 leader_election.cc:290] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67:44555)
I20260709 12:31:08.576781 19423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:08.577157 19571 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a3720e40313144c897120c99a6308819" candidate_uuid: "051c8a7691c2489bb513f0e150a6089b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
I20260709 12:31:08.577512 19423 raft_consensus.cc:3055] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:08.577715 19571 raft_consensus.cc:3060] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.577877 19423 raft_consensus.cc:740] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:08.578775 19423 consensus_queue.cc:260] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.580049 19423 raft_consensus.cc:3060] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.585237 19571 raft_consensus.cc:2468] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.586580 19464 leader_election.cc:304] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:08.586603 19423 raft_consensus.cc:2468] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 051c8a7691c2489bb513f0e150a6089b in term 2.
I20260709 12:31:08.587445 19714 raft_consensus.cc:2804] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:08.588035 19714 raft_consensus.cc:697] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Becoming Leader. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:08.588896 19714 consensus_queue.cc:237] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:08.596096 19320 catalog_manager.cc:5697] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b reported cstate change: term changed from 1 to 2, leader changed from d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65) to 051c8a7691c2489bb513f0e150a6089b (127.5.4.66). New cstate: current_term: 2 leader_uuid: "051c8a7691c2489bb513f0e150a6089b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:08.600205 19571 raft_consensus.cc:1275] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.602278 19705 consensus_queue.cc:1048] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.610918 19500 raft_consensus.cc:3060] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:08.613879 19574 raft_consensus.cc:1275] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.615978 19747 consensus_queue.cc:1048] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.620253 19500 raft_consensus.cc:1275] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.624413 19705 consensus_queue.cc:1048] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.630398 19500 raft_consensus.cc:1275] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.633528 19705 consensus_queue.cc:1048] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.704272 19571 raft_consensus.cc:1275] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.705704 19714 consensus_queue.cc:1048] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.716112 19423 raft_consensus.cc:1275] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.718266 19767 consensus_queue.cc:1048] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.854009 19423 raft_consensus.cc:1275] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.855654 19767 consensus_queue.cc:1048] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.860464 19574 raft_consensus.cc:1275] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.862181 19714 consensus_queue.cc:1048] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:08.866786 19571 raft_consensus.cc:1275] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.869869 19767 consensus_queue.cc:1048] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.871970 19500 raft_consensus.cc:1275] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.876188 19781 consensus_queue.cc:1048] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.884662 19423 raft_consensus.cc:1275] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.886780 19738 consensus_queue.cc:1048] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.890645 19571 raft_consensus.cc:1275] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.895934 19747 consensus_queue.cc:1048] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.908346 19498 raft_consensus.cc:1275] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.911335 19574 raft_consensus.cc:1275] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer d55397b85c414ed0ad7f5481cd5a6c87: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.912321 19713 consensus_queue.cc:1048] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.918605 19781 consensus_queue.cc:1048] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:08.991452 19423 raft_consensus.cc:1275] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:08.993078 19711 consensus_queue.cc:1048] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:09.003588 19498 raft_consensus.cc:1275] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:09.005842 19800 consensus_queue.cc:1048] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:09.037226 19574 raft_consensus.cc:1275] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:09.038691 19767 consensus_queue.cc:1048] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:09.048137 19423 raft_consensus.cc:1275] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer 051c8a7691c2489bb513f0e150a6089b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:09.049798 19738 consensus_queue.cc:1048] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:11.109817 19321 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35510:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:31:11.132190 19321 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35510:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:31:11.173558  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:11.292743 19839 tablet.cc:1724] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.331338 19854 tablet.cc:1724] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:11.339226 19388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44361, user_credentials={real_user=slave}} blocked reactor thread for 49296.5us
I20260709 12:31:11.354228 19853 tablet.cc:1724] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.359599 19423 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
mode: GRACEFUL
new_leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:34100
I20260709 12:31:11.369536 19423 raft_consensus.cc:606] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Received request to transfer leadership to TS a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:11.412195 19857 tablet.cc:1724] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:11.415905 19526 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:0a52e3c0cab0418f8795b7514ba56c87)
W20260709 12:31:11.456884 19526 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:0a52e3c0cab0418f8795b7514ba56c87)
W20260709 12:31:11.488198 19388 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.66:44361, user_credentials={real_user=slave}} blocked reactor thread for 121807us
W20260709 12:31:11.548179 19526 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:0a52e3c0cab0418f8795b7514ba56c87)
W20260709 12:31:11.556494 19841 log.cc:927] Time spent T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:31:11.569434 19861 tablet.cc:1724] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.582770 19863 tablet.cc:1724] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.580346 19858 raft_consensus.cc:993] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: : Instructing follower a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:11.585296 19851 raft_consensus.cc:1081] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Signalling peer a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
W20260709 12:31:11.602744 19464 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.67:44555, user_credentials={real_user=slave}} blocked reactor thread for 83826.7us
I20260709 12:31:11.569795 19862 tablet.cc:1724] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:11.600557 19538 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:40457, user_credentials={real_user=slave}} blocked reactor thread for 46177.5us
W20260709 12:31:11.635761 19865 log.cc:927] Time spent T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:31:11.638453 19570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:41662
I20260709 12:31:11.639170 19570 raft_consensus.cc:493] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:11.639582 19570 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:11.675395 19883 tablet.cc:1724] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.688720 19857 tablet.cc:1724] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.637502 19871 tablet.cc:1724] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.702080 19862 tablet.cc:1724] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.637507 19854 tablet.cc:1724] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.712594 19570 raft_consensus.cc:515] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:11.713982 19882 tablet.cc:1724] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.716776 19861 tablet.cc:1724] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.754467 19863 tablet.cc:1724] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.758889 19421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:11.779702 19570 leader_election.cc:290] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 3 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:11.782552 19853 tablet.cc:1724] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.768844 19421 raft_consensus.cc:3055] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:11.788412 19421 raft_consensus.cc:740] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:11.789319 19421 consensus_queue.cc:260] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
W20260709 12:31:11.793084 19887 log.cc:927] Time spent T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:31:11.754900 19884 tablet.cc:1724] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.797842 19421 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:11.800418 19839 tablet.cc:1724] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.810834 19496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a52e3c0cab0418f8795b7514ba56c87" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:11.811571 19496 raft_consensus.cc:3060] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:11.813405 19572 raft_consensus.cc:1240] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Rejecting Update request from peer d55397b85c414ed0ad7f5481cd5a6c87 for earlier term 2. Current term is 3. Ops: []
I20260709 12:31:11.819653 19499 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:49452
I20260709 12:31:11.820434 19499 raft_consensus.cc:606] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Received request to transfer leadership to TS a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:11.820739 19854 tablet.cc:1724] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.822356  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:31:11.825418  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:11.849392 19499 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "922473629e704fbabcee362b44ed7409"
dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
mode: GRACEFUL
new_leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:49452
I20260709 12:31:11.850114 19499 raft_consensus.cc:606] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Received request to transfer leadership to TS a2feef6cf9664cc18c8cdb7e5a4719f9
W20260709 12:31:11.856621 19890 log.cc:927] Time spent T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: Append to log took a long time: real 0.115s	user 0.000s	sys 0.002s
I20260709 12:31:11.870725 19857 tablet.cc:1724] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.873170 19861 tablet.cc:1724] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.885939 19883 tablet.cc:1724] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.880911 19420 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd"
dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
mode: GRACEFUL
new_leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:34100
I20260709 12:31:11.894928 19882 tablet.cc:1724] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.897069 19889 tablet.cc:1724] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.898983 19420 raft_consensus.cc:606] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Received request to transfer leadership to TS a2feef6cf9664cc18c8cdb7e5a4719f9
I20260709 12:31:11.898797 19871 tablet.cc:1724] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.920497 19862 tablet.cc:1724] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.922577  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:31:11.923266  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:11.952281 19845 raft_consensus.cc:993] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: : Instructing follower a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
W20260709 12:31:11.925491 19891 log.cc:927] Time spent T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:31:11.899569 19853 tablet.cc:1724] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:11.955861 19713 raft_consensus.cc:1081] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Signalling peer a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:11.923943 19421 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 3.
I20260709 12:31:11.958913 19538 leader_election.cc:304] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:11.959959 19874 raft_consensus.cc:2804] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:11.960445 19874 raft_consensus.cc:697] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:11.961268 19874 consensus_queue.cc:237] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:11.936942 19496 raft_consensus.cc:2468] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 3.
I20260709 12:31:11.978786 19572 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:41662
I20260709 12:31:11.979549 19572 raft_consensus.cc:493] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:11.979947 19572 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:11.993983 19872 raft_consensus.cc:993] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:11.994696 19880 raft_consensus.cc:1081] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Signalling peer a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:11.996336 19570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "922473629e704fbabcee362b44ed7409"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:41656
I20260709 12:31:11.996893 19570 raft_consensus.cc:493] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:11.997251 19570 raft_consensus.cc:3060] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:11.998546 19572 raft_consensus.cc:515] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.001529 19421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:12.002315 19421 raft_consensus.cc:3055] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:12.002710 19421 raft_consensus.cc:740] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d55397b85c414ed0ad7f5481cd5a6c87, State: Running, Role: LEADER
I20260709 12:31:12.000842 19322 catalog_manager.cc:6242] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab: soft_deleted_table [id=86838154c66a450481c98de157ec201f] alter complete (version 1)
I20260709 12:31:12.003623 19496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e077c0962e947c6bd92d06245f244fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:12.003844 19421 consensus_queue.cc:260] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.004369 19496 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:12.005113 19421 raft_consensus.cc:3060] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:12.005803 19572 leader_election.cc:290] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 3 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:12.008333 19570 raft_consensus.cc:515] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.010740 19570 leader_election.cc:290] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 2 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:12.015964 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:12.016594 19499 raft_consensus.cc:3055] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:12.016477 19422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "922473629e704fbabcee362b44ed7409" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:12.017036 19499 raft_consensus.cc:740] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:12.017228 19421 raft_consensus.cc:2468] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 3.
I20260709 12:31:12.017251 19422 raft_consensus.cc:3060] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.018004 19499 consensus_queue.cc:260] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.018712 19496 raft_consensus.cc:2468] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 3.
I20260709 12:31:12.019943 19499 raft_consensus.cc:3060] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:12.021497 19537 leader_election.cc:304] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:12.022332 19874 raft_consensus.cc:2804] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:12.022984 19874 raft_consensus.cc:697] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:12.023849 19874 consensus_queue.cc:237] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.027984 19499 raft_consensus.cc:2468] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 2.
I20260709 12:31:12.029170 19422 raft_consensus.cc:2468] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 2.
I20260709 12:31:12.029311 19537 leader_election.cc:304] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051c8a7691c2489bb513f0e150a6089b, a2feef6cf9664cc18c8cdb7e5a4719f9; no voters: 
I20260709 12:31:12.030263 19879 raft_consensus.cc:2804] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:12.017540 19322 catalog_manager.cc:5697] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 2 to 3, leader changed from d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65) to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:12.030781 19879 raft_consensus.cc:697] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:12.031579 19879 consensus_queue.cc:237] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.049449 19320 catalog_manager.cc:5697] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 1 to 2, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:12.050783 19320 catalog_manager.cc:5697] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 2 to 3, leader changed from d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65) to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:12.096719 19880 raft_consensus.cc:993] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: : Instructing follower a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:12.097316 19873 raft_consensus.cc:1081] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Signalling peer a2feef6cf9664cc18c8cdb7e5a4719f9 to start an election
I20260709 12:31:12.099247 19570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd"
dest_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9"
 from {username='slave'} at 127.0.0.1:41656
I20260709 12:31:12.099862 19570 raft_consensus.cc:493] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:12.100306 19570 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:12.106436 19570 raft_consensus.cc:515] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.108801 19570 leader_election.cc:290] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 3 election: Requested vote from peers d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457), 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361)
I20260709 12:31:12.109236 19422 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "d55397b85c414ed0ad7f5481cd5a6c87"
I20260709 12:31:12.109983 19422 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:12.109436 19499 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed20fe260b974e08b2aab4294797a6fd" candidate_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "051c8a7691c2489bb513f0e150a6089b"
I20260709 12:31:12.110930 19499 raft_consensus.cc:3055] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:12.111205 19499 raft_consensus.cc:740] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 051c8a7691c2489bb513f0e150a6089b, State: Running, Role: LEADER
I20260709 12:31:12.111927 19499 consensus_queue.cc:260] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.112847 19499 raft_consensus.cc:3060] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:12.117108 19422 raft_consensus.cc:2468] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 3.
I20260709 12:31:12.118526 19538 leader_election.cc:304] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2feef6cf9664cc18c8cdb7e5a4719f9, d55397b85c414ed0ad7f5481cd5a6c87; no voters: 
I20260709 12:31:12.119307 19879 raft_consensus.cc:2804] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:12.120044 19879 raft_consensus.cc:697] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 LEADER]: Becoming Leader. State: Replica: a2feef6cf9664cc18c8cdb7e5a4719f9, State: Running, Role: LEADER
I20260709 12:31:12.120877 19879 consensus_queue.cc:237] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } }
I20260709 12:31:12.122176 19499 raft_consensus.cc:2468] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2feef6cf9664cc18c8cdb7e5a4719f9 in term 3.
I20260709 12:31:12.132208 19320 catalog_manager.cc:5697] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 reported cstate change: term changed from 2 to 3, leader changed from 051c8a7691c2489bb513f0e150a6089b (127.5.4.66) to a2feef6cf9664cc18c8cdb7e5a4719f9 (127.5.4.67). New cstate: current_term: 3 leader_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2feef6cf9664cc18c8cdb7e5a4719f9" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 44555 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:12.465248 19422 raft_consensus.cc:1275] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:12.467198 19879 consensus_queue.cc:1048] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:12.481384 19499 raft_consensus.cc:1275] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:12.484706 19805 consensus_queue.cc:1048] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:31:12.561781 19499 raft_consensus.cc:1275] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:12.563270 19805 consensus_queue.cc:1048] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:12.578526 19422 raft_consensus.cc:1275] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:12.580781 19879 consensus_queue.cc:1048] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:12.616618 19499 raft_consensus.cc:1275] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:12.618111 19879 consensus_queue.cc:1048] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:12.631731 19422 raft_consensus.cc:1275] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:12.634933 19874 consensus_queue.cc:1048] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:12.729110 19499 raft_consensus.cc:1275] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:12.729538 19422 raft_consensus.cc:1275] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Refusing update from remote peer a2feef6cf9664cc18c8cdb7e5a4719f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:31:12.730683 19879 consensus_queue.cc:1048] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051c8a7691c2489bb513f0e150a6089b" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 44361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:12.731545 19874 consensus_queue.cc:1048] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d55397b85c414ed0ad7f5481cd5a6c87" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 40457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:31:12.924954  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:12.931070  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:12.931562  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:12.937317  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:12.937860  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:13.939348  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:13.945199  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:13.945741  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:13.952268  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:13.952721  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:14.953773  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:14.959189  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:14.959704  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:14.966403  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:14.966897  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:15.968036  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:15.973256  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:15.973716  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:15.979357  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:15.980005  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:17.254552  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:17.260111  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:17.260622  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:17.265676  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:17.266117  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:31:17.716481 19446 debug-util.cc:398] Leaking SignalData structure 0x7b08000c45c0 after lost signal to thread 5140
W20260709 12:31:17.717875 19446 debug-util.cc:398] Leaking SignalData structure 0x7b08001c2b60 after lost signal to thread 19357
W20260709 12:31:17.719343 19446 debug-util.cc:398] Leaking SignalData structure 0x7b08001e6e60 after lost signal to thread 19449
W20260709 12:31:17.720871 19446 debug-util.cc:398] Leaking SignalData structure 0x7b08000e2b20 after lost signal to thread 19524
W20260709 12:31:17.722160 19446 debug-util.cc:398] Leaking SignalData structure 0x7b0800337b40 after lost signal to thread 19598
I20260709 12:31:18.267392  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:18.275687  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:18.276232  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:18.284072  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:18.284570  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:19.285779  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:19.294375  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:19.295004  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:19.303048  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:19.303594  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:20.304860  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:20.313009  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:20.313557  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:20.321462  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:20.322022  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:21.323258  5137 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:31:21.328521  5137 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:31:21.328897  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:21.333433  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:21.333741  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:22.412106  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
W20260709 12:31:22.416477 19464 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 12:31:22.421773 19464 consensus_peers.cc:597] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.424592 19464 consensus_peers.cc:597] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.437357 19538 consensus_peers.cc:597] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.461081 19464 consensus_peers.cc:597] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.501387  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:22.502722  5137 tablet_replica.cc:333] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.504128  5137 raft_consensus.cc:2243] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.505781  5137 raft_consensus.cc:2272] T 21d4bb10d9ce456685da713f02091158 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.509384  5137 tablet_replica.cc:333] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.510082  5137 raft_consensus.cc:2243] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.511552  5137 raft_consensus.cc:2272] T 06b85cf2ec024e79bdb45a772107022b P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.515831  5137 tablet_replica.cc:333] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.516773  5137 raft_consensus.cc:2243] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.517921  5137 raft_consensus.cc:2272] T 0a52e3c0cab0418f8795b7514ba56c87 P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.521013  5137 tablet_replica.cc:333] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.522199  5137 raft_consensus.cc:2243] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.523669  5137 raft_consensus.cc:2272] T a3720e40313144c897120c99a6308819 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:22.526974 19464 consensus_peers.cc:597] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.527892 19464 consensus_peers.cc:597] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.528808  5137 tablet_replica.cc:333] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.529539  5137 raft_consensus.cc:2243] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.530239  5137 raft_consensus.cc:2272] T 295ce06a4c544374b969e439c410f2b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.533298  5137 tablet_replica.cc:333] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.533900  5137 raft_consensus.cc:2243] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.534538  5137 raft_consensus.cc:2272] T 9e077c0962e947c6bd92d06245f244fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.537377  5137 tablet_replica.cc:333] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.538017  5137 raft_consensus.cc:2243] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.538676  5137 raft_consensus.cc:2272] T a5d7d0da151049a08e785e94683b05b3 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.541538  5137 tablet_replica.cc:333] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.542092  5137 raft_consensus.cc:2243] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.542977  5137 raft_consensus.cc:2272] T ed20fe260b974e08b2aab4294797a6fd P d55397b85c414ed0ad7f5481cd5a6c87 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.548127  5137 tablet_replica.cc:333] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.549901  5137 raft_consensus.cc:2243] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.551007  5137 raft_consensus.cc:2272] T 3c642d6fa088414e84482a66916fe5cf P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.555117  5137 tablet_replica.cc:333] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.556046  5137 raft_consensus.cc:2243] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.557875  5137 raft_consensus.cc:2272] T 959339bed78841e5b523d05b7cc25273 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.560683  5137 tablet_replica.cc:333] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.561288  5137 raft_consensus.cc:2243] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.561895  5137 raft_consensus.cc:2272] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.564865  5137 tablet_replica.cc:333] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.565794  5137 raft_consensus.cc:2243] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.566469  5137 raft_consensus.cc:2272] T efd53179cce647948755bc8fc079cbba P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.569074  5137 tablet_replica.cc:333] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.569723  5137 raft_consensus.cc:2243] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.570349  5137 raft_consensus.cc:2272] T 9b0aeda8cdb248f2833025fda78b4f9c P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.572875  5137 tablet_replica.cc:333] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.573475  5137 raft_consensus.cc:2243] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.574813  5137 raft_consensus.cc:2272] T 013fb25300cb44e6ac3d9a1ece427992 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.577205  5137 tablet_replica.cc:333] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.577795  5137 raft_consensus.cc:2243] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.578868  5137 raft_consensus.cc:2272] T 1df2d8c357c44bc08e7478ac539f508a P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.581954  5137 tablet_replica.cc:333] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.583165  5137 raft_consensus.cc:2243] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.584095  5137 raft_consensus.cc:2272] T 922473629e704fbabcee362b44ed7409 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:22.586346 19538 consensus_peers.cc:597] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.588641  5137 tablet_replica.cc:333] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
W20260709 12:31:22.588937 19464 consensus_peers.cc:597] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.589473  5137 raft_consensus.cc:2243] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.590199  5137 raft_consensus.cc:2272] T 9f21d676c01f48bab23326a671a9c6b1 P d55397b85c414ed0ad7f5481cd5a6c87 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.592978  5137 tablet_replica.cc:333] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87: stopping tablet replica
I20260709 12:31:22.593516  5137 raft_consensus.cc:2243] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.594614  5137 raft_consensus.cc:2272] T 9372849a880b49e49e1753cc6f57c87e P d55397b85c414ed0ad7f5481cd5a6c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.665606  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
I20260709 12:31:22.716717  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
W20260709 12:31:22.717087 19538 consensus_peers.cc:597] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:40457: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.744853 19537 consensus_peers.cc:597] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Couldn't send request to peer 051c8a7691c2489bb513f0e150a6089b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.752063 19538 consensus_peers.cc:597] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:40457: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.769501  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:22.770648  5137 tablet_replica.cc:333] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.772037  5137 raft_consensus.cc:2243] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.772737  5137 raft_consensus.cc:2272] T 0a52e3c0cab0418f8795b7514ba56c87 P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.775804  5137 tablet_replica.cc:333] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.776456  5137 raft_consensus.cc:2243] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.777060  5137 raft_consensus.cc:2272] T 295ce06a4c544374b969e439c410f2b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.779871  5137 tablet_replica.cc:333] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.780517  5137 raft_consensus.cc:2243] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.781587  5137 raft_consensus.cc:2272] T a3720e40313144c897120c99a6308819 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.784413  5137 tablet_replica.cc:333] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.784971  5137 raft_consensus.cc:2243] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.785559  5137 raft_consensus.cc:2272] T 06b85cf2ec024e79bdb45a772107022b P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.788540  5137 tablet_replica.cc:333] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.789163  5137 raft_consensus.cc:2243] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.789820  5137 raft_consensus.cc:2272] T 21d4bb10d9ce456685da713f02091158 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.792496  5137 tablet_replica.cc:333] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.793226  5137 raft_consensus.cc:2243] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.793794  5137 raft_consensus.cc:2272] T 922473629e704fbabcee362b44ed7409 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.796447  5137 tablet_replica.cc:333] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.797067  5137 raft_consensus.cc:2243] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.797668  5137 raft_consensus.cc:2272] T 1df2d8c357c44bc08e7478ac539f508a P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.800120  5137 tablet_replica.cc:333] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.800822  5137 raft_consensus.cc:2243] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.801402  5137 raft_consensus.cc:2272] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.804894  5137 tablet_replica.cc:333] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.805858  5137 raft_consensus.cc:2243] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:31:22.806098 19538 consensus_peers.cc:597] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:40457: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.806660  5137 raft_consensus.cc:2272] T 959339bed78841e5b523d05b7cc25273 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.809453  5137 tablet_replica.cc:333] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.810180  5137 raft_consensus.cc:2243] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.811329  5137 raft_consensus.cc:2272] T efd53179cce647948755bc8fc079cbba P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.813989  5137 tablet_replica.cc:333] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.814653  5137 raft_consensus.cc:2243] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.815654  5137 raft_consensus.cc:2272] T 9b0aeda8cdb248f2833025fda78b4f9c P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.818200  5137 tablet_replica.cc:333] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.818842  5137 raft_consensus.cc:2243] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.819805  5137 raft_consensus.cc:2272] T 9f21d676c01f48bab23326a671a9c6b1 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.822191  5137 tablet_replica.cc:333] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.822865  5137 raft_consensus.cc:2243] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.823400  5137 raft_consensus.cc:2272] T 013fb25300cb44e6ac3d9a1ece427992 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.825839  5137 tablet_replica.cc:333] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.826392  5137 raft_consensus.cc:2243] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.826992  5137 raft_consensus.cc:2272] T 9e077c0962e947c6bd92d06245f244fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.829704  5137 tablet_replica.cc:333] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.830385  5137 raft_consensus.cc:2243] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.832105  5137 raft_consensus.cc:2272] T a5d7d0da151049a08e785e94683b05b3 P 051c8a7691c2489bb513f0e150a6089b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:22.835915 19538 consensus_peers.cc:597] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer d55397b85c414ed0ad7f5481cd5a6c87 (127.5.4.65:40457): Couldn't send request to peer d55397b85c414ed0ad7f5481cd5a6c87. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:40457: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.836396  5137 tablet_replica.cc:333] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.837257  5137 raft_consensus.cc:2243] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.838021  5137 raft_consensus.cc:2272] T ed20fe260b974e08b2aab4294797a6fd P 051c8a7691c2489bb513f0e150a6089b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.841799  5137 tablet_replica.cc:333] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.842393  5137 raft_consensus.cc:2243] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.843680  5137 raft_consensus.cc:2272] T 3c642d6fa088414e84482a66916fe5cf P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.846766  5137 tablet_replica.cc:333] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b: stopping tablet replica
I20260709 12:31:22.847287  5137 raft_consensus.cc:2243] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.847787  5137 raft_consensus.cc:2272] T 9372849a880b49e49e1753cc6f57c87e P 051c8a7691c2489bb513f0e150a6089b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.897783  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
W20260709 12:31:22.911507 19537 consensus_peers.cc:597] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Couldn't send request to peer 051c8a7691c2489bb513f0e150a6089b. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:22.938253 19537 consensus_peers.cc:597] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Couldn't send request to peer 051c8a7691c2489bb513f0e150a6089b. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.942454  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
W20260709 12:31:22.964681 19537 consensus_peers.cc:597] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Couldn't send request to peer 051c8a7691c2489bb513f0e150a6089b. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:22.986169  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:22.987262  5137 tablet_replica.cc:333] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:22.988412  5137 raft_consensus.cc:2243] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.989250  5137 raft_consensus.cc:2272] T 06b85cf2ec024e79bdb45a772107022b P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.992040  5137 tablet_replica.cc:333] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:22.992666  5137 raft_consensus.cc:2243] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:22.993816  5137 raft_consensus.cc:2272] T 0a52e3c0cab0418f8795b7514ba56c87 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:22.996364  5137 tablet_replica.cc:333] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:22.997000  5137 raft_consensus.cc:2243] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:22.997614  5137 raft_consensus.cc:2272] T 21d4bb10d9ce456685da713f02091158 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.000008  5137 tablet_replica.cc:333] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.000646  5137 raft_consensus.cc:2243] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.001186  5137 raft_consensus.cc:2272] T a3720e40313144c897120c99a6308819 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.003703  5137 tablet_replica.cc:333] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.004276  5137 raft_consensus.cc:2243] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:23.005280  5137 raft_consensus.cc:2272] T 295ce06a4c544374b969e439c410f2b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.007771  5137 tablet_replica.cc:333] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.008404  5137 raft_consensus.cc:2243] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.008874  5137 raft_consensus.cc:2272] T 3c642d6fa088414e84482a66916fe5cf P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.011547  5137 tablet_replica.cc:333] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.012173  5137 raft_consensus.cc:2243] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:23.013208  5137 raft_consensus.cc:2272] T ed20fe260b974e08b2aab4294797a6fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.015663  5137 tablet_replica.cc:333] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.016317  5137 raft_consensus.cc:2243] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.016876  5137 raft_consensus.cc:2272] T efd53179cce647948755bc8fc079cbba P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.019317  5137 tablet_replica.cc:333] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.019977  5137 raft_consensus.cc:2243] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.020499  5137 raft_consensus.cc:2272] T 9b0aeda8cdb248f2833025fda78b4f9c P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.022877  5137 tablet_replica.cc:333] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.023449  5137 raft_consensus.cc:2243] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.024049  5137 raft_consensus.cc:2272] T 1df2d8c357c44bc08e7478ac539f508a P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.027161  5137 tablet_replica.cc:333] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.027850  5137 raft_consensus.cc:2243] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:23.029152  5137 raft_consensus.cc:2272] T 922473629e704fbabcee362b44ed7409 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:23.030418 19537 consensus_peers.cc:597] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 -> Peer 051c8a7691c2489bb513f0e150a6089b (127.5.4.66:44361): Couldn't send request to peer 051c8a7691c2489bb513f0e150a6089b. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:44361: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:23.031926  5137 tablet_replica.cc:333] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.032667  5137 raft_consensus.cc:2243] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.033208  5137 raft_consensus.cc:2272] T 9f21d676c01f48bab23326a671a9c6b1 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.035801  5137 tablet_replica.cc:333] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.036399  5137 raft_consensus.cc:2243] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.036917  5137 raft_consensus.cc:2272] T 013fb25300cb44e6ac3d9a1ece427992 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.039383  5137 tablet_replica.cc:333] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.039955  5137 raft_consensus.cc:2243] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.040545  5137 raft_consensus.cc:2272] T a5d7d0da151049a08e785e94683b05b3 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.042972  5137 tablet_replica.cc:333] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.043756  5137 raft_consensus.cc:2243] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:23.044816  5137 raft_consensus.cc:2272] T 9e077c0962e947c6bd92d06245f244fd P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.047207  5137 tablet_replica.cc:333] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.047730  5137 raft_consensus.cc:2243] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.048257  5137 raft_consensus.cc:2272] T 959339bed78841e5b523d05b7cc25273 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.050479  5137 tablet_replica.cc:333] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.051011  5137 raft_consensus.cc:2243] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:23.051870  5137 raft_consensus.cc:2272] T 8b2f618923fa4bb9bde5f0acd9a47fb2 P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.054239  5137 tablet_replica.cc:333] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9: stopping tablet replica
I20260709 12:31:23.054826  5137 raft_consensus.cc:2243] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:23.055294  5137 raft_consensus.cc:2272] T 9372849a880b49e49e1753cc6f57c87e P a2feef6cf9664cc18c8cdb7e5a4719f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.102224  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:31:23.139497  5137 master.cc:562] Master@127.5.4.126:34849 shutting down...
I20260709 12:31:23.170665  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:23.171481  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:23.171871  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P c2357f374958434d988d6958ee7a75ab: stopping tablet replica
I20260709 12:31:23.195739  5137 master.cc:584] Master@127.5.4.126:34849 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21701 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:31:23.257490  5137 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.5.4.126:34231
I20260709 12:31:23.258960  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:23.265441 19939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:23.265753 19938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:23.266600 19941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.268445  5137 server_base.cc:1061] running on GCE node
I20260709 12:31:23.269519  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:23.269704  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:23.269910  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600283269877 us; error 0 us; skew 500 ppm
I20260709 12:31:23.270462  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:23.273033  5137 webserver.cc:533] Webserver started at http://127.5.4.126:43917/ using document root <none> and password file <none>
I20260709 12:31:23.273516  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:23.273715  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:23.274009  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:23.275219  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/master-0-root/instance:
uuid: "711e6333830a4d7f9b0331814a98eb76"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.279692  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:31:23.283067 19946 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.283805  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:31:23.284101  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/master-0-root
uuid: "711e6333830a4d7f9b0331814a98eb76"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.284376  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:23.308033  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:23.309283  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:23.350558  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.126:34231
I20260709 12:31:23.350700 19997 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.126:34231 every 8 connection(s)
I20260709 12:31:23.354846 19998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:23.367503 19998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76: Bootstrap starting.
I20260709 12:31:23.373057 19998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:23.378137 19998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76: No bootstrap required, opened a new log
I20260709 12:31:23.380900 19998 raft_consensus.cc:359] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711e6333830a4d7f9b0331814a98eb76" member_type: VOTER }
I20260709 12:31:23.381443 19998 raft_consensus.cc:385] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:23.381692 19998 raft_consensus.cc:740] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 711e6333830a4d7f9b0331814a98eb76, State: Initialized, Role: FOLLOWER
I20260709 12:31:23.382416 19998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711e6333830a4d7f9b0331814a98eb76" member_type: VOTER }
I20260709 12:31:23.383091 19998 raft_consensus.cc:399] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:31:23.383373 19998 raft_consensus.cc:493] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:31:23.383688 19998 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:23.389586 19998 raft_consensus.cc:515] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711e6333830a4d7f9b0331814a98eb76" member_type: VOTER }
I20260709 12:31:23.390197 19998 leader_election.cc:304] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 711e6333830a4d7f9b0331814a98eb76; no voters: 
I20260709 12:31:23.391532 19998 leader_election.cc:290] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:31:23.392002 20001 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:23.393576 20001 raft_consensus.cc:697] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 1 LEADER]: Becoming Leader. State: Replica: 711e6333830a4d7f9b0331814a98eb76, State: Running, Role: LEADER
I20260709 12:31:23.394382 20001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711e6333830a4d7f9b0331814a98eb76" member_type: VOTER }
I20260709 12:31:23.395244 19998 sys_catalog.cc:565] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:31:23.399699 20003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 711e6333830a4d7f9b0331814a98eb76. Latest consensus state: current_term: 1 leader_uuid: "711e6333830a4d7f9b0331814a98eb76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711e6333830a4d7f9b0331814a98eb76" member_type: VOTER } }
I20260709 12:31:23.399900 20002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "711e6333830a4d7f9b0331814a98eb76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "711e6333830a4d7f9b0331814a98eb76" member_type: VOTER } }
I20260709 12:31:23.400599 20003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:23.400982 20002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [sys.catalog]: This master's current role is: LEADER
I20260709 12:31:23.404624 20007 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:31:23.412225 20007 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:31:23.413899  5137 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:31:23.422874 20007 catalog_manager.cc:1383] Generated new cluster ID: e675b04766f1415c9815a4ce9932593b
I20260709 12:31:23.423241 20007 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:31:23.446318 20007 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:31:23.447952 20007 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:31:23.467088 20007 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76: Generated new TSK 0
I20260709 12:31:23.467823 20007 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:31:23.480556  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:23.487071 20019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:23.488514 20020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.490597  5137 server_base.cc:1061] running on GCE node
W20260709 12:31:23.490805 20022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.491952  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:23.492208  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:23.492403  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600283492375 us; error 0 us; skew 500 ppm
I20260709 12:31:23.492959  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:23.495601  5137 webserver.cc:533] Webserver started at http://127.5.4.65:40013/ using document root <none> and password file <none>
I20260709 12:31:23.496111  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:23.496337  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:23.496621  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:23.497967  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-0-root/instance:
uuid: "9f1e938b2be54950b77fb71c531a1d34"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.502728  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:31:23.506342 20027 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.507383  5137 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:31:23.507782  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-0-root
uuid: "9f1e938b2be54950b77fb71c531a1d34"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.508112  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:23.527420  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:23.528641  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:23.530166  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:23.532617  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:23.532858  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.533116  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:23.533308  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.579875  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.65:39279
I20260709 12:31:23.580056 20089 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.65:39279 every 8 connection(s)
I20260709 12:31:23.585258  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:23.593051 20094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:23.593807 20095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.597420  5137 server_base.cc:1061] running on GCE node
W20260709 12:31:23.598173 20098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.599496  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:23.599792  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:23.600018  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600283599992 us; error 0 us; skew 500 ppm
I20260709 12:31:23.600642  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:23.600867 20090 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34231
I20260709 12:31:23.601365 20090 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:23.602283 20090 heartbeater.cc:507] Master 127.5.4.126:34231 requested a full tablet report, sending...
I20260709 12:31:23.603494  5137 webserver.cc:533] Webserver started at http://127.5.4.66:38245/ using document root <none> and password file <none>
I20260709 12:31:23.604151  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:23.604372  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:23.604669  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:23.604753 19963 ts_manager.cc:194] Registered new tserver with Master: 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279)
I20260709 12:31:23.606026  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-1-root/instance:
uuid: "301528e0176e4eff834c92ca750633a4"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.606830 19963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52448
I20260709 12:31:23.611584  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.003s
I20260709 12:31:23.615353 20102 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.616104  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:31:23.616432  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-1-root
uuid: "301528e0176e4eff834c92ca750633a4"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.616771  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:23.633556  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:23.634980  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:23.636556  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:23.638932  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:23.639171  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.639422  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:23.639605  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.685612  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.66:41919
I20260709 12:31:23.685802 20164 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.66:41919 every 8 connection(s)
I20260709 12:31:23.691254  5137 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:31:23.698375 20168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:31:23.699652 20169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.709321 20165 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34231
I20260709 12:31:23.709854 20165 heartbeater.cc:461] Registering TS with master...
W20260709 12:31:23.710237 20171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:31:23.710892 20165 heartbeater.cc:507] Master 127.5.4.126:34231 requested a full tablet report, sending...
I20260709 12:31:23.713093 19963 ts_manager.cc:194] Registered new tserver with Master: 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:23.714391 19963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52452
I20260709 12:31:23.716913  5137 server_base.cc:1061] running on GCE node
I20260709 12:31:23.718029  5137 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:31:23.718256  5137 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:31:23.718463  5137 hybrid_clock.cc:648] HybridClock initialized: now 1783600283718423 us; error 0 us; skew 500 ppm
I20260709 12:31:23.719090  5137 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:31:23.721982  5137 webserver.cc:533] Webserver started at http://127.5.4.67:38001/ using document root <none> and password file <none>
I20260709 12:31:23.722565  5137 fs_manager.cc:362] Metadata directory not provided
I20260709 12:31:23.722779  5137 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:31:23.723095  5137 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:31:23.724397  5137 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-2-root/instance:
uuid: "ffbb6758e9844e559889a1a5c44bcc22"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.729393  5137 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:31:23.733144 20176 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.734071  5137 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:31:23.734409  5137 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-2-root
uuid: "ffbb6758e9844e559889a1a5c44bcc22"
format_stamp: "Formatted at 2026-07-09 12:31:23 on dist-test-slave-lg8f"
I20260709 12:31:23.734804  5137 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskEyW0_v/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599805609796-5137-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:31:23.751410  5137 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:31:23.752767  5137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:31:23.754426  5137 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:31:23.756944  5137 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:31:23.757179  5137 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.757431  5137 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:31:23.757614  5137 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:31:23.808773  5137 rpc_server.cc:307] RPC server started. Bound to: 127.5.4.67:36159
I20260709 12:31:23.808890 20238 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.4.67:36159 every 8 connection(s)
I20260709 12:31:23.825181 20239 heartbeater.cc:344] Connected to a master server at 127.5.4.126:34231
I20260709 12:31:23.825712 20239 heartbeater.cc:461] Registering TS with master...
I20260709 12:31:23.826440 20239 heartbeater.cc:507] Master 127.5.4.126:34231 requested a full tablet report, sending...
I20260709 12:31:23.828526 19963 ts_manager.cc:194] Registered new tserver with Master: ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:23.828986  5137 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016117974s
I20260709 12:31:23.830291 19963 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52454
I20260709 12:31:23.865659 19963 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52470:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:23.868544 19963 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:23.945946 20055 tablet_service.cc:1511] Processing CreateTablet for tablet 258ca134b3c142f1a31f9927b6dad39c (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:23.946772 20054 tablet_service.cc:1511] Processing CreateTablet for tablet 990d8c3ee6ea4287be6ea7395d2f8a80 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:23.947495 20055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258ca134b3c142f1a31f9927b6dad39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:23.948197 20054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990d8c3ee6ea4287be6ea7395d2f8a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:23.951665 20053 tablet_service.cc:1511] Processing CreateTablet for tablet 606dc20b79e94d53bfd993f4d5ba55d3 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:23.953189 20053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 606dc20b79e94d53bfd993f4d5ba55d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:23.960357 20052 tablet_service.cc:1511] Processing CreateTablet for tablet 6b63e5939ed24b32a9a4999bff0a9fcf (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:23.962764 20052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b63e5939ed24b32a9a4999bff0a9fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:23.968457 20051 tablet_service.cc:1511] Processing CreateTablet for tablet a8a76362a29c4d9b8633412de6dec355 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:23.970513 20051 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a76362a29c4d9b8633412de6dec355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:23.994720 20050 tablet_service.cc:1511] Processing CreateTablet for tablet 7583a33a07df428ea67f04fe63b39531 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:23.996331 20050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7583a33a07df428ea67f04fe63b39531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.001124 20130 tablet_service.cc:1511] Processing CreateTablet for tablet 258ca134b3c142f1a31f9927b6dad39c (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:24.002640 20130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258ca134b3c142f1a31f9927b6dad39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.003825 20126 tablet_service.cc:1511] Processing CreateTablet for tablet a8a76362a29c4d9b8633412de6dec355 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:24.005410 20126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a76362a29c4d9b8633412de6dec355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.007642 20128 tablet_service.cc:1511] Processing CreateTablet for tablet 606dc20b79e94d53bfd993f4d5ba55d3 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:24.009191 20128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 606dc20b79e94d53bfd993f4d5ba55d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.011772 20127 tablet_service.cc:1511] Processing CreateTablet for tablet 6b63e5939ed24b32a9a4999bff0a9fcf (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:24.013173 20127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b63e5939ed24b32a9a4999bff0a9fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.015703 20129 tablet_service.cc:1511] Processing CreateTablet for tablet 990d8c3ee6ea4287be6ea7395d2f8a80 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:24.017086 20129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990d8c3ee6ea4287be6ea7395d2f8a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.028868 20204 tablet_service.cc:1511] Processing CreateTablet for tablet 258ca134b3c142f1a31f9927b6dad39c (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:24.031697 20203 tablet_service.cc:1511] Processing CreateTablet for tablet 990d8c3ee6ea4287be6ea7395d2f8a80 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:24.033965 20202 tablet_service.cc:1511] Processing CreateTablet for tablet 606dc20b79e94d53bfd993f4d5ba55d3 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:24.036320 20201 tablet_service.cc:1511] Processing CreateTablet for tablet 6b63e5939ed24b32a9a4999bff0a9fcf (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:24.038419 20200 tablet_service.cc:1511] Processing CreateTablet for tablet a8a76362a29c4d9b8633412de6dec355 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:24.030364 20204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 258ca134b3c142f1a31f9927b6dad39c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.042994 20200 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8a76362a29c4d9b8633412de6dec355. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.046830 20201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b63e5939ed24b32a9a4999bff0a9fcf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.050215 20130 tablet_service.cc:1511] Processing CreateTablet for tablet 7583a33a07df428ea67f04fe63b39531 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:24.051900 20199 tablet_service.cc:1511] Processing CreateTablet for tablet 7583a33a07df428ea67f04fe63b39531 (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:24.051909 20130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7583a33a07df428ea67f04fe63b39531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.053357 20199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7583a33a07df428ea67f04fe63b39531. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.056010 20203 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 990d8c3ee6ea4287be6ea7395d2f8a80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.058840 20202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 606dc20b79e94d53bfd993f4d5ba55d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.072582 20050 tablet_service.cc:1511] Processing CreateTablet for tablet 6c70cfe6a061403c91a1ccbdf236b4ed (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:24.074272 20050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c70cfe6a061403c91a1ccbdf236b4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.086969 20127 tablet_service.cc:1511] Processing CreateTablet for tablet 6c70cfe6a061403c91a1ccbdf236b4ed (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:24.088877 20127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c70cfe6a061403c91a1ccbdf236b4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.101974 20198 tablet_service.cc:1511] Processing CreateTablet for tablet 6c70cfe6a061403c91a1ccbdf236b4ed (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:24.103502 20198 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c70cfe6a061403c91a1ccbdf236b4ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.103300 20055 tablet_service.cc:1511] Processing CreateTablet for tablet bc53759e2a1e4085a643cb708d27e7ca (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:24.104815 20055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53759e2a1e4085a643cb708d27e7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.105904 20128 tablet_service.cc:1511] Processing CreateTablet for tablet bc53759e2a1e4085a643cb708d27e7ca (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:24.110271 20128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53759e2a1e4085a643cb708d27e7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.120810 20197 tablet_service.cc:1511] Processing CreateTablet for tablet bc53759e2a1e4085a643cb708d27e7ca (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:24.122279 20197 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53759e2a1e4085a643cb708d27e7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.122331 20053 tablet_service.cc:1511] Processing CreateTablet for tablet 4371078417de4d60a32baa99b7d2087a (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:24.123896 20053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4371078417de4d60a32baa99b7d2087a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.123708 20125 tablet_service.cc:1511] Processing CreateTablet for tablet 4371078417de4d60a32baa99b7d2087a (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:24.125560 20125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4371078417de4d60a32baa99b7d2087a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.132807 20202 tablet_service.cc:1511] Processing CreateTablet for tablet 4371078417de4d60a32baa99b7d2087a (DEFAULT_TABLE table=test-workload [id=7e10e49613cd4833b3c8672afb9c8b02]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:24.134600 20202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4371078417de4d60a32baa99b7d2087a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:24.151320 20256 tablet_bootstrap.cc:492] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.177167 20254 tablet_bootstrap.cc:492] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.184554 20254 tablet_bootstrap.cc:654] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.187978 20256 tablet_bootstrap.cc:654] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.204397 20255 tablet_bootstrap.cc:492] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.216250 20254 tablet_bootstrap.cc:492] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.218945 20254 ts_tablet_manager.cc:1403] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.042s	user 0.012s	sys 0.024s
I20260709 12:31:24.221179 20256 tablet_bootstrap.cc:492] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.221328 20255 tablet_bootstrap.cc:654] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.221781 20256 ts_tablet_manager.cc:1403] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.071s	user 0.008s	sys 0.006s
I20260709 12:31:24.222396 20254 raft_consensus.cc:359] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.223344 20254 raft_consensus.cc:385] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.223693 20254 raft_consensus.cc:740] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.224483 20256 raft_consensus.cc:359] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.224539 20254 consensus_queue.cc:260] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.225533 20256 raft_consensus.cc:385] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.225936 20256 raft_consensus.cc:740] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.226711 20256 consensus_queue.cc:260] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.227473 20090 heartbeater.cc:499] Master 127.5.4.126:34231 was elected leader, sending a full tablet report...
I20260709 12:31:24.227674 20255 tablet_bootstrap.cc:492] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.228130 20254 ts_tablet_manager.cc:1434] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:31:24.228292 20255 ts_tablet_manager.cc:1403] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.001s
I20260709 12:31:24.229326 20254 tablet_bootstrap.cc:492] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.231599 20255 raft_consensus.cc:359] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.232501 20255 raft_consensus.cc:385] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.234697 20239 heartbeater.cc:499] Master 127.5.4.126:34231 was elected leader, sending a full tablet report...
I20260709 12:31:24.235222 20256 ts_tablet_manager.cc:1434] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.013s	user 0.014s	sys 0.000s
I20260709 12:31:24.236464 20256 tablet_bootstrap.cc:492] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.236945 20254 tablet_bootstrap.cc:654] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.242687 20256 tablet_bootstrap.cc:654] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.232913 20255 raft_consensus.cc:740] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.249963 20256 tablet_bootstrap.cc:492] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.250798 20256 ts_tablet_manager.cc:1403] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.003s
I20260709 12:31:24.249675 20255 consensus_queue.cc:260] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.255473 20256 raft_consensus.cc:359] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.256526 20256 raft_consensus.cc:385] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.256801 20256 raft_consensus.cc:740] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.257791 20256 consensus_queue.cc:260] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.258927 20254 tablet_bootstrap.cc:492] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.259579 20254 ts_tablet_manager.cc:1403] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.015s
I20260709 12:31:24.260573 20256 ts_tablet_manager.cc:1434] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 12:31:24.261770 20256 tablet_bootstrap.cc:492] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.262914 20254 raft_consensus.cc:359] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.263717 20254 raft_consensus.cc:385] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.264070 20254 raft_consensus.cc:740] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.265071 20254 consensus_queue.cc:260] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.267782 20254 ts_tablet_manager.cc:1434] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.008s	user 0.000s	sys 0.006s
I20260709 12:31:24.268886 20254 tablet_bootstrap.cc:492] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.270010 20256 tablet_bootstrap.cc:654] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.276727 20254 tablet_bootstrap.cc:654] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.283223 20256 tablet_bootstrap.cc:492] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.284024 20256 ts_tablet_manager.cc:1403] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.022s	user 0.007s	sys 0.013s
I20260709 12:31:24.288636 20256 raft_consensus.cc:359] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.289748 20256 raft_consensus.cc:385] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.290187 20256 raft_consensus.cc:740] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.291507 20256 consensus_queue.cc:260] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.297996 20256 ts_tablet_manager.cc:1434] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.002s
I20260709 12:31:24.300594 20256 tablet_bootstrap.cc:492] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.301338 20254 tablet_bootstrap.cc:492] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.302042 20254 ts_tablet_manager.cc:1403] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.033s	user 0.020s	sys 0.012s
I20260709 12:31:24.305414 20254 raft_consensus.cc:359] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.306250 20254 raft_consensus.cc:385] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.306633 20254 raft_consensus.cc:740] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.307842 20256 tablet_bootstrap.cc:654] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.307618 20254 consensus_queue.cc:260] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.310703 20254 ts_tablet_manager.cc:1434] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:31:24.311990 20254 tablet_bootstrap.cc:492] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
W20260709 12:31:24.313984 20240 tablet.cc:2406] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:31:24.320156 20254 tablet_bootstrap.cc:654] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.323778 20256 tablet_bootstrap.cc:492] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.324455 20256 ts_tablet_manager.cc:1403] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.008s
I20260709 12:31:24.325587 20165 heartbeater.cc:499] Master 127.5.4.126:34231 was elected leader, sending a full tablet report...
I20260709 12:31:24.326210 20255 ts_tablet_manager.cc:1434] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.098s	user 0.039s	sys 0.039s
I20260709 12:31:24.327486 20255 tablet_bootstrap.cc:492] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.328706 20256 raft_consensus.cc:359] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.330052 20256 raft_consensus.cc:385] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.330554 20256 raft_consensus.cc:740] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.331702 20256 consensus_queue.cc:260] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.334955 20256 ts_tablet_manager.cc:1434] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:31:24.336108 20256 tablet_bootstrap.cc:492] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.339053 20254 tablet_bootstrap.cc:492] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.339769 20254 ts_tablet_manager.cc:1403] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.028s	user 0.023s	sys 0.002s
I20260709 12:31:24.343926 20255 tablet_bootstrap.cc:654] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.344383 20256 tablet_bootstrap.cc:654] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.344520 20260 raft_consensus.cc:493] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.345099 20260 raft_consensus.cc:515] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.354450 20260 leader_election.cc:290] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.356455 20255 tablet_bootstrap.cc:492] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.357236 20255 ts_tablet_manager.cc:1403] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.008s
I20260709 12:31:24.343358 20254 raft_consensus.cc:359] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.359735 20254 raft_consensus.cc:385] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.360689 20256 tablet_bootstrap.cc:492] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.360615 20254 raft_consensus.cc:740] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.361321 20256 ts_tablet_manager.cc:1403] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.004s
I20260709 12:31:24.362746 20254 consensus_queue.cc:260] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.364243 20255 raft_consensus.cc:359] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.368971 20255 raft_consensus.cc:385] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.365417 20256 raft_consensus.cc:359] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.369326 20255 raft_consensus.cc:740] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.369566 20256 raft_consensus.cc:385] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.369349 20260 raft_consensus.cc:493] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.370601 20255 consensus_queue.cc:260] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.370205 20256 raft_consensus.cc:740] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.370800 20260 raft_consensus.cc:515] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.375803 20260 leader_election.cc:290] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.373148 20256 consensus_queue.cc:260] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.376914 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.378194 20255 ts_tablet_manager.cc:1434] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:31:24.379220 20256 ts_tablet_manager.cc:1434] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.001s
I20260709 12:31:24.380628 20256 tablet_bootstrap.cc:492] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.380555 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258ca134b3c142f1a31f9927b6dad39c" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.381352 20140 raft_consensus.cc:2468] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
W20260709 12:31:24.383136 20031 leader_election.cc:343] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.384116 20031 leader_election.cc:304] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:24.383193 20255 tablet_bootstrap.cc:492] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.389019 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.389829 20214 raft_consensus.cc:2468] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:24.391157 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258ca134b3c142f1a31f9927b6dad39c" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.392627 20255 tablet_bootstrap.cc:654] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.392879 20260 raft_consensus.cc:2804] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.393013 20028 leader_election.cc:304] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4
I20260709 12:31:24.392627 20256 tablet_bootstrap.cc:654] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.393388 20260 raft_consensus.cc:493] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:31:24.394374 20028 leader_election.cc:343] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.394780 20260 raft_consensus.cc:3060] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.398103 20254 ts_tablet_manager.cc:1434] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.058s	user 0.004s	sys 0.003s
I20260709 12:31:24.399139 20273 raft_consensus.cc:2804] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.399626 20273 raft_consensus.cc:493] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.399991 20273 raft_consensus.cc:3060] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.403728 20254 tablet_bootstrap.cc:492] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.403825 20260 raft_consensus.cc:515] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.405728 20255 tablet_bootstrap.cc:492] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.406571 20260 leader_election.cc:290] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.407575 20256 tablet_bootstrap.cc:492] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.407445 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258ca134b3c142f1a31f9927b6dad39c" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.408257 20256 ts_tablet_manager.cc:1403] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.004s
I20260709 12:31:24.408226 20140 raft_consensus.cc:3060] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:31:24.409296 20011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:24.410226 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "258ca134b3c142f1a31f9927b6dad39c" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
W20260709 12:31:24.411727 20028 leader_election.cc:343] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.412039 20256 raft_consensus.cc:359] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.412880 20256 raft_consensus.cc:385] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.413192 20256 raft_consensus.cc:740] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.414053 20256 consensus_queue.cc:260] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.414640 20255 ts_tablet_manager.cc:1403] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
I20260709 12:31:24.417953 20140 raft_consensus.cc:2468] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:24.419140 20255 raft_consensus.cc:359] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.419725 20256 ts_tablet_manager.cc:1434] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.007s
I20260709 12:31:24.419760 20031 leader_election.cc:304] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:24.420197 20255 raft_consensus.cc:385] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.420625 20255 raft_consensus.cc:740] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.421149 20256 tablet_bootstrap.cc:492] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.421823 20260 raft_consensus.cc:2804] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.421402 20255 consensus_queue.cc:260] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.424088 20255 ts_tablet_manager.cc:1434] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:31:24.425491 20255 tablet_bootstrap.cc:492] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.428718 20260 raft_consensus.cc:697] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:24.430476 20256 tablet_bootstrap.cc:654] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.429929 20260 consensus_queue.cc:237] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.435607 20254 tablet_bootstrap.cc:654] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.443754 20256 tablet_bootstrap.cc:492] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.444793 20256 ts_tablet_manager.cc:1403] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.024s	user 0.018s	sys 0.004s
I20260709 12:31:24.445268 20255 tablet_bootstrap.cc:654] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.445880 20264 raft_consensus.cc:493] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.439707 20273 raft_consensus.cc:515] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.446480 20264 raft_consensus.cc:515] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.449753 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:24.450462 20213 raft_consensus.cc:3060] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.458350 20261 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.458781 20264 leader_election.cc:290] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.459106 20261 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.460304 20255 tablet_bootstrap.cc:492] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.461191 20255 ts_tablet_manager.cc:1403] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.005s
I20260709 12:31:24.465516 20255 raft_consensus.cc:359] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.466698 20255 raft_consensus.cc:385] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.467182 20255 raft_consensus.cc:740] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.468297 20255 consensus_queue.cc:260] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.469962 20261 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.474664 20255 ts_tablet_manager.cc:1434] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.013s	user 0.012s	sys 0.001s
I20260709 12:31:24.476303 20255 tablet_bootstrap.cc:492] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.485495 20255 tablet_bootstrap.cc:654] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.494931 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.495687 20140 raft_consensus.cc:3060] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.497476 20273 leader_election.cc:290] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.498656 20255 tablet_bootstrap.cc:492] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.499661 20255 ts_tablet_manager.cc:1403] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.011s
I20260709 12:31:24.502043 20213 raft_consensus.cc:2468] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:24.504333 20255 raft_consensus.cc:359] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.505609 20255 raft_consensus.cc:385] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.506100 20255 raft_consensus.cc:740] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.504943 20140 raft_consensus.cc:2468] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:24.510350 20031 leader_election.cc:304] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:24.506141 20256 raft_consensus.cc:359] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.511363 20256 raft_consensus.cc:385] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.511924 20273 raft_consensus.cc:2804] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.508781 20255 consensus_queue.cc:260] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.513660 20273 raft_consensus.cc:697] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:24.515436 20273 consensus_queue.cc:237] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.518388 20261 raft_consensus.cc:493] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.519137 20261 raft_consensus.cc:515] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.520784 20255 ts_tablet_manager.cc:1434] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.003s
I20260709 12:31:24.525098 20255 tablet_bootstrap.cc:492] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.525314 20264 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.525954 20264 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.528215 20264 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.523286 20261 leader_election.cc:290] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.530740 19963 catalog_manager.cc:5697] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:24.512840 20256 raft_consensus.cc:740] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.532920 20255 tablet_bootstrap.cc:654] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.533334 20256 consensus_queue.cc:260] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.536468 20256 ts_tablet_manager.cc:1434] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.091s	user 0.008s	sys 0.000s
I20260709 12:31:24.538326 20256 tablet_bootstrap.cc:492] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:24.551653 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.552508 20139 raft_consensus.cc:2468] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:24.554066 20180 leader_election.cc:304] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.554837 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.555986 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.555116 20261 raft_consensus.cc:2804] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.556665 20214 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:24.557375 20261 raft_consensus.cc:493] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.558023 20261 raft_consensus.cc:3060] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.558882 20103 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.560055 20264 raft_consensus.cc:2804] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.560452 20264 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.560798 20264 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.568852 20261 raft_consensus.cc:515] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.570351 20213 raft_consensus.cc:2393] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 301528e0176e4eff834c92ca750633a4 in current term 1: Already voted for candidate ffbb6758e9844e559889a1a5c44bcc22 in this term.
I20260709 12:31:24.574877 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
W20260709 12:31:24.576431 20180 leader_election.cc:343] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.580385 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.581203 20140 raft_consensus.cc:3060] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.581576 20264 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.577993 20261 leader_election.cc:290] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.586436 20254 tablet_bootstrap.cc:492] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.587074 20254 ts_tablet_manager.cc:1403] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.184s	user 0.040s	sys 0.043s
I20260709 12:31:24.588790 20264 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.590143 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:24.591041 20213 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.598438 20255 tablet_bootstrap.cc:492] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.590422 20254 raft_consensus.cc:359] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.598686 20213 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:24.599022 20255 ts_tablet_manager.cc:1403] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.074s	user 0.023s	sys 0.031s
I20260709 12:31:24.599099 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.601217 20064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.602216 20064 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:24.602959 20255 raft_consensus.cc:359] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.603778 20255 raft_consensus.cc:385] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.598929 20254 raft_consensus.cc:385] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.604108 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:24.605134 20063 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.596567 20256 tablet_bootstrap.cc:654] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.605746 20103 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.604072 20255 raft_consensus.cc:740] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.607584 20255 consensus_queue.cc:260] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.604388 20254 raft_consensus.cc:740] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.609283 20062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.610730 20064 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.610772 20061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:24.616178 20140 raft_consensus.cc:2468] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:24.590231 20261 raft_consensus.cc:493] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:31:24.617082 20178 leader_election.cc:343] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.618145 20178 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:24.618517 20180 leader_election.cc:304] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.618039 20261 raft_consensus.cc:515] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.611506 20264 raft_consensus.cc:2804] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.620116 20254 consensus_queue.cc:260] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.626875 20261 leader_election.cc:290] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.627285 20254 ts_tablet_manager.cc:1434] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.001s
I20260709 12:31:24.627399 20255 ts_tablet_manager.cc:1434] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.005s
I20260709 12:31:24.627645 20261 raft_consensus.cc:2749] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:24.629552 20264 raft_consensus.cc:697] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:24.628854 20065 raft_consensus.cc:2468] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:24.630766 20261 raft_consensus.cc:2804] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.632164 20255 tablet_bootstrap.cc:492] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.632779 20063 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:24.633249 20104 leader_election.cc:304] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:24.631392 20261 raft_consensus.cc:697] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:24.635769 20264 consensus_queue.cc:237] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.640311 20254 tablet_bootstrap.cc:492] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.639119 20273 raft_consensus.cc:493] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.640882 20273 raft_consensus.cc:515] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.642448 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.643082 20273 leader_election.cc:290] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.639763 20292 raft_consensus.cc:2764] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:31:24.638835 20256 tablet_bootstrap.cc:492] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.637635 20261 consensus_queue.cc:237] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.644284 20256 ts_tablet_manager.cc:1403] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.106s	user 0.016s	sys 0.006s
I20260709 12:31:24.643517 20140 raft_consensus.cc:2468] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:24.646163 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.647716 20139 raft_consensus.cc:2468] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:24.648429 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.649794 20031 leader_election.cc:304] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:24.649899 20256 raft_consensus.cc:359] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
W20260709 12:31:24.650380 20028 leader_election.cc:343] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.650853 20273 raft_consensus.cc:2804] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.651274 20273 raft_consensus.cc:493] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.651469 20256 raft_consensus.cc:385] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.651942 20256 raft_consensus.cc:740] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.652948 20256 consensus_queue.cc:260] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.653187 20180 leader_election.cc:304] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.651815 20273 raft_consensus.cc:3060] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.655122 20295 raft_consensus.cc:2804] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.655874 20256 ts_tablet_manager.cc:1434] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:31:24.655191 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.657828 20256 tablet_bootstrap.cc:492] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
W20260709 12:31:24.661345 20178 leader_election.cc:343] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.663933 20255 tablet_bootstrap.cc:654] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.656927 20295 raft_consensus.cc:493] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.665323 20273 raft_consensus.cc:515] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.677824 20254 tablet_bootstrap.cc:654] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.679664 20273 leader_election.cc:290] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.668788 20295 raft_consensus.cc:3060] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.679418 19960 catalog_manager.cc:5697] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:24.669189 19962 catalog_manager.cc:5697] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:24.682116 20256 tablet_bootstrap.cc:654] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.680454 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.684110 20139 raft_consensus.cc:3060] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.684484 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
W20260709 12:31:24.689643 20028 leader_election.cc:343] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.690307 20295 raft_consensus.cc:515] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.693390 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
W20260709 12:31:24.695271 20178 leader_election.cc:343] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.696399 20295 leader_election.cc:290] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.699229 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.699901 20140 raft_consensus.cc:3060] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.702580 19963 catalog_manager.cc:5697] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:24.707733 20255 tablet_bootstrap.cc:492] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.708452 20255 ts_tablet_manager.cc:1403] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.077s	user 0.019s	sys 0.007s
I20260709 12:31:24.712397 20255 raft_consensus.cc:359] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.713256 20255 raft_consensus.cc:385] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.713598 20255 raft_consensus.cc:740] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.714774 20255 consensus_queue.cc:260] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.716441 20139 raft_consensus.cc:2468] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:24.719269 20255 ts_tablet_manager.cc:1434] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.002s
I20260709 12:31:24.720402 20255 tablet_bootstrap.cc:492] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.720999 20256 tablet_bootstrap.cc:492] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:24.720638 20031 leader_election.cc:304] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:24.721637 20256 ts_tablet_manager.cc:1403] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.064s	user 0.020s	sys 0.013s
I20260709 12:31:24.722625 20273 raft_consensus.cc:2804] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.723433 20273 raft_consensus.cc:697] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:24.724617 20273 consensus_queue.cc:237] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.728819 20140 raft_consensus.cc:2468] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:24.724560 20256 raft_consensus.cc:359] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.731968 20180 leader_election.cc:304] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:24.734002 20295 raft_consensus.cc:2804] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.734651 20295 raft_consensus.cc:697] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:24.735280 20254 tablet_bootstrap.cc:492] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.735391 20255 tablet_bootstrap.cc:654] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.735930 20254 ts_tablet_manager.cc:1403] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.096s	user 0.028s	sys 0.010s
I20260709 12:31:24.735474 20295 consensus_queue.cc:237] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.741163 20256 raft_consensus.cc:385] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.740738 20254 raft_consensus.cc:359] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.741936 20256 raft_consensus.cc:740] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.742312 20254 raft_consensus.cc:385] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.742920 20254 raft_consensus.cc:740] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.743748 20256 consensus_queue.cc:260] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.744318 20254 consensus_queue.cc:260] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.745752 19963 catalog_manager.cc:5697] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:24.747517 20256 ts_tablet_manager.cc:1434] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.000s
I20260709 12:31:24.752771 20261 raft_consensus.cc:493] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.753304 20261 raft_consensus.cc:515] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.756311 20254 ts_tablet_manager.cc:1434] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.004s
I20260709 12:31:24.757025 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.757452 20254 tablet_bootstrap.cc:492] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.757901 20140 raft_consensus.cc:2393] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ffbb6758e9844e559889a1a5c44bcc22 in current term 1: Already voted for candidate 9f1e938b2be54950b77fb71c531a1d34 in this term.
I20260709 12:31:24.760331 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.763098 20178 leader_election.cc:304] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:24.764103 20261 leader_election.cc:290] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.764940 20261 raft_consensus.cc:3060] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.767760 20255 tablet_bootstrap.cc:492] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.768498 20255 ts_tablet_manager.cc:1403] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.048s	user 0.013s	sys 0.023s
I20260709 12:31:24.769105 20254 tablet_bootstrap.cc:654] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.771852 20255 raft_consensus.cc:359] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.772917 20255 raft_consensus.cc:385] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.773393 20255 raft_consensus.cc:740] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.773461 20261 raft_consensus.cc:2749] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:24.774580 19962 catalog_manager.cc:5697] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:24.774732 20255 consensus_queue.cc:260] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.778167 20255 ts_tablet_manager.cc:1434] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:31:24.779495 20255 tablet_bootstrap.cc:492] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:24.781718 20261 raft_consensus.cc:493] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.782249 20261 raft_consensus.cc:515] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.784611 20261 leader_election.cc:290] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.785842 20255 tablet_bootstrap.cc:654] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.787160 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.788405 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
W20260709 12:31:24.788595 20178 leader_election.cc:343] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.789351 20140 raft_consensus.cc:2468] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:24.790743 20180 leader_election.cc:304] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:24.792927 20261 raft_consensus.cc:2804] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.793040 20254 tablet_bootstrap.cc:492] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.794436 20261 raft_consensus.cc:493] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.794950 20261 raft_consensus.cc:3060] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.795084 20254 ts_tablet_manager.cc:1403] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.010s
I20260709 12:31:24.797119 20255 tablet_bootstrap.cc:492] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:24.797820 20255 ts_tablet_manager.cc:1403] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.019s	user 0.010s	sys 0.006s
I20260709 12:31:24.801185 20255 raft_consensus.cc:359] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.801774 20255 raft_consensus.cc:385] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.802337 20255 raft_consensus.cc:740] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.801693 20254 raft_consensus.cc:359] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.802893 20254 raft_consensus.cc:385] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.802533 20261 raft_consensus.cc:515] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.803233 20254 raft_consensus.cc:740] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.803223 20255 consensus_queue.cc:260] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.804486 20254 consensus_queue.cc:260] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.805135 20261 leader_election.cc:290] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.806034 20255 ts_tablet_manager.cc:1434] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:31:24.807003 20254 ts_tablet_manager.cc:1434] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:31:24.808274 20254 tablet_bootstrap.cc:492] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.808071 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.809262 20140 raft_consensus.cc:3060] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.809804 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
W20260709 12:31:24.812148 20178 leader_election.cc:343] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:24.815554 20254 tablet_bootstrap.cc:654] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.819221 20140 raft_consensus.cc:2468] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:24.820492 20180 leader_election.cc:304] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:24.821810 20261 raft_consensus.cc:2804] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.822083 20254 tablet_bootstrap.cc:492] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.822798 20254 ts_tablet_manager.cc:1403] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.004s
I20260709 12:31:24.823202 20261 raft_consensus.cc:697] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:24.824182 20261 consensus_queue.cc:237] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.825939 20254 raft_consensus.cc:359] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.826380 20213 raft_consensus.cc:3060] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.826689 20254 raft_consensus.cc:385] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.827239 20254 raft_consensus.cc:740] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.827955 20254 consensus_queue.cc:260] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.830029 20254 ts_tablet_manager.cc:1434] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:31:24.831355 20254 tablet_bootstrap.cc:492] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:24.838042 20254 tablet_bootstrap.cc:654] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:24.838660 20273 consensus_queue.cc:1048] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:24.838148 19960 catalog_manager.cc:5697] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:24.846418 20261 raft_consensus.cc:493] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.847304 20261 raft_consensus.cc:515] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.849781 20264 raft_consensus.cc:493] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.850378 20264 raft_consensus.cc:515] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.851485 20261 leader_election.cc:290] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.852178 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.852713 20264 leader_election.cc:290] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.852864 20063 raft_consensus.cc:2468] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:24.855023 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:24.855382 20178 leader_election.cc:304] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.855808 20065 raft_consensus.cc:2468] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:24.856431 20261 raft_consensus.cc:2804] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.856508 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.857012 20261 raft_consensus.cc:493] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.857445 20261 raft_consensus.cc:3060] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.858559 20104 leader_election.cc:304] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:24.860107 20254 tablet_bootstrap.cc:492] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:24.860715 20254 ts_tablet_manager.cc:1403] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.005s
I20260709 12:31:24.860991 20264 raft_consensus.cc:2804] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.855291 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.862142 20140 raft_consensus.cc:2468] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:24.864049 20264 raft_consensus.cc:493] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.863914 20254 raft_consensus.cc:359] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.864563 20264 raft_consensus.cc:3060] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.864758 20254 raft_consensus.cc:385] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:24.865110 20254 raft_consensus.cc:740] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:24.865890 20254 consensus_queue.cc:260] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.868062 20254 ts_tablet_manager.cc:1434] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:31:24.873898 20261 raft_consensus.cc:515] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.875420 20214 raft_consensus.cc:2393] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 301528e0176e4eff834c92ca750633a4 in current term 1: Already voted for candidate ffbb6758e9844e559889a1a5c44bcc22 in this term.
I20260709 12:31:24.879220 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:24.880098 20065 raft_consensus.cc:3060] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.880146 20264 raft_consensus.cc:515] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.884833 20276 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.885525 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:24.886080 20261 leader_election.cc:290] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:24.887640 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.888514 20140 raft_consensus.cc:2393] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ffbb6758e9844e559889a1a5c44bcc22 in current term 1: Already voted for candidate 301528e0176e4eff834c92ca750633a4 in this term.
I20260709 12:31:24.885478 20276 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.890938 20065 raft_consensus.cc:2468] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:24.891551 20276 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.894395 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.895004 20178 leader_election.cc:304] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4
I20260709 12:31:24.895543 20139 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:24.895767 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.896406 20261 raft_consensus.cc:2804] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.896495 20214 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:24.897075 20261 raft_consensus.cc:697] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:24.898656 20031 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:24.898583 20261 consensus_queue.cc:237] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.899700 20273 consensus_queue.cc:1048] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:24.901249 20273 raft_consensus.cc:2804] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:24.901590 20273 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:24.901935 20273 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.912420 20273 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.921576 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:24.915035 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:24.918097 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:24.924574 20103 leader_election.cc:304] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 301528e0176e4eff834c92ca750633a4; no voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:24.925856 20292 raft_consensus.cc:2749] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:31:24.925210 19960 catalog_manager.cc:5697] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:24.925253 20140 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.915753 20264 leader_election.cc:290] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.915663 20273 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.929419 20214 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.926301 20276 consensus_queue.cc:1048] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:31:24.932909 20276 raft_consensus.cc:493] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.933370 20276 raft_consensus.cc:515] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.935312 20276 leader_election.cc:290] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.936652 20137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.937551 20137 raft_consensus.cc:2393] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f1e938b2be54950b77fb71c531a1d34 in current term 1: Already voted for candidate ffbb6758e9844e559889a1a5c44bcc22 in this term.
I20260709 12:31:24.941532 20214 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:24.943346 20028 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:24.944245 20276 raft_consensus.cc:2804] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:24.944725 20276 raft_consensus.cc:697] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:24.945551 20276 consensus_queue.cc:237] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.951013 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.954088 20028 leader_election.cc:304] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34; no voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:24.953544 20140 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:24.957895 20313 raft_consensus.cc:3060] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.958464 20273 consensus_queue.cc:1048] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:24.969774 19962 catalog_manager.cc:5697] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:24.970983 20273 raft_consensus.cc:493] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:24.971594 20273 raft_consensus.cc:515] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:24.973970 20313 raft_consensus.cc:2749] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:24.974655 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:24.975088 20273 leader_election.cc:290] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:24.975538 20140 raft_consensus.cc:2393] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9f1e938b2be54950b77fb71c531a1d34 in current term 1: Already voted for candidate ffbb6758e9844e559889a1a5c44bcc22 in this term.
I20260709 12:31:24.976097 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:24.977543 20028 leader_election.cc:304] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34; no voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:24.978271 20273 raft_consensus.cc:3060] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:24.985313 20273 raft_consensus.cc:2749] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:25.059315 20264 consensus_queue.cc:1048] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:25.071602 20294 consensus_queue.cc:1048] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:25.110674 20213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
mode: GRACEFUL
new_leader_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:53968
I20260709 12:31:25.111248 20213 raft_consensus.cc:606] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Received request to transfer leadership to TS 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:25.123085 20213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
mode: GRACEFUL
new_leader_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:53968
I20260709 12:31:25.123598 20213 raft_consensus.cc:606] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Received request to transfer leadership to TS 301528e0176e4eff834c92ca750633a4
I20260709 12:31:25.140722 20065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
mode: GRACEFUL
new_leader_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:31:25.141215 20065 raft_consensus.cc:606] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Received request to transfer leadership to TS 301528e0176e4eff834c92ca750633a4
I20260709 12:31:25.146435 20213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
mode: GRACEFUL
new_leader_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:53968
I20260709 12:31:25.146905 20213 raft_consensus.cc:606] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Received request to transfer leadership to TS 301528e0176e4eff834c92ca750633a4
I20260709 12:31:25.220047 20261 consensus_queue.cc:1048] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.227192 20295 raft_consensus.cc:993] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: : Instructing follower 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:25.227679 20295 raft_consensus.cc:1081] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Signalling peer 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:25.229483 20139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:56212
I20260709 12:31:25.231341 20065 raft_consensus.cc:3060] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:25.232463 20139 raft_consensus.cc:493] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.233047 20139 raft_consensus.cc:3060] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.237428 20295 consensus_queue.cc:1048] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.240578 20295 consensus_queue.cc:1048] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.242349 20139 raft_consensus.cc:515] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.249415 20139 leader_election.cc:290] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:25.251411 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:25.251695 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:25.252292 20065 raft_consensus.cc:3060] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.252468 20213 raft_consensus.cc:3055] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:25.253005 20213 raft_consensus.cc:740] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:25.254002 20213 consensus_queue.cc:260] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.255396 20213 raft_consensus.cc:3060] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.256093 20293 consensus_queue.cc:1048] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.265015 20213 raft_consensus.cc:2468] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:25.265247 20065 raft_consensus.cc:2468] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:25.266408 20103 leader_election.cc:304] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:25.267537 20294 raft_consensus.cc:2804] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:25.268067 20294 raft_consensus.cc:697] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:25.268811 20294 consensus_queue.cc:237] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.272343 20261 raft_consensus.cc:993] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: : Instructing follower 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:25.273025 20295 raft_consensus.cc:1081] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Signalling peer 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:25.274948 20063 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:53240
I20260709 12:31:25.275578 20063 raft_consensus.cc:493] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.276000 20063 raft_consensus.cc:3060] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.281615 19962 catalog_manager.cc:5697] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 1 to 2, leader changed from ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67) to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:25.283274 20063 raft_consensus.cc:515] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.285506 20063 leader_election.cc:290] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:25.286386 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:25.287078 20139 raft_consensus.cc:3060] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.288309 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "990d8c3ee6ea4287be6ea7395d2f8a80" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:25.289076 20213 raft_consensus.cc:3055] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:25.289500 20213 raft_consensus.cc:740] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:25.290750 20213 consensus_queue.cc:260] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.292325 20213 raft_consensus.cc:3060] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.295526 20139 raft_consensus.cc:2468] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:25.296437 20295 consensus_queue.cc:1048] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.296885 20031 leader_election.cc:304] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:25.297785 20273 raft_consensus.cc:2804] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:25.298318 20273 raft_consensus.cc:697] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:25.299204 20273 consensus_queue.cc:237] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.303263 20213 raft_consensus.cc:2468] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:25.309413 20261 consensus_queue.cc:1048] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:25.308969 19963 catalog_manager.cc:5697] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 1 to 2, leader changed from ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67) to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:25.324365 20273 consensus_queue.cc:1048] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.338531 20260 consensus_queue.cc:1048] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:25.365041 20260 consensus_queue.cc:1048] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.376732 20313 consensus_queue.cc:1048] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.385250 20260 raft_consensus.cc:993] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: : Instructing follower 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:25.385888 20313 raft_consensus.cc:1081] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Signalling peer 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:25.387419 20139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:56208
I20260709 12:31:25.387946 20139 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.388350 20139 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.393533 20139 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.395375 20139 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:25.396278 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:25.396579 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:25.397122 20063 raft_consensus.cc:3055] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:25.397324 20213 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.397526 20063 raft_consensus.cc:740] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:25.398655 20063 consensus_queue.cc:260] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.400147 20063 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.403605 20213 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:25.404853 20103 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:25.405715 20294 raft_consensus.cc:2804] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:25.406157 20294 raft_consensus.cc:697] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:25.406831 20063 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:25.406878 20294 consensus_queue.cc:237] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.414882 19962 catalog_manager.cc:5697] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 1 to 2, leader changed from 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65) to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:25.446434 20261 consensus_queue.cc:1048] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:25.453786 20293 raft_consensus.cc:993] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: : Instructing follower 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:25.454393 20295 raft_consensus.cc:1081] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Signalling peer 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:25.456176 20140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:56212
I20260709 12:31:25.456816 20140 raft_consensus.cc:493] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:25.457237 20140 raft_consensus.cc:3060] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.460871 20293 consensus_queue.cc:1048] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:25.466125 20140 raft_consensus.cc:515] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.467516 20139 raft_consensus.cc:1240] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Rejecting Update request from peer ffbb6758e9844e559889a1a5c44bcc22 for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:25.469429 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:25.469900 20140 leader_election.cc:290] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:25.470132 20065 raft_consensus.cc:3060] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.473318 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53759e2a1e4085a643cb708d27e7ca" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:25.473943 20213 raft_consensus.cc:3055] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:25.474300 20213 raft_consensus.cc:740] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:25.473960 20293 consensus_queue.cc:1059] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:25.475839 20213 consensus_queue.cc:260] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.477285 20213 raft_consensus.cc:3060] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:25.478614 20065 raft_consensus.cc:2468] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:25.482367 20104 leader_election.cc:304] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:25.483323 20294 raft_consensus.cc:2804] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:25.483892 20294 raft_consensus.cc:697] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:25.485015 20213 raft_consensus.cc:2468] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:25.484753 20294 consensus_queue.cc:237] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:25.491977 19960 catalog_manager.cc:5697] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 1 to 2, leader changed from ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67) to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:25.719532 20213 raft_consensus.cc:1275] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.721012 20313 consensus_queue.cc:1048] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.729346 20139 raft_consensus.cc:1275] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.731809 20260 consensus_queue.cc:1048] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.734429 20065 raft_consensus.cc:1275] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.736075 20264 consensus_queue.cc:1048] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.745530 20213 raft_consensus.cc:1275] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.747593 20294 consensus_queue.cc:1048] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.828409 20213 raft_consensus.cc:1275] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.829972 20292 consensus_queue.cc:1048] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:25.842023 20065 raft_consensus.cc:1275] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:25.844085 20264 consensus_queue.cc:1048] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:26.060019 20065 raft_consensus.cc:1275] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:26.061739 20264 consensus_queue.cc:1048] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:26.071280 20213 raft_consensus.cc:1275] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:26.072973 20264 consensus_queue.cc:1048] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:28.180197 19962 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60048:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:31:28.182772 19962 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:31:28.234424 20050 tablet_service.cc:1511] Processing CreateTablet for tablet d9ea30d483d3479a9ebd702ea048bf89 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:28.235934 20050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ea30d483d3479a9ebd702ea048bf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.235759 20128 tablet_service.cc:1511] Processing CreateTablet for tablet d9ea30d483d3479a9ebd702ea048bf89 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:28.237027 20128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ea30d483d3479a9ebd702ea048bf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.237651 20201 tablet_service.cc:1511] Processing CreateTablet for tablet d9ea30d483d3479a9ebd702ea048bf89 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:31:28.238936 20201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9ea30d483d3479a9ebd702ea048bf89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.238811 20053 tablet_service.cc:1511] Processing CreateTablet for tablet a90e46fa6f0841c5a441d6f7b2f4ea4b (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:28.240085 20053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a90e46fa6f0841c5a441d6f7b2f4ea4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.240360 20127 tablet_service.cc:1511] Processing CreateTablet for tablet a90e46fa6f0841c5a441d6f7b2f4ea4b (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:28.241839 20127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a90e46fa6f0841c5a441d6f7b2f4ea4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.253834 20204 tablet_service.cc:1511] Processing CreateTablet for tablet a90e46fa6f0841c5a441d6f7b2f4ea4b (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:31:28.255288 20204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a90e46fa6f0841c5a441d6f7b2f4ea4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.255558 20052 tablet_service.cc:1511] Processing CreateTablet for tablet 1e7d03c715ef4366ab4277a3db8ae2ac (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:28.256816 20052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e7d03c715ef4366ab4277a3db8ae2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.260564 20125 tablet_service.cc:1511] Processing CreateTablet for tablet 1e7d03c715ef4366ab4277a3db8ae2ac (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:28.262090 20125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e7d03c715ef4366ab4277a3db8ae2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.273099 20202 tablet_service.cc:1511] Processing CreateTablet for tablet 1e7d03c715ef4366ab4277a3db8ae2ac (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:31:28.274673 20202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e7d03c715ef4366ab4277a3db8ae2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.282895 20351 tablet_bootstrap.cc:492] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.286455 20055 tablet_service.cc:1511] Processing CreateTablet for tablet 9270a5ddd9e34e5d80c3fb17458d1e8a (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:28.288403 20055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9270a5ddd9e34e5d80c3fb17458d1e8a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:31:28.291121 20104 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:39279, user_credentials={real_user=slave}} blocked reactor thread for 46393.4us
I20260709 12:31:28.291143 20351 tablet_bootstrap.cc:654] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.294404 20128 tablet_service.cc:1511] Processing CreateTablet for tablet 9270a5ddd9e34e5d80c3fb17458d1e8a (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:28.295904 20128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9270a5ddd9e34e5d80c3fb17458d1e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.310343 20353 tablet_bootstrap.cc:492] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.314390 20202 tablet_service.cc:1511] Processing CreateTablet for tablet 9270a5ddd9e34e5d80c3fb17458d1e8a (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:31:28.316742 20351 tablet_bootstrap.cc:492] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.316949 20202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9270a5ddd9e34e5d80c3fb17458d1e8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.317411 20351 ts_tablet_manager.cc:1403] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.000s
I20260709 12:31:28.317942 20127 tablet_service.cc:1511] Processing CreateTablet for tablet 3d7f52bd35e44dfb87d7441b4c42155d (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:28.319407 20127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d7f52bd35e44dfb87d7441b4c42155d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.321012 20354 tablet_bootstrap.cc:492] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.320595 20351 raft_consensus.cc:359] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.322072 20351 raft_consensus.cc:385] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.322453 20351 raft_consensus.cc:740] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.323290 20351 consensus_queue.cc:260] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.329541 20351 ts_tablet_manager.cc:1434] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.012s	user 0.000s	sys 0.008s
I20260709 12:31:28.330070 20354 tablet_bootstrap.cc:654] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.331038 20055 tablet_service.cc:1511] Processing CreateTablet for tablet 3d7f52bd35e44dfb87d7441b4c42155d (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:28.332461 20055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d7f52bd35e44dfb87d7441b4c42155d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.333740 20264 raft_consensus.cc:493] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.333988 20351 tablet_bootstrap.cc:492] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.334398 20264 raft_consensus.cc:515] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.336952 20264 leader_election.cc:290] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.338510 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ea30d483d3479a9ebd702ea048bf89" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:28.340680 20351 tablet_bootstrap.cc:654] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.340826 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ea30d483d3479a9ebd702ea048bf89" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
W20260709 12:31:28.340894 20104 leader_election.cc:343] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:31:28.344789 20103 leader_election.cc:343] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.345270 20103 leader_election.cc:304] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 301528e0176e4eff834c92ca750633a4; no voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:28.346101 20264 raft_consensus.cc:2749] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:28.354023 20201 tablet_service.cc:1511] Processing CreateTablet for tablet 3d7f52bd35e44dfb87d7441b4c42155d (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:31:28.355948 20201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d7f52bd35e44dfb87d7441b4c42155d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.353986 20351 tablet_bootstrap.cc:492] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.361845 20351 ts_tablet_manager.cc:1403] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.001s
I20260709 12:31:28.364838 20055 tablet_service.cc:1511] Processing CreateTablet for tablet 91d954ab55504e1c871e1f595131acc7 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:28.366143 20055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d954ab55504e1c871e1f595131acc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.366842 20129 tablet_service.cc:1511] Processing CreateTablet for tablet 91d954ab55504e1c871e1f595131acc7 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:28.368104 20129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d954ab55504e1c871e1f595131acc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.375586 20354 tablet_bootstrap.cc:492] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.376372 20354 ts_tablet_manager.cc:1403] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.017s
I20260709 12:31:28.377861 20204 tablet_service.cc:1511] Processing CreateTablet for tablet 91d954ab55504e1c871e1f595131acc7 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:31:28.379333 20204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91d954ab55504e1c871e1f595131acc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.379915 20354 raft_consensus.cc:359] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.380723 20354 raft_consensus.cc:385] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.381016 20354 raft_consensus.cc:740] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.383561 20353 tablet_bootstrap.cc:654] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.383762 20351 raft_consensus.cc:359] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.388410 20351 raft_consensus.cc:385] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.388711 20351 raft_consensus.cc:740] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.389460 20351 consensus_queue.cc:260] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.386196 20354 consensus_queue.cc:260] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.391775 20351 ts_tablet_manager.cc:1434] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.003s
I20260709 12:31:28.392838 20351 tablet_bootstrap.cc:492] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.392966 20354 ts_tablet_manager.cc:1434] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:31:28.394189 20354 tablet_bootstrap.cc:492] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.408960 20354 tablet_bootstrap.cc:654] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.409041 20055 tablet_service.cc:1511] Processing CreateTablet for tablet ff2b19ad91124ab1a278e2acd46ac9d8 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:28.412612 20055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2b19ad91124ab1a278e2acd46ac9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.410658 20351 tablet_bootstrap.cc:654] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.416349 20053 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8f16c406474c98a27604be7c0aca64 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:28.418401 20053 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8f16c406474c98a27604be7c0aca64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.420149 20128 tablet_service.cc:1511] Processing CreateTablet for tablet ff2b19ad91124ab1a278e2acd46ac9d8 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:28.421320 20201 tablet_service.cc:1511] Processing CreateTablet for tablet ff2b19ad91124ab1a278e2acd46ac9d8 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:31:28.423622 20201 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2b19ad91124ab1a278e2acd46ac9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.427145 20128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff2b19ad91124ab1a278e2acd46ac9d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.432297 20127 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8f16c406474c98a27604be7c0aca64 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:28.433799 20127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8f16c406474c98a27604be7c0aca64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.434306 20052 tablet_service.cc:1511] Processing CreateTablet for tablet ac90934570c4402ead594675203444e2 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:28.435745 20052 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac90934570c4402ead594675203444e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.438639 20204 tablet_service.cc:1511] Processing CreateTablet for tablet 9f8f16c406474c98a27604be7c0aca64 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:31:28.440126 20204 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f8f16c406474c98a27604be7c0aca64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.440218 20202 tablet_service.cc:1511] Processing CreateTablet for tablet ac90934570c4402ead594675203444e2 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:28.441548 20202 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac90934570c4402ead594675203444e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:31:28.444864 20129 tablet_service.cc:1511] Processing CreateTablet for tablet ac90934570c4402ead594675203444e2 (DEFAULT_TABLE table=soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:31:28.446372 20129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac90934570c4402ead594675203444e2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:31:28.451033 20011 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:31:28.468309 20353 tablet_bootstrap.cc:492] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.468890 20353 ts_tablet_manager.cc:1403] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.012s
I20260709 12:31:28.470110 20354 tablet_bootstrap.cc:492] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.470779 20354 ts_tablet_manager.cc:1403] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.077s	user 0.010s	sys 0.003s
I20260709 12:31:28.473054 20353 raft_consensus.cc:359] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.473973 20353 raft_consensus.cc:385] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.474707 20353 raft_consensus.cc:740] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.477351 20351 tablet_bootstrap.cc:492] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.475744 20353 consensus_queue.cc:260] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.478044 20351 ts_tablet_manager.cc:1403] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.085s	user 0.027s	sys 0.009s
I20260709 12:31:28.482020 20351 raft_consensus.cc:359] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.482829 20351 raft_consensus.cc:385] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.483155 20351 raft_consensus.cc:740] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.484109 20351 consensus_queue.cc:260] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.485260 20353 ts_tablet_manager.cc:1434] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.016s	user 0.003s	sys 0.003s
I20260709 12:31:28.474088 20354 raft_consensus.cc:359] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.486109 20354 raft_consensus.cc:385] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.486575 20354 raft_consensus.cc:740] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.487347 20354 consensus_queue.cc:260] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.491959 20353 tablet_bootstrap.cc:492] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.494872 20354 ts_tablet_manager.cc:1434] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.001s
I20260709 12:31:28.499665 20353 tablet_bootstrap.cc:654] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.504258 20354 tablet_bootstrap.cc:492] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.504781 20351 ts_tablet_manager.cc:1434] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.001s
I20260709 12:31:28.508975 20351 tablet_bootstrap.cc:492] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.521243 20351 tablet_bootstrap.cc:654] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.526077 20353 tablet_bootstrap.cc:492] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.526684 20353 ts_tablet_manager.cc:1403] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.036s	user 0.021s	sys 0.011s
I20260709 12:31:28.528021 20351 tablet_bootstrap.cc:492] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.528571 20351 ts_tablet_manager.cc:1403] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.002s
I20260709 12:31:28.529032 20354 tablet_bootstrap.cc:654] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.529767 20353 raft_consensus.cc:359] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.530645 20353 raft_consensus.cc:385] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.530941 20353 raft_consensus.cc:740] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.531710 20351 raft_consensus.cc:359] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.532672 20351 raft_consensus.cc:385] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.531778 20353 consensus_queue.cc:260] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.533092 20351 raft_consensus.cc:740] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.533973 20351 consensus_queue.cc:260] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.535926 20353 ts_tablet_manager.cc:1434] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 12:31:28.537104 20353 tablet_bootstrap.cc:492] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.537663 20351 ts_tablet_manager.cc:1434] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 12:31:28.538765 20351 tablet_bootstrap.cc:492] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.543757 20353 tablet_bootstrap.cc:654] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.545042 20354 tablet_bootstrap.cc:492] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.545742 20354 ts_tablet_manager.cc:1403] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.007s
I20260709 12:31:28.546473 20351 tablet_bootstrap.cc:654] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.550158 20264 raft_consensus.cc:493] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.550890 20264 raft_consensus.cc:515] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.556412 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:28.558858 20264 leader_election.cc:290] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.549463 20354 raft_consensus.cc:359] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.561523 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
W20260709 12:31:28.564080 20103 leader_election.cc:343] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.565083 20354 raft_consensus.cc:385] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.565696 20354 raft_consensus.cc:740] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.567087 20354 consensus_queue.cc:260] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.568514 20065 raft_consensus.cc:2468] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:28.571430 20104 leader_election.cc:304] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:28.572535 20264 raft_consensus.cc:2804] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:28.572921 20264 raft_consensus.cc:493] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:28.573257 20264 raft_consensus.cc:3060] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.574352 20353 tablet_bootstrap.cc:492] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.574357 20351 tablet_bootstrap.cc:492] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.576097 20354 ts_tablet_manager.cc:1434] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 12:31:28.576602 20351 ts_tablet_manager.cc:1403] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.013s
I20260709 12:31:28.577533 20354 tablet_bootstrap.cc:492] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.578038 20295 raft_consensus.cc:493] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.578637 20295 raft_consensus.cc:515] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.582309 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:28.583108 20139 raft_consensus.cc:2468] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:28.582854 20264 raft_consensus.cc:515] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.575124 20353 ts_tablet_manager.cc:1403] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.038s	user 0.026s	sys 0.011s
I20260709 12:31:28.585456 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:28.586186 20065 raft_consensus.cc:2468] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:28.584682 20180 leader_election.cc:304] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:28.587786 20264 leader_election.cc:290] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.584990 20354 tablet_bootstrap.cc:654] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.588872 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:28.588950 20353 raft_consensus.cc:359] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.590061 20353 raft_consensus.cc:385] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.590377 20353 raft_consensus.cc:740] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
W20260709 12:31:28.590900 20103 leader_election.cc:343] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.591239 20353 consensus_queue.cc:260] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.592005 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:28.592846 20065 raft_consensus.cc:3060] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.594230 20351 raft_consensus.cc:359] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.595346 20351 raft_consensus.cc:385] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.595749 20351 raft_consensus.cc:740] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.596091 20264 raft_consensus.cc:493] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.591393 20295 leader_election.cc:290] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:28.597499 20295 raft_consensus.cc:2804] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:28.596774 20351 consensus_queue.cc:260] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.597988 20295 raft_consensus.cc:493] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:28.600646 20351 ts_tablet_manager.cc:1434] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.004s
I20260709 12:31:28.605029 20351 tablet_bootstrap.cc:492] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.598450 20295 raft_consensus.cc:3060] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.607916 20350 raft_consensus.cc:493] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.608444 20350 raft_consensus.cc:515] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.611907 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
W20260709 12:31:28.613492 20104 leader_election.cc:343] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.614470 20350 leader_election.cc:290] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.614992 20351 tablet_bootstrap.cc:654] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.615765 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
W20260709 12:31:28.617547 20103 leader_election.cc:343] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.617671 20065 raft_consensus.cc:2468] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:28.619542 20104 leader_election.cc:304] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:28.621315 20350 raft_consensus.cc:2804] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:28.618155 20103 leader_election.cc:304] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 301528e0176e4eff834c92ca750633a4; no voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:28.624217 20350 raft_consensus.cc:697] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:28.629676 20369 raft_consensus.cc:2749] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:28.635174 20350 consensus_queue.cc:237] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.640136 20264 raft_consensus.cc:515] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.646076 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:28.646451 20351 tablet_bootstrap.cc:492] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.647219 20351 ts_tablet_manager.cc:1403] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.013s
I20260709 12:31:28.647092 20065 raft_consensus.cc:2468] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:28.648633 20295 raft_consensus.cc:515] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.649114 20104 leader_election.cc:304] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:28.650797 20351 raft_consensus.cc:359] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.652436 20351 raft_consensus.cc:385] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.652519 20353 ts_tablet_manager.cc:1434] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.067s	user 0.020s	sys 0.023s
I20260709 12:31:28.652776 20351 raft_consensus.cc:740] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.653612 20350 raft_consensus.cc:2804] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:28.654011 20350 raft_consensus.cc:493] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:28.654556 20350 raft_consensus.cc:3060] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.656781 20264 leader_election.cc:290] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.658792 20295 leader_election.cc:290] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:28.654258 20351 consensus_queue.cc:260] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.659571 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:28.659795 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:28.660329 20065 raft_consensus.cc:3060] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.660431 20139 raft_consensus.cc:3060] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.663194 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:28.663854 20213 raft_consensus.cc:2468] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:28.665194 20350 raft_consensus.cc:515] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.668174 20351 ts_tablet_manager.cc:1434] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:31:28.668275 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:28.667762 19963 catalog_manager.cc:5697] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:28.653972 20353 tablet_bootstrap.cc:492] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.669096 20063 raft_consensus.cc:3060] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.671535 20065 raft_consensus.cc:2468] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:28.669337 20351 tablet_bootstrap.cc:492] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.674947 20178 leader_election.cc:304] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:28.691138 20351 tablet_bootstrap.cc:654] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.692727 20353 tablet_bootstrap.cc:654] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.694176 20295 raft_consensus.cc:2804] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:28.668803 20139 raft_consensus.cc:2468] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:28.699095 20350 leader_election.cc:290] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.699546 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:28.697513 20295 raft_consensus.cc:697] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:28.700390 20213 raft_consensus.cc:3060] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.702095 20063 raft_consensus.cc:2468] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:28.702694 20354 tablet_bootstrap.cc:492] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.705236 20354 ts_tablet_manager.cc:1403] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.128s	user 0.024s	sys 0.043s
I20260709 12:31:28.703869 20295 consensus_queue.cc:237] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.704296 20104 leader_election.cc:304] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:28.710223 20213 raft_consensus.cc:2468] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:28.711059 20350 raft_consensus.cc:2804] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:28.711637 20350 raft_consensus.cc:697] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:28.713088 20353 tablet_bootstrap.cc:492] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.712603 20350 consensus_queue.cc:237] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.713770 20353 ts_tablet_manager.cc:1403] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.060s	user 0.016s	sys 0.007s
I20260709 12:31:28.717469 20353 raft_consensus.cc:359] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.718516 20353 raft_consensus.cc:385] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.718917 20353 raft_consensus.cc:740] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.719801 20353 consensus_queue.cc:260] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.709803 20354 raft_consensus.cc:359] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.722946 20354 raft_consensus.cc:385] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.723322 20354 raft_consensus.cc:740] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.724315 20354 consensus_queue.cc:260] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.726478 20353 ts_tablet_manager.cc:1434] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.003s
I20260709 12:31:28.727371 20354 ts_tablet_manager.cc:1434] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.003s
I20260709 12:31:28.727905 20353 tablet_bootstrap.cc:492] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.728606 20354 tablet_bootstrap.cc:492] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.739364 20354 tablet_bootstrap.cc:654] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.740828 20353 tablet_bootstrap.cc:654] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.744900 20351 tablet_bootstrap.cc:492] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.741428 19962 catalog_manager.cc:5697] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:28.745559 20351 ts_tablet_manager.cc:1403] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.076s	user 0.041s	sys 0.007s
I20260709 12:31:28.750058 20351 raft_consensus.cc:359] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.751298 20351 raft_consensus.cc:385] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.751617 20351 raft_consensus.cc:740] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.752179 19960 catalog_manager.cc:5697] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:28.756358 20351 consensus_queue.cc:260] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.759229 20353 tablet_bootstrap.cc:492] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.761497 20353 ts_tablet_manager.cc:1403] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.034s	user 0.005s	sys 0.012s
I20260709 12:31:28.762558 20351 ts_tablet_manager.cc:1434] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.005s
I20260709 12:31:28.760139 20354 tablet_bootstrap.cc:492] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.763379 20354 ts_tablet_manager.cc:1403] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.000s
I20260709 12:31:28.765259 20353 raft_consensus.cc:359] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.766175 20353 raft_consensus.cc:385] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.766587 20353 raft_consensus.cc:740] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.766865 20354 raft_consensus.cc:359] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.767637 20353 consensus_queue.cc:260] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.768929 20354 raft_consensus.cc:385] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.769383 20354 raft_consensus.cc:740] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.770596 20350 raft_consensus.cc:493] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.771436 20353 ts_tablet_manager.cc:1434] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:31:28.771255 20350 raft_consensus.cc:515] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.772699 20353 tablet_bootstrap.cc:492] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.773505 20350 leader_election.cc:290] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.775307 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
W20260709 12:31:28.776897 20104 leader_election.cc:343] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.770301 20354 consensus_queue.cc:260] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.768582 20351 tablet_bootstrap.cc:492] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.779296 20354 ts_tablet_manager.cc:1434] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:31:28.781036 20353 tablet_bootstrap.cc:654] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.786201 20354 tablet_bootstrap.cc:492] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.787125 20351 tablet_bootstrap.cc:654] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.787478 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
W20260709 12:31:28.790817 20103 leader_election.cc:343] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:31:28.791345 20103 leader_election.cc:304] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 301528e0176e4eff834c92ca750633a4; no voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:28.794240 20264 raft_consensus.cc:2749] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:28.794888 20354 tablet_bootstrap.cc:654] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.801265 20351 tablet_bootstrap.cc:492] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.801905 20351 ts_tablet_manager.cc:1403] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.034s	user 0.015s	sys 0.000s
I20260709 12:31:28.802178 20353 tablet_bootstrap.cc:492] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.802897 20353 ts_tablet_manager.cc:1403] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.007s
I20260709 12:31:28.805045 20351 raft_consensus.cc:359] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.805989 20351 raft_consensus.cc:385] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.806473 20351 raft_consensus.cc:740] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.807981 20353 raft_consensus.cc:359] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.807791 20351 consensus_queue.cc:260] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.809238 20353 raft_consensus.cc:385] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.809790 20353 raft_consensus.cc:740] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.810861 20353 consensus_queue.cc:260] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.812598 20351 ts_tablet_manager.cc:1434] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 12:31:28.813722 20351 tablet_bootstrap.cc:492] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: Bootstrap starting.
I20260709 12:31:28.815063 20354 tablet_bootstrap.cc:492] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.815806 20354 ts_tablet_manager.cc:1403] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.007s
I20260709 12:31:28.819662 20351 tablet_bootstrap.cc:654] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.821326 20353 ts_tablet_manager.cc:1434] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.003s
I20260709 12:31:28.819335 20354 raft_consensus.cc:359] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.822597 20353 tablet_bootstrap.cc:492] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.822619 20354 raft_consensus.cc:385] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.823339 20354 raft_consensus.cc:740] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.824146 20354 consensus_queue.cc:260] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.826529 20354 ts_tablet_manager.cc:1434] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:31:28.829023 20351 tablet_bootstrap.cc:492] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: No bootstrap required, opened a new log
I20260709 12:31:28.829548 20351 ts_tablet_manager.cc:1403] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:31:28.830986 20353 tablet_bootstrap.cc:654] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.832222 20351 raft_consensus.cc:359] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.832971 20351 raft_consensus.cc:385] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.833235 20351 raft_consensus.cc:740] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.833877 20351 consensus_queue.cc:260] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.835850 20351 ts_tablet_manager.cc:1434] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:31:28.837013 20354 tablet_bootstrap.cc:492] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.843122 20354 tablet_bootstrap.cc:654] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.850045 20353 tablet_bootstrap.cc:492] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.850792 20353 ts_tablet_manager.cc:1403] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.014s
I20260709 12:31:28.854750 20353 raft_consensus.cc:359] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.855532 20353 raft_consensus.cc:385] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.855901 20353 raft_consensus.cc:740] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.856407 20354 tablet_bootstrap.cc:492] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.856935 20354 ts_tablet_manager.cc:1403] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 12:31:28.856812 20353 consensus_queue.cc:260] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.859824 20353 ts_tablet_manager.cc:1434] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:31:28.859962 20354 raft_consensus.cc:359] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.860770 20354 raft_consensus.cc:385] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.861068 20354 raft_consensus.cc:740] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.861152 20353 tablet_bootstrap.cc:492] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.862016 20354 consensus_queue.cc:260] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.864521 20354 ts_tablet_manager.cc:1434] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:31:28.867628 20354 tablet_bootstrap.cc:492] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.869796 20353 tablet_bootstrap.cc:654] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.874003 20354 tablet_bootstrap.cc:654] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.876912 20353 tablet_bootstrap.cc:492] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.877568 20353 ts_tablet_manager.cc:1403] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.001s
I20260709 12:31:28.879257 20354 tablet_bootstrap.cc:492] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.880537 20354 ts_tablet_manager.cc:1403] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:31:28.880373 20353 raft_consensus.cc:359] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.881274 20353 raft_consensus.cc:385] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.881665 20353 raft_consensus.cc:740] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.882568 20353 consensus_queue.cc:260] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.883334 20354 raft_consensus.cc:359] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.884132 20353 ts_tablet_manager.cc:1434] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:31:28.884141 20354 raft_consensus.cc:385] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.884874 20354 raft_consensus.cc:740] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.885819 20353 tablet_bootstrap.cc:492] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: Bootstrap starting.
I20260709 12:31:28.885633 20354 consensus_queue.cc:260] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.887897 20354 ts_tablet_manager.cc:1434] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:31:28.889367 20354 tablet_bootstrap.cc:492] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: Bootstrap starting.
I20260709 12:31:28.892578 20353 tablet_bootstrap.cc:654] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.896456 20354 tablet_bootstrap.cc:654] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: Neither blocks nor log segments found. Creating new log.
I20260709 12:31:28.904327 20313 raft_consensus.cc:493] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.904994 20313 raft_consensus.cc:515] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.907225 20313 leader_election.cc:290] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.910315 20353 tablet_bootstrap.cc:492] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: No bootstrap required, opened a new log
I20260709 12:31:28.910009 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ea30d483d3479a9ebd702ea048bf89" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:28.910348 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ea30d483d3479a9ebd702ea048bf89" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:28.910944 20139 raft_consensus.cc:2468] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:28.911537 20214 raft_consensus.cc:2468] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:28.910861 20353 ts_tablet_manager.cc:1403] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.000s
I20260709 12:31:28.912711 20028 leader_election.cc:304] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:28.913434 20313 raft_consensus.cc:2804] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:28.913740 20354 tablet_bootstrap.cc:492] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: No bootstrap required, opened a new log
I20260709 12:31:28.913913 20313 raft_consensus.cc:493] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:28.914357 20354 ts_tablet_manager.cc:1403] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.008s
I20260709 12:31:28.914376 20313 raft_consensus.cc:3060] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.914919 20353 raft_consensus.cc:359] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.915659 20353 raft_consensus.cc:385] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.915930 20353 raft_consensus.cc:740] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.916603 20353 consensus_queue.cc:260] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.917249 20354 raft_consensus.cc:359] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.917898 20354 raft_consensus.cc:385] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:31:28.918164 20354 raft_consensus.cc:740] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Initialized, Role: FOLLOWER
I20260709 12:31:28.918803 20353 ts_tablet_manager.cc:1434] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260709 12:31:28.918816 20354 consensus_queue.cc:260] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.921928 20354 ts_tablet_manager.cc:1434] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:31:28.922854 20369 raft_consensus.cc:493] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.923437 20369 raft_consensus.cc:515] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.925637 20369 leader_election.cc:290] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.926200 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2b19ad91124ab1a278e2acd46ac9d8" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:28.926901 20063 raft_consensus.cc:2468] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:28.927237 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2b19ad91124ab1a278e2acd46ac9d8" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:28.927842 20214 raft_consensus.cc:2468] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:28.929085 20313 raft_consensus.cc:515] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.929368 20104 leader_election.cc:304] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:28.930308 20369 raft_consensus.cc:2804] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:28.930799 20369 raft_consensus.cc:493] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:28.931178 20369 raft_consensus.cc:3060] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.936386 20313 leader_election.cc:290] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.937191 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ea30d483d3479a9ebd702ea048bf89" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:28.937937 20139 raft_consensus.cc:3060] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.940335 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d9ea30d483d3479a9ebd702ea048bf89" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:28.940768 20369 raft_consensus.cc:515] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.943310 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2b19ad91124ab1a278e2acd46ac9d8" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:28.943533 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff2b19ad91124ab1a278e2acd46ac9d8" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:28.943889 20063 raft_consensus.cc:3060] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.944051 20213 raft_consensus.cc:3060] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.945122 20214 raft_consensus.cc:3060] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.946601 20139 raft_consensus.cc:2468] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:28.948014 20031 leader_election.cc:304] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:28.950745 20213 raft_consensus.cc:2468] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:28.951229 20313 raft_consensus.cc:493] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.951539 20063 raft_consensus.cc:2468] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:28.951997 20103 leader_election.cc:304] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:28.952275 20388 raft_consensus.cc:2804] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:28.952924 20368 raft_consensus.cc:2804] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:28.953047 20369 leader_election.cc:290] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.951881 20313 raft_consensus.cc:515] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.953833 20368 raft_consensus.cc:697] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:28.953127 20214 raft_consensus.cc:2468] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:28.954813 20368 consensus_queue.cc:237] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.956314 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:28.956844 20139 raft_consensus.cc:2468] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:28.958014 20031 leader_election.cc:304] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:28.958573 20295 raft_consensus.cc:493] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:28.958961 20313 leader_election.cc:290] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.959323 20295 raft_consensus.cc:515] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.961992 20313 raft_consensus.cc:2804] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:28.962646 20313 raft_consensus.cc:493] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:28.963281 20295 leader_election.cc:290] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:28.965101 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:28.964658 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:28.966456 20063 raft_consensus.cc:2393] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ffbb6758e9844e559889a1a5c44bcc22 in current term 1: Already voted for candidate 301528e0176e4eff834c92ca750633a4 in this term.
I20260709 12:31:28.963740 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:28.967979 20178 leader_election.cc:304] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:28.969053 20214 raft_consensus.cc:2468] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:28.969851 20295 raft_consensus.cc:3060] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.963197 20313 raft_consensus.cc:3060] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.974139 20388 raft_consensus.cc:697] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:28.976719 20388 consensus_queue.cc:237] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.981464 20295 raft_consensus.cc:2749] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:31:28.987329 20313 raft_consensus.cc:515] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:28.986675 19961 catalog_manager.cc:5697] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:28.989827 20313 leader_election.cc:290] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:28.989454 19962 catalog_manager.cc:5697] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:28.992493 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:28.993157 20214 raft_consensus.cc:3060] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:28.995541 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:28.996277 20139 raft_consensus.cc:3060] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.002292 20214 raft_consensus.cc:2468] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:29.003643 20028 leader_election.cc:304] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.004467 20313 raft_consensus.cc:2804] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:29.004999 20313 raft_consensus.cc:697] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.005734 20139 raft_consensus.cc:2468] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:29.005716 20313 consensus_queue.cc:237] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.014588 19962 catalog_manager.cc:5697] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.069442 20368 raft_consensus.cc:493] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:29.070053 20368 raft_consensus.cc:515] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.072039 20368 leader_election.cc:290] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.073190 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d7f52bd35e44dfb87d7441b4c42155d" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:29.073419 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d7f52bd35e44dfb87d7441b4c42155d" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:29.074118 20063 raft_consensus.cc:2468] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:29.074244 20214 raft_consensus.cc:2468] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 0.
I20260709 12:31:29.075357 20104 leader_election.cc:304] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:29.076162 20295 raft_consensus.cc:493] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:29.076242 20368 raft_consensus.cc:2804] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:29.076843 20368 raft_consensus.cc:493] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:29.076771 20295 raft_consensus.cc:515] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.077282 20368 raft_consensus.cc:3060] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.078820 20295 leader_election.cc:290] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:29.079692 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:29.079916 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:29.080451 20063 raft_consensus.cc:2468] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:29.080559 20139 raft_consensus.cc:2468] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 0.
I20260709 12:31:29.081741 20178 leader_election.cc:304] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.082614 20295 raft_consensus.cc:2804] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:29.083010 20295 raft_consensus.cc:493] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:29.083294 20295 raft_consensus.cc:3060] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.084499 20368 raft_consensus.cc:515] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.086265 20368 leader_election.cc:290] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.087282 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d7f52bd35e44dfb87d7441b4c42155d" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:29.087838 20063 raft_consensus.cc:3060] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.087584 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d7f52bd35e44dfb87d7441b4c42155d" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.088399 20214 raft_consensus.cc:3060] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.090963 20295 raft_consensus.cc:515] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.094184 20063 raft_consensus.cc:2468] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:29.094305 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:29.093638 20295 leader_election.cc:290] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:29.095186 20139 raft_consensus.cc:3060] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.095907 20104 leader_election.cc:304] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:29.096894 20368 raft_consensus.cc:2804] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:29.097551 20214 raft_consensus.cc:2468] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 1.
I20260709 12:31:29.098341 20369 consensus_queue.cc:1048] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.098644 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:29.099292 20063 raft_consensus.cc:3060] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.099833 20368 raft_consensus.cc:697] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:29.102222 20368 consensus_queue.cc:237] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.104640 20139 raft_consensus.cc:2468] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:29.106097 20180 leader_election.cc:304] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.107049 20295 raft_consensus.cc:2804] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:29.107611 20295 raft_consensus.cc:697] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:29.108750 20295 consensus_queue.cc:237] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.106421 20063 raft_consensus.cc:2468] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 1.
I20260709 12:31:29.114666 20264 consensus_queue.cc:1048] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:29.123926 20368 consensus_queue.cc:1048] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.124276 19962 catalog_manager.cc:5697] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 1 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:29.128707 19961 catalog_manager.cc:5697] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 1 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.135388 20369 consensus_queue.cc:1048] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:29.200093 20313 raft_consensus.cc:493] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:29.200585 20313 raft_consensus.cc:515] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.202826 20313 leader_election.cc:290] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.203887 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac90934570c4402ead594675203444e2" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:29.203733 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac90934570c4402ead594675203444e2" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:29.204591 20213 raft_consensus.cc:2468] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:29.204669 20139 raft_consensus.cc:2468] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 0.
I20260709 12:31:29.205720 20031 leader_election.cc:304] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:29.206350 20313 raft_consensus.cc:2804] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:31:29.206724 20313 raft_consensus.cc:493] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:29.206959 20313 raft_consensus.cc:3060] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.212527 20313 raft_consensus.cc:515] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.214177 20313 leader_election.cc:290] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.215005 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac90934570c4402ead594675203444e2" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:29.215339 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac90934570c4402ead594675203444e2" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.215708 20139 raft_consensus.cc:3060] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.215883 20213 raft_consensus.cc:3060] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:31:29.220005 20295 consensus_queue.cc:1048] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:29.222093 20213 raft_consensus.cc:2468] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:29.222105 20139 raft_consensus.cc:2468] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 1.
I20260709 12:31:29.223403 20028 leader_election.cc:304] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.224659 20313 raft_consensus.cc:2804] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:31:29.225140 20313 raft_consensus.cc:697] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.226013 20313 consensus_queue.cc:237] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.235055 20295 consensus_queue.cc:1048] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.247938 19963 catalog_manager.cc:5697] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 1 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.366783 20368 consensus_queue.cc:1048] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.382547 20264 consensus_queue.cc:1048] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.386817 20313 consensus_queue.cc:1048] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.415244 20388 consensus_queue.cc:1048] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.424266 20388 consensus_queue.cc:1048] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.425256 20140 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
mode: GRACEFUL
new_leader_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:49804
I20260709 12:31:29.425951 20140 raft_consensus.cc:606] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Received request to transfer leadership to TS 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:29.434031 20213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
mode: GRACEFUL
new_leader_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:53968
I20260709 12:31:29.434671 20213 raft_consensus.cc:606] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Received request to transfer leadership to TS 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:29.445116 20139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
mode: GRACEFUL
new_leader_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:49804
I20260709 12:31:29.445812 20139 raft_consensus.cc:606] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Received request to transfer leadership to TS 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:29.450284 20313 consensus_queue.cc:1048] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.462029 20213 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
mode: GRACEFUL
new_leader_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:53968
I20260709 12:31:29.462760 20213 raft_consensus.cc:606] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Received request to transfer leadership to TS 301528e0176e4eff834c92ca750633a4
I20260709 12:31:29.473526 20065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
mode: GRACEFUL
new_leader_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:31:29.474227 20065 raft_consensus.cc:606] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Received request to transfer leadership to TS 301528e0176e4eff834c92ca750633a4
I20260709 12:31:29.480183 20065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac90934570c4402ead594675203444e2"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
mode: GRACEFUL
new_leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
 from {username='slave'} at 127.0.0.1:43026
I20260709 12:31:29.480890 20065 raft_consensus.cc:606] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Received request to transfer leadership to TS ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:29.524444 20368 raft_consensus.cc:993] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: : Instructing follower 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:29.525163 20264 raft_consensus.cc:1081] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Signalling peer 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:29.527312 20065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:53224
I20260709 12:31:29.527881 20065 raft_consensus.cc:493] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:29.528206 20065 raft_consensus.cc:3060] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.535773 20065 raft_consensus.cc:515] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.538120 20065 leader_election.cc:290] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.539369 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.540076 20214 raft_consensus.cc:3060] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.541488 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a90e46fa6f0841c5a441d6f7b2f4ea4b" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:29.542296 20139 raft_consensus.cc:3055] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:29.542659 20139 raft_consensus.cc:740] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:29.543540 20139 consensus_queue.cc:260] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.544888 20139 raft_consensus.cc:3060] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.549850 20214 raft_consensus.cc:2468] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:29.551389 20028 leader_election.cc:304] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.551756 20139 raft_consensus.cc:2468] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:29.552472 20313 raft_consensus.cc:2804] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:29.553043 20313 raft_consensus.cc:697] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.554257 20313 consensus_queue.cc:237] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.560091 20368 consensus_queue.cc:1048] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:29.568395 19960 catalog_manager.cc:5697] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 1 to 2, leader changed from 301528e0176e4eff834c92ca750633a4 (127.5.4.66) to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.577165 20264 consensus_queue.cc:1048] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.587991 20367 consensus_queue.cc:1048] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.610198 20313 consensus_queue.cc:1048] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.611109 20367 consensus_queue.cc:1048] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.629568 20367 raft_consensus.cc:993] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: : Instructing follower 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:29.630049 20367 raft_consensus.cc:1081] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Signalling peer 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:29.631959 20139 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:56212
I20260709 12:31:29.632624 20139 raft_consensus.cc:493] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:29.633013 20139 raft_consensus.cc:3060] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.635236 20389 raft_consensus.cc:993] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: : Instructing follower ffbb6758e9844e559889a1a5c44bcc22 to start an election
I20260709 12:31:29.635792 20389 raft_consensus.cc:1081] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Signalling peer ffbb6758e9844e559889a1a5c44bcc22 to start an election
I20260709 12:31:29.637976 20213 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac90934570c4402ead594675203444e2"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
 from {username='slave'} at 127.0.0.1:53946
I20260709 12:31:29.638605 20213 raft_consensus.cc:493] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:29.638940 20213 raft_consensus.cc:3060] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.640520 20139 raft_consensus.cc:515] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.642760 20139 leader_election.cc:290] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.644987 20212 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.645325 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91d954ab55504e1c871e1f595131acc7" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:29.646075 20063 raft_consensus.cc:3060] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.652412 20388 consensus_queue.cc:1048] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:31:29.654372 20264 raft_consensus.cc:993] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: : Instructing follower 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:29.654963 20350 raft_consensus.cc:1081] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Signalling peer 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:29.655196 20212 raft_consensus.cc:3055] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:29.657932 20213 raft_consensus.cc:515] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.658205 20212 raft_consensus.cc:740] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:29.659058 20214 raft_consensus.cc:1240] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Rejecting Update request from peer 9f1e938b2be54950b77fb71c531a1d34 for earlier term 1. Current term is 2. Ops: []
I20260709 12:31:29.657152 20065 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:53224
I20260709 12:31:29.659718 20065 raft_consensus.cc:493] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:29.660107 20065 raft_consensus.cc:3060] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.661374 20213 leader_election.cc:290] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:29.661309 20063 raft_consensus.cc:2468] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:29.661998 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac90934570c4402ead594675203444e2" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:29.662688 20140 raft_consensus.cc:3060] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.663090 20104 leader_election.cc:304] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:29.662717 20212 consensus_queue.cc:260] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.663496 20388 consensus_queue.cc:1059] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:29.663995 20264 raft_consensus.cc:2804] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:29.664496 20313 raft_consensus.cc:3055] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:29.664600 20212 raft_consensus.cc:3060] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.664697 20264 raft_consensus.cc:697] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:29.665014 20313 raft_consensus.cc:740] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.666029 20264 consensus_queue.cc:237] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.666615 20313 consensus_queue.cc:260] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.667691 20313 raft_consensus.cc:3060] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.680689 20140 raft_consensus.cc:2468] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 2.
I20260709 12:31:29.682024 20180 leader_election.cc:304] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.682451 20367 raft_consensus.cc:993] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: : Instructing follower 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:29.683066 20367 raft_consensus.cc:1081] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Signalling peer 9f1e938b2be54950b77fb71c531a1d34 to start an election
I20260709 12:31:29.683104 20295 raft_consensus.cc:2804] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:29.684141 20295 raft_consensus.cc:697] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:29.685226 20295 consensus_queue.cc:237] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.686597 20061 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
 from {username='slave'} at 127.0.0.1:53240
I20260709 12:31:29.687299 20061 raft_consensus.cc:493] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:29.687724 20061 raft_consensus.cc:3060] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.689932 20139 raft_consensus.cc:1240] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Rejecting Update request from peer 9f1e938b2be54950b77fb71c531a1d34 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:31:29.683454 19960 catalog_manager.cc:5697] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 1 to 2, leader changed from ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67) to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.697103 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac90934570c4402ead594675203444e2" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:29.697263 20061 raft_consensus.cc:515] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.699383 20065 raft_consensus.cc:515] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.701120 20212 raft_consensus.cc:2468] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:29.701460 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:29.702085 20139 raft_consensus.cc:3060] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.702780 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:29.703047 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.703506 20140 raft_consensus.cc:3055] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:29.703603 20213 raft_consensus.cc:3060] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.703940 20140 raft_consensus.cc:740] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:29.704910 20140 consensus_queue.cc:260] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.706152 20140 raft_consensus.cc:3060] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.708367 20139 raft_consensus.cc:2468] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:29.708945 20065 leader_election.cc:290] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.698058 19961 catalog_manager.cc:5697] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 1 to 2, leader changed from 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65) to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:29.709959 20031 leader_election.cc:304] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:29.711140 20388 raft_consensus.cc:2804] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:29.711670 20388 raft_consensus.cc:697] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.712489 20213 raft_consensus.cc:2468] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:29.712666 20388 consensus_queue.cc:237] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.717969 20028 leader_election.cc:304] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.718847 20313 raft_consensus.cc:2804] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:29.719403 20313 raft_consensus.cc:697] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.720183 20313 consensus_queue.cc:237] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.723773 20063 raft_consensus.cc:2468] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 2.
I20260709 12:31:29.728607 20061 leader_election.cc:290] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.728114 19961 catalog_manager.cc:5697] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 1 to 2, leader changed from ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67) to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.729701 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e7d03c715ef4366ab4277a3db8ae2ac" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.730355 20213 raft_consensus.cc:3055] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:29.730522 20140 raft_consensus.cc:2468] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:29.730754 20213 raft_consensus.cc:740] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:29.731846 20213 consensus_queue.cc:260] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.732824 20213 raft_consensus.cc:3060] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.739495 20213 raft_consensus.cc:2468] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:29.744498 19961 catalog_manager.cc:5697] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 1 to 2, leader changed from 301528e0176e4eff834c92ca750633a4 (127.5.4.66) to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 2 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:29.946473 20313 raft_consensus.cc:993] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: : Instructing follower 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:29.947021 20389 raft_consensus.cc:1081] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Signalling peer 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:29.948751 20140 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:56208
I20260709 12:31:29.948967 20213 raft_consensus.cc:1275] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:29.949359 20140 raft_consensus.cc:493] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:29.949894 20140 raft_consensus.cc:3060] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.950558 20389 consensus_queue.cc:1048] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:29.959493 20140 raft_consensus.cc:515] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.961953 20140 leader_election.cc:290] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:29.962549 20139 raft_consensus.cc:1275] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:29.963706 20213 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:29.964370 20213 raft_consensus.cc:3060] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.965058 20389 consensus_queue.cc:1048] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:29.967993 20061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f8f16c406474c98a27604be7c0aca64" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:29.968689 20061 raft_consensus.cc:3055] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:29.969009 20061 raft_consensus.cc:740] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:29.969823 20061 consensus_queue.cc:260] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.971205 20061 raft_consensus.cc:3060] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:29.976058 20213 raft_consensus.cc:2468] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:29.977485 20103 leader_election.cc:304] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:29.978406 20264 raft_consensus.cc:2804] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:29.978929 20264 raft_consensus.cc:697] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:29.979624 20264 consensus_queue.cc:237] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:29.980990 20061 raft_consensus.cc:2468] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:29.989885 19961 catalog_manager.cc:5697] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 1 to 2, leader changed from 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65) to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:30.050266 20061 raft_consensus.cc:1275] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.051695 20264 consensus_queue.cc:1048] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.059388 20213 raft_consensus.cc:1275] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.061614 20264 consensus_queue.cc:1048] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.117026 20139 raft_consensus.cc:1275] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.118324 20389 consensus_queue.cc:1048] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.125957 20213 raft_consensus.cc:1275] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.128568 20389 consensus_queue.cc:1048] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.138815 20061 raft_consensus.cc:1275] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.140272 20295 consensus_queue.cc:1048] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.149118 20139 raft_consensus.cc:1275] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.150818 20421 consensus_queue.cc:1048] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:31:30.250047 20139 raft_consensus.cc:1275] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.251470 20389 consensus_queue.cc:1048] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.260241 20213 raft_consensus.cc:1275] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.262564 20389 consensus_queue.cc:1048] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.510231 20061 raft_consensus.cc:1275] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.511660 20368 consensus_queue.cc:1048] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:30.522102 20212 raft_consensus.cc:1275] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:30.523793 20368 consensus_queue.cc:1048] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:32.485422 19960 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60048:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:31:32.531246 19960 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:60048:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:31:32.570744  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:32.898705 20062 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
mode: GRACEFUL
new_leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
 from {username='slave'} at 127.0.0.1:43042
W20260709 12:31:32.899971 20464 log.cc:927] Time spent T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:31:32.900038 20062 raft_consensus.cc:606] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Received request to transfer leadership to TS ffbb6758e9844e559889a1a5c44bcc22
W20260709 12:31:32.903250 20462 log.cc:927] Time spent T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:31:32.936972 20484 tablet.cc:1724] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:32.954890 20178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:39279, user_credentials={real_user=slave}} blocked reactor thread for 47501.6us
I20260709 12:31:32.954998 20475 tablet.cc:1724] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:32.986613 20479 log.cc:927] Time spent T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: Append to log took a long time: real 0.059s	user 0.003s	sys 0.000s
I20260709 12:31:32.988981 20489 tablet.cc:1724] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.025688 20493 tablet.cc:1724] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.040242 20488 tablet.cc:1724] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.045821 20490 tablet.cc:1724] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.055480 20494 tablet.cc:1724] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.066059 20501 tablet.cc:1724] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.068766 20499 tablet.cc:1724] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:31:33.089548 20466 log.cc:927] Time spent T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: Append to log took a long time: real 0.211s	user 0.002s	sys 0.001s
I20260709 12:31:33.110311 20491 tablet.cc:1724] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.119866 20495 tablet.cc:1724] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.138532 20139 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
mode: GRACEFUL
new_leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
 from {username='slave'} at 127.0.0.1:49806
I20260709 12:31:33.139443 20139 raft_consensus.cc:606] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Received request to transfer leadership to TS ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:33.140422 20499 tablet.cc:1724] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.143198  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:31:33.143949  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:33.129007 20491 tablet.cc:1724] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.149818 20495 tablet.cc:1724] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.161059 20491 tablet.cc:1724] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.137415 20496 tablet.cc:1724] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.173036 20489 tablet.cc:1724] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.174015 20491 tablet.cc:1724] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.187265 20495 tablet.cc:1724] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.188357 20491 tablet.cc:1724] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.210438 20484 tablet.cc:1724] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.213171 20494 tablet.cc:1724] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.209878 20475 tablet.cc:1724] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.232419 20494 tablet.cc:1724] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.233739 20495 tablet.cc:1724] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.270560 20489 tablet.cc:1724] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.273864 20499 tablet.cc:1724] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:31:33.300099 19960 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76: soft_deleted_table [id=402478a4f2e74d5da82cb7ec0d7da258] alter complete (version 1)
I20260709 12:31:33.311225 20481 raft_consensus.cc:993] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: : Instructing follower ffbb6758e9844e559889a1a5c44bcc22 to start an election
I20260709 12:31:33.311985 20469 raft_consensus.cc:1081] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Signalling peer ffbb6758e9844e559889a1a5c44bcc22 to start an election
I20260709 12:31:33.319010 20207 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
 from {username='slave'} at 127.0.0.1:53946
I20260709 12:31:33.319607 20207 raft_consensus.cc:493] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:33.320127 20207 raft_consensus.cc:3060] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:33.320860 20264 raft_consensus.cc:993] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: : Instructing follower ffbb6758e9844e559889a1a5c44bcc22 to start an election
I20260709 12:31:33.321664 20473 raft_consensus.cc:1081] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Signalling peer ffbb6758e9844e559889a1a5c44bcc22 to start an election
I20260709 12:31:33.323938 20214 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531"
dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
 from {username='slave'} at 127.0.0.1:53952
I20260709 12:31:33.324436 20214 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:33.324769 20214 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:33.327173 20207 raft_consensus.cc:515] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:33.329710 20207 leader_election.cc:290] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:33.330204 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:33.331223 20063 raft_consensus.cc:3055] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:33.331135 20138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "606dc20b79e94d53bfd993f4d5ba55d3" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:33.331941 20063 raft_consensus.cc:740] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:33.332173 20138 raft_consensus.cc:3060] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:33.333141 20063 consensus_queue.cc:260] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:33.333262 20214 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:33.334697 20063 raft_consensus.cc:3060] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:33.336712 20214 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:33.338711 20062 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:33.339434 20062 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:33.340807 20138 raft_consensus.cc:2468] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 2.
I20260709 12:31:33.341725 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:33.342347 20139 raft_consensus.cc:3055] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:33.342463 20180 leader_election.cc:304] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:33.342736 20139 raft_consensus.cc:740] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:33.343544 20063 raft_consensus.cc:2468] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 2.
I20260709 12:31:33.343665 20430 raft_consensus.cc:2804] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:33.343850 20139 consensus_queue.cc:260] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:33.344516 20430 raft_consensus.cc:697] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:33.345535 20139 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:33.345567 20430 consensus_queue.cc:237] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:33.347716 20062 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 2.
I20260709 12:31:33.348927 20178 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:33.349900 20487 raft_consensus.cc:2804] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:33.350451 20487 raft_consensus.cc:697] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:33.351657 20487 consensus_queue.cc:237] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:33.353858 20139 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 2.
I20260709 12:31:33.358431 19960 catalog_manager.cc:5697] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 1 to 2, leader changed from 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65) to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:33.371227 19960 catalog_manager.cc:5697] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 1 to 2, leader changed from 301528e0176e4eff834c92ca750633a4 (127.5.4.66) to ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67). New cstate: current_term: 2 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:34.185511  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:34.505571  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:34.506244  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:34.835175 20510 raft_consensus.cc:493] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 301528e0176e4eff834c92ca750633a4)
I20260709 12:31:34.836004 20510 raft_consensus.cc:515] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.840045 20510 leader_election.cc:290] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:34.846864 20512 raft_consensus.cc:493] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ffbb6758e9844e559889a1a5c44bcc22)
I20260709 12:31:34.847481 20512 raft_consensus.cc:515] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.849579 20512 leader_election.cc:290] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:34.855726 20062 raft_consensus.cc:1275] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:34.856529 20063 raft_consensus.cc:1275] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:34.856840 20209 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:34.857762 20206 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:34.858861 20512 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 301528e0176e4eff834c92ca750633a4)
I20260709 12:31:34.859629 20512 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.871794 20512 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:34.885087 20487 raft_consensus.cc:493] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 301528e0176e4eff834c92ca750633a4)
I20260709 12:31:34.886590 20516 consensus_queue.cc:1048] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:34.886754 20487 raft_consensus.cc:515] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.894554 20061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d7f52bd35e44dfb87d7441b4c42155d" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:34.894798 20518 consensus_queue.cc:1048] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:34.897639 20207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:34.898568 20207 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:34.900907 20028 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:34.889487 20487 leader_election.cc:290] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:34.902760 20512 raft_consensus.cc:2804] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:31:34.903298 20512 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 301528e0176e4eff834c92ca750633a4)
I20260709 12:31:34.903667 20512 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:34.899677 20514 raft_consensus.cc:493] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9f1e938b2be54950b77fb71c531a1d34)
I20260709 12:31:34.907038 20514 raft_consensus.cc:515] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.907814 20487 raft_consensus.cc:493] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 301528e0176e4eff834c92ca750633a4)
I20260709 12:31:34.910233 20487 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.910683 20207 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:34.916015 20487 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:34.916847 20138 raft_consensus.cc:1275] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:34.917003 20510 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:34.917615 20510 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.919862 20510 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:34.920250 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8a76362a29c4d9b8633412de6dec355" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:34.909505 20514 leader_election.cc:290] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:34.922525 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:34.924993 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4371078417de4d60a32baa99b7d2087a" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:34.924752 20512 raft_consensus.cc:515] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.926180 20061 raft_consensus.cc:1240] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Rejecting Update request from peer 301528e0176e4eff834c92ca750633a4 for earlier term 2. Current term is 3. Ops: []
I20260709 12:31:34.926620 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:34.930459 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:34.931119 20140 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 2.
I20260709 12:31:34.934439 20512 leader_election.cc:290] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:34.937500 20210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:34.944307 20061 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9270a5ddd9e34e5d80c3fb17458d1e8a" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:34.943163 20210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:34.947048 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d7f52bd35e44dfb87d7441b4c42155d" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:34.948835 20031 leader_election.cc:304] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34; no voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:34.950031 20139 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:34.950783 20515 raft_consensus.cc:2749] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:31:34.953841 20031 leader_election.cc:304] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34; no voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:34.955344 20513 consensus_queue.cc:1059] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:34.956341 20516 consensus_queue.cc:1048] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:31:34.956996 20180 leader_election.cc:304] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:34.958644 20180 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34
I20260709 12:31:34.958902 20514 raft_consensus.cc:3055] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:34.959175 20518 raft_consensus.cc:2749] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:34.959941 20515 raft_consensus.cc:2749] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:31:34.961853 20518 raft_consensus.cc:2749] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:31:34.962849 20104 leader_election.cc:304] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 301528e0176e4eff834c92ca750633a4; no voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:34.960896 20514 raft_consensus.cc:740] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
I20260709 12:31:34.965778 20514 consensus_queue.cc:260] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:34.967288 20514 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:34.960135 20140 raft_consensus.cc:1275] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:34.971939 20513 raft_consensus.cc:2749] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:31:34.972110 20487 consensus_queue.cc:1048] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:34.973789 20031 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:34.976408 20140 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c70cfe6a061403c91a1ccbdf236b4ed" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:34.976972 20515 raft_consensus.cc:2804] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:31:34.979447 20515 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Starting leader election (detected failure of leader ffbb6758e9844e559889a1a5c44bcc22)
I20260709 12:31:34.980057 20515 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:35.002866 20515 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:35.006876 20140 raft_consensus.cc:2468] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1e938b2be54950b77fb71c531a1d34 in term 3.
I20260709 12:31:35.010785 20515 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 election: Requested vote from peers 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:35.011410 20031 leader_election.cc:304] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:35.012548 20138 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:35.013245 20210 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "9f1e938b2be54950b77fb71c531a1d34" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:35.014285 20515 raft_consensus.cc:2804] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:31:35.014914 20515 raft_consensus.cc:697] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 3 LEADER]: Becoming Leader. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:35.015326 20028 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34; no voters: 301528e0176e4eff834c92ca750633a4, ffbb6758e9844e559889a1a5c44bcc22
I20260709 12:31:35.016320 20521 raft_consensus.cc:2749] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:31:35.016193 20515 consensus_queue.cc:237] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:35.028424 19961 catalog_manager.cc:5697] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 reported cstate change: term changed from 2 to 3, leader changed from 301528e0176e4eff834c92ca750633a4 (127.5.4.66) to 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65). New cstate: current_term: 3 leader_uuid: "9f1e938b2be54950b77fb71c531a1d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
I20260709 12:31:35.524498  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:31:35.636859 19994 debug-util.cc:398] Leaking SignalData structure 0x7b080018e020 after lost signal to thread 5140
W20260709 12:31:35.638585 19994 debug-util.cc:398] Leaking SignalData structure 0x7b08001c52a0 after lost signal to thread 19997
W20260709 12:31:35.640206 19994 debug-util.cc:398] Leaking SignalData structure 0x7b08002abd00 after lost signal to thread 20089
W20260709 12:31:35.641877 19994 debug-util.cc:398] Leaking SignalData structure 0x7b0800305d00 after lost signal to thread 20164
W20260709 12:31:35.644784 19994 debug-util.cc:398] Leaking SignalData structure 0x7b08001ea3c0 after lost signal to thread 20238
I20260709 12:31:35.667338 20065 raft_consensus.cc:1240] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Rejecting Update request from peer ffbb6758e9844e559889a1a5c44bcc22 for earlier term 2. Current term is 3. Ops: []
I20260709 12:31:35.670583 20065 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf"
dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
mode: GRACEFUL
new_leader_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:43042
I20260709 12:31:35.686206 20065 raft_consensus.cc:606] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Received request to transfer leadership to TS 301528e0176e4eff834c92ca750633a4
I20260709 12:31:35.702695  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:31:35.703565  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:35.716678 20140 raft_consensus.cc:1275] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 3 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:35.731093 20511 consensus_queue.cc:1048] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:35.734249 20539 raft_consensus.cc:993] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: : Instructing follower 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:35.734934 20539 raft_consensus.cc:1081] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Signalling peer 301528e0176e4eff834c92ca750633a4 to start an election
I20260709 12:31:35.712025 20541 consensus_queue.cc:1059] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:31:35.736207 20208 raft_consensus.cc:3060] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:35.736567 20541 raft_consensus.cc:3055] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:31:35.737099 20541 raft_consensus.cc:740] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:35.753351 20136 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf"
dest_uuid: "301528e0176e4eff834c92ca750633a4"
 from {username='slave'} at 127.0.0.1:56208
I20260709 12:31:35.754006 20136 raft_consensus.cc:493] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:31:35.754351 20136 raft_consensus.cc:3060] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:35.762140 20136 raft_consensus.cc:515] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:35.737865 20541 consensus_queue.cc:260] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:35.765328 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22"
I20260709 12:31:35.766052 20214 raft_consensus.cc:3060] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:35.768121 20541 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:31:35.769248 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b63e5939ed24b32a9a4999bff0a9fcf" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:35.769834 20063 raft_consensus.cc:3055] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:31:35.770105 20063 raft_consensus.cc:740] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f1e938b2be54950b77fb71c531a1d34, State: Running, Role: LEADER
I20260709 12:31:35.764586 20136 leader_election.cc:290] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:35.773983 20063 consensus_queue.cc:260] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:35.775249 20063 raft_consensus.cc:3060] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:31:35.780452 20063 raft_consensus.cc:2468] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:35.781803 20104 leader_election.cc:304] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 301528e0176e4eff834c92ca750633a4, 9f1e938b2be54950b77fb71c531a1d34; no voters: 
I20260709 12:31:35.782651 20544 raft_consensus.cc:2804] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:31:35.782861 20208 raft_consensus.cc:1275] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Refusing update from remote peer 9f1e938b2be54950b77fb71c531a1d34: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:31:35.783212 20544 raft_consensus.cc:697] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Becoming Leader. State: Replica: 301528e0176e4eff834c92ca750633a4, State: Running, Role: LEADER
W20260709 12:31:35.773046 20178 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.4.65:39279, user_credentials={real_user=slave}} blocked reactor thread for 65705.2us
I20260709 12:31:35.784327 20544 consensus_queue.cc:237] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:35.786410 20539 consensus_queue.cc:1048] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:35.792786 20214 raft_consensus.cc:2468] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 301528e0176e4eff834c92ca750633a4 in term 2.
I20260709 12:31:35.818382 19959 catalog_manager.cc:5697] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 reported cstate change: term changed from 1 to 2, leader changed from 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65) to 301528e0176e4eff834c92ca750633a4 (127.5.4.66). New cstate: current_term: 2 leader_uuid: "301528e0176e4eff834c92ca750633a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: UNKNOWN } } }
W20260709 12:31:36.173002 20086 debug-util.cc:398] Leaking SignalData structure 0x7b08001e0f80 after lost signal to thread 5140
W20260709 12:31:36.174592 20086 debug-util.cc:398] Leaking SignalData structure 0x7b080020f2c0 after lost signal to thread 19997
W20260709 12:31:36.176317 20086 debug-util.cc:398] Leaking SignalData structure 0x7b080024cb60 after lost signal to thread 20089
W20260709 12:31:36.177589 20086 debug-util.cc:398] Leaking SignalData structure 0x7b080033fd80 after lost signal to thread 20164
W20260709 12:31:36.178777 20086 debug-util.cc:398] Leaking SignalData structure 0x7b08000b9e00 after lost signal to thread 20238
I20260709 12:31:36.235639 20065 raft_consensus.cc:1275] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:36.237617 20514 consensus_queue.cc:1048] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:36.258808 20214 raft_consensus.cc:1275] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Refusing update from remote peer 301528e0176e4eff834c92ca750633a4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:31:36.263024 20545 consensus_queue.cc:1048] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:31:36.705008  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:36.713573  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:36.714164  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:36.932145 20546 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:31:36.932809 20546 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:36.939441 20546 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:36.940177 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:36.940972 20065 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 3.
I20260709 12:31:36.942819 20137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4" is_pre_election: true
I20260709 12:31:36.945075 20178 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 
I20260709 12:31:36.946763 20546 raft_consensus.cc:2804] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:31:36.947229 20546 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:31:36.947628 20546 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:36.958714 20546 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:36.961176 20546 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 4 election: Requested vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919)
I20260709 12:31:36.962421 20065 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34"
I20260709 12:31:36.964426 20065 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:31:36.963371 20137 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "ffbb6758e9844e559889a1a5c44bcc22" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "301528e0176e4eff834c92ca750633a4"
I20260709 12:31:36.979012 20065 raft_consensus.cc:2468] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffbb6758e9844e559889a1a5c44bcc22 in term 4.
I20260709 12:31:36.980922 20178 leader_election.cc:304] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f1e938b2be54950b77fb71c531a1d34, ffbb6758e9844e559889a1a5c44bcc22; no voters: 301528e0176e4eff834c92ca750633a4
I20260709 12:31:36.982273 20559 raft_consensus.cc:2804] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:31:36.982962 20559 raft_consensus.cc:697] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 4 LEADER]: Becoming Leader. State: Replica: ffbb6758e9844e559889a1a5c44bcc22, State: Running, Role: LEADER
I20260709 12:31:36.983886 20559 consensus_queue.cc:237] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:36.996531 19959 catalog_manager.cc:5697] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "ffbb6758e9844e559889a1a5c44bcc22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } health_report { overall_health: HEALTHY } } }
I20260709 12:31:37.310316 20558 raft_consensus.cc:493] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader ffbb6758e9844e559889a1a5c44bcc22)
I20260709 12:31:37.311020 20558 raft_consensus.cc:515] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 } } peers { permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 } } peers { permanent_uuid: "ffbb6758e9844e559889a1a5c44bcc22" member_type: VOTER last_known_addr { host: "127.5.4.67" port: 36159 } }
I20260709 12:31:37.314173 20063 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9f1e938b2be54950b77fb71c531a1d34" is_pre_election: true
I20260709 12:31:37.314603 20214 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7583a33a07df428ea67f04fe63b39531" candidate_uuid: "301528e0176e4eff834c92ca750633a4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "ffbb6758e9844e559889a1a5c44bcc22" is_pre_election: true
I20260709 12:31:37.315064 20063 raft_consensus.cc:2368] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 301528e0176e4eff834c92ca750633a4 for earlier term 3. Current term is 4.
I20260709 12:31:37.316396 20558 leader_election.cc:290] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279), ffbb6758e9844e559889a1a5c44bcc22 (127.5.4.67:36159)
I20260709 12:31:37.316521 20104 leader_election.cc:400] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279) with higher term. Message: Invalid argument: T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 301528e0176e4eff834c92ca750633a4 for earlier term 3. Current term is 4.
I20260709 12:31:37.317261 20104 leader_election.cc:403] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:31:37.317909 20544 raft_consensus.cc:3060] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:31:37.327394 20544 raft_consensus.cc:2749] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279) with higher term. Message: Invalid argument: T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 301528e0176e4eff834c92ca750633a4 for earlier term 3. Current term is 4.
I20260709 12:31:37.412724 20063 raft_consensus.cc:1275] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:31:37.414392 20559 consensus_queue.cc:1048] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1e938b2be54950b77fb71c531a1d34" member_type: VOTER last_known_addr { host: "127.5.4.65" port: 39279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:31:37.440569 20137 raft_consensus.cc:1275] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 4 FOLLOWER]: Refusing update from remote peer ffbb6758e9844e559889a1a5c44bcc22: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:31:37.442449 20546 consensus_queue.cc:1048] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "301528e0176e4eff834c92ca750633a4" member_type: VOTER last_known_addr { host: "127.5.4.66" port: 41919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:31:37.715610  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:37.725960  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:37.726779  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:38.728123  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:38.740126  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:38.740870  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:39.743724  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:39.751153  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:39.751768  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:40.752960  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:40.759456  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:40.759987  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:41.761277  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:41.767596  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:41.768146  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:42.769791  5137 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:31:42.777688  5137 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:31:42.778112  5137 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:31:43.848145  5137 tablet_server.cc:179] TabletServer@127.5.4.65:0 shutting down...
I20260709 12:31:43.903959  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:43.904903  5137 tablet_replica.cc:333] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.906191  5137 raft_consensus.cc:2243] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.907171  5137 raft_consensus.cc:2272] T ac90934570c4402ead594675203444e2 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.910327  5137 tablet_replica.cc:333] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.911049  5137 raft_consensus.cc:2243] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.911671  5137 raft_consensus.cc:2272] T 91d954ab55504e1c871e1f595131acc7 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.914397  5137 tablet_replica.cc:333] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.915125  5137 raft_consensus.cc:2243] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.915679  5137 raft_consensus.cc:2272] T 3d7f52bd35e44dfb87d7441b4c42155d P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.918219  5137 tablet_replica.cc:333] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.918912  5137 raft_consensus.cc:2243] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:43.919930  5137 raft_consensus.cc:2272] T d9ea30d483d3479a9ebd702ea048bf89 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.922585  5137 tablet_replica.cc:333] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.923233  5137 raft_consensus.cc:2243] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:43.924656  5137 raft_consensus.cc:2272] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.927966  5137 tablet_replica.cc:333] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.928602  5137 raft_consensus.cc:2243] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:43.929525  5137 raft_consensus.cc:2272] T 1e7d03c715ef4366ab4277a3db8ae2ac P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.932420  5137 tablet_replica.cc:333] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.933087  5137 raft_consensus.cc:2243] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.933713  5137 raft_consensus.cc:2272] T 7583a33a07df428ea67f04fe63b39531 P 9f1e938b2be54950b77fb71c531a1d34 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.936138  5137 tablet_replica.cc:333] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.936746  5137 raft_consensus.cc:2243] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.937265  5137 raft_consensus.cc:2272] T bc53759e2a1e4085a643cb708d27e7ca P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.940021  5137 tablet_replica.cc:333] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.940708  5137 raft_consensus.cc:2243] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.941406  5137 raft_consensus.cc:2272] T 6b63e5939ed24b32a9a4999bff0a9fcf P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.944190  5137 tablet_replica.cc:333] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.944770  5137 raft_consensus.cc:2243] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.945263  5137 raft_consensus.cc:2272] T 606dc20b79e94d53bfd993f4d5ba55d3 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.947880  5137 tablet_replica.cc:333] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.948498  5137 raft_consensus.cc:2243] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:43.949453  5137 raft_consensus.cc:2272] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.952044  5137 tablet_replica.cc:333] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.952591  5137 raft_consensus.cc:2243] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.953084  5137 raft_consensus.cc:2272] T 4371078417de4d60a32baa99b7d2087a P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.955610  5137 tablet_replica.cc:333] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.956212  5137 raft_consensus.cc:2243] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.956692  5137 raft_consensus.cc:2272] T a8a76362a29c4d9b8633412de6dec355 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.960009  5137 tablet_replica.cc:333] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.960646  5137 raft_consensus.cc:2243] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:31:43.960664 20104 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 17 similar messages]
I20260709 12:31:43.961930  5137 raft_consensus.cc:2272] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:43.965482 20104 consensus_peers.cc:597] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:43.966053  5137 tablet_replica.cc:333] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.967011  5137 raft_consensus.cc:2243] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.967980  5137 raft_consensus.cc:2272] T 9f8f16c406474c98a27604be7c0aca64 P 9f1e938b2be54950b77fb71c531a1d34 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:43.970168 20104 consensus_peers.cc:597] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:43.971381  5137 tablet_replica.cc:333] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.971969  5137 raft_consensus.cc:2243] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:31:43.973469  5137 raft_consensus.cc:2272] T 6c70cfe6a061403c91a1ccbdf236b4ed P 9f1e938b2be54950b77fb71c531a1d34 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.975838  5137 tablet_replica.cc:333] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.976367  5137 raft_consensus.cc:2243] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:43.976940  5137 raft_consensus.cc:2272] T ff2b19ad91124ab1a278e2acd46ac9d8 P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:43.979401  5137 tablet_replica.cc:333] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34: stopping tablet replica
I20260709 12:31:43.979933  5137 raft_consensus.cc:2243] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:43.980903  5137 raft_consensus.cc:2272] T 258ca134b3c142f1a31f9927b6dad39c P 9f1e938b2be54950b77fb71c531a1d34 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:43.991097 20104 consensus_peers.cc:597] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:44.023857 20178 consensus_peers.cc:597] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:44.056609  5137 tablet_server.cc:196] TabletServer@127.5.4.65:0 shutdown complete.
W20260709 12:31:44.067885 20104 consensus_peers.cc:597] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:44.095242 20104 consensus_peers.cc:597] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:44.116591  5137 tablet_server.cc:179] TabletServer@127.5.4.66:0 shutting down...
I20260709 12:31:44.161087  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:44.161974  5137 tablet_replica.cc:333] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.163214  5137 raft_consensus.cc:2243] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.163987  5137 raft_consensus.cc:2272] T ac90934570c4402ead594675203444e2 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.167281  5137 tablet_replica.cc:333] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.168365  5137 raft_consensus.cc:2243] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.170004  5137 raft_consensus.cc:2272] T 3d7f52bd35e44dfb87d7441b4c42155d P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:44.172541 20104 consensus_peers.cc:597] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:44.173941  5137 tablet_replica.cc:333] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.175125  5137 raft_consensus.cc:2243] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.176283  5137 raft_consensus.cc:2272] T 91d954ab55504e1c871e1f595131acc7 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.179486  5137 tablet_replica.cc:333] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.180141  5137 raft_consensus.cc:2243] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.180855  5137 raft_consensus.cc:2272] T 9270a5ddd9e34e5d80c3fb17458d1e8a P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.183748  5137 tablet_replica.cc:333] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.184408  5137 raft_consensus.cc:2243] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.184984  5137 raft_consensus.cc:2272] T 1e7d03c715ef4366ab4277a3db8ae2ac P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.188530  5137 tablet_replica.cc:333] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.189393  5137 raft_consensus.cc:2243] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.190234  5137 raft_consensus.cc:2272] T d9ea30d483d3479a9ebd702ea048bf89 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:31:44.191895 20180 consensus_peers.cc:597] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 -> Peer 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919): Couldn't send request to peer 301528e0176e4eff834c92ca750633a4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:44.193902  5137 tablet_replica.cc:333] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.194800  5137 raft_consensus.cc:2243] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.195367  5137 raft_consensus.cc:2272] T 7583a33a07df428ea67f04fe63b39531 P 301528e0176e4eff834c92ca750633a4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.198266  5137 tablet_replica.cc:333] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.198928  5137 raft_consensus.cc:2243] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.200071  5137 raft_consensus.cc:2272] T bc53759e2a1e4085a643cb708d27e7ca P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.203078  5137 tablet_replica.cc:333] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.203786  5137 raft_consensus.cc:2243] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.204324  5137 raft_consensus.cc:2272] T 990d8c3ee6ea4287be6ea7395d2f8a80 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.207082  5137 tablet_replica.cc:333] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.207679  5137 raft_consensus.cc:2243] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.208242  5137 raft_consensus.cc:2272] T a90e46fa6f0841c5a441d6f7b2f4ea4b P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.211143  5137 tablet_replica.cc:333] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.211838  5137 raft_consensus.cc:2243] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.212790  5137 raft_consensus.cc:2272] T 6b63e5939ed24b32a9a4999bff0a9fcf P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.215493  5137 tablet_replica.cc:333] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.216068  5137 raft_consensus.cc:2243] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.216533  5137 raft_consensus.cc:2272] T 606dc20b79e94d53bfd993f4d5ba55d3 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.219236  5137 tablet_replica.cc:333] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.219854  5137 raft_consensus.cc:2243] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.220824  5137 raft_consensus.cc:2272] T 9f8f16c406474c98a27604be7c0aca64 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.223341  5137 tablet_replica.cc:333] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.223897  5137 raft_consensus.cc:2243] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.224361  5137 raft_consensus.cc:2272] T 6c70cfe6a061403c91a1ccbdf236b4ed P 301528e0176e4eff834c92ca750633a4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.226886  5137 tablet_replica.cc:333] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.227403  5137 raft_consensus.cc:2243] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.228439  5137 raft_consensus.cc:2272] T ff2b19ad91124ab1a278e2acd46ac9d8 P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.231228  5137 tablet_replica.cc:333] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.231858  5137 raft_consensus.cc:2243] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.232463  5137 raft_consensus.cc:2272] T 258ca134b3c142f1a31f9927b6dad39c P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.235162  5137 tablet_replica.cc:333] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.235795  5137 raft_consensus.cc:2243] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.236348  5137 raft_consensus.cc:2272] T 4371078417de4d60a32baa99b7d2087a P 301528e0176e4eff834c92ca750633a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.239159  5137 tablet_replica.cc:333] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4: stopping tablet replica
I20260709 12:31:44.239773  5137 raft_consensus.cc:2243] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.240891  5137 raft_consensus.cc:2272] T a8a76362a29c4d9b8633412de6dec355 P 301528e0176e4eff834c92ca750633a4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.292773  5137 tablet_server.cc:196] TabletServer@127.5.4.66:0 shutdown complete.
W20260709 12:31:44.307258 20180 consensus_peers.cc:597] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 -> Peer 301528e0176e4eff834c92ca750633a4 (127.5.4.66:41919): Couldn't send request to peer 301528e0176e4eff834c92ca750633a4. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.66:41919: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:44.326387 20178 consensus_peers.cc:597] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:44.346572  5137 tablet_server.cc:179] TabletServer@127.5.4.67:0 shutting down...
W20260709 12:31:44.373647 20178 consensus_peers.cc:597] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:31:44.378840 20178 consensus_peers.cc:597] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 -> Peer 9f1e938b2be54950b77fb71c531a1d34 (127.5.4.65:39279): Couldn't send request to peer 9f1e938b2be54950b77fb71c531a1d34. Status: Network error: Client connection negotiation failed: client connection to 127.5.4.65:39279: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:31:44.390450  5137 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:31:44.391491  5137 tablet_replica.cc:333] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.392392  5137 raft_consensus.cc:2243] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.393486  5137 raft_consensus.cc:2272] T ac90934570c4402ead594675203444e2 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.396354  5137 tablet_replica.cc:333] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.397046  5137 raft_consensus.cc:2243] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.397672  5137 raft_consensus.cc:2272] T 91d954ab55504e1c871e1f595131acc7 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.400345  5137 tablet_replica.cc:333] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.401021  5137 raft_consensus.cc:2243] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.401600  5137 raft_consensus.cc:2272] T 3d7f52bd35e44dfb87d7441b4c42155d P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.404261  5137 tablet_replica.cc:333] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.404875  5137 raft_consensus.cc:2243] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.405404  5137 raft_consensus.cc:2272] T 9270a5ddd9e34e5d80c3fb17458d1e8a P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.408174  5137 tablet_replica.cc:333] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.408804  5137 raft_consensus.cc:2243] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.409369  5137 raft_consensus.cc:2272] T d9ea30d483d3479a9ebd702ea048bf89 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.411803  5137 tablet_replica.cc:333] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.412370  5137 raft_consensus.cc:2243] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.412951  5137 raft_consensus.cc:2272] T 1e7d03c715ef4366ab4277a3db8ae2ac P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.415385  5137 tablet_replica.cc:333] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.415987  5137 raft_consensus.cc:2243] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.416556  5137 raft_consensus.cc:2272] T bc53759e2a1e4085a643cb708d27e7ca P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.418973  5137 tablet_replica.cc:333] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.419576  5137 raft_consensus.cc:2243] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.420568  5137 raft_consensus.cc:2272] T 7583a33a07df428ea67f04fe63b39531 P ffbb6758e9844e559889a1a5c44bcc22 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.423062  5137 tablet_replica.cc:333] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.423677  5137 raft_consensus.cc:2243] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.424211  5137 raft_consensus.cc:2272] T 990d8c3ee6ea4287be6ea7395d2f8a80 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.427035  5137 tablet_replica.cc:333] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.427657  5137 raft_consensus.cc:2243] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.428689  5137 raft_consensus.cc:2272] T 606dc20b79e94d53bfd993f4d5ba55d3 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.431039  5137 tablet_replica.cc:333] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.431677  5137 raft_consensus.cc:2243] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.432236  5137 raft_consensus.cc:2272] T 6b63e5939ed24b32a9a4999bff0a9fcf P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.434605  5137 tablet_replica.cc:333] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.435207  5137 raft_consensus.cc:2243] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.435758  5137 raft_consensus.cc:2272] T a90e46fa6f0841c5a441d6f7b2f4ea4b P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.438179  5137 tablet_replica.cc:333] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.438756  5137 raft_consensus.cc:2243] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.439713  5137 raft_consensus.cc:2272] T 4371078417de4d60a32baa99b7d2087a P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.442049  5137 tablet_replica.cc:333] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.442793  5137 raft_consensus.cc:2243] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.443441  5137 raft_consensus.cc:2272] T a8a76362a29c4d9b8633412de6dec355 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.445881  5137 tablet_replica.cc:333] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.446431  5137 raft_consensus.cc:2243] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.446961  5137 raft_consensus.cc:2272] T ff2b19ad91124ab1a278e2acd46ac9d8 P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.449307  5137 tablet_replica.cc:333] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.449880  5137 raft_consensus.cc:2243] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.450382  5137 raft_consensus.cc:2272] T 258ca134b3c142f1a31f9927b6dad39c P ffbb6758e9844e559889a1a5c44bcc22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.452736  5137 tablet_replica.cc:333] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.453259  5137 raft_consensus.cc:2243] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.453776  5137 raft_consensus.cc:2272] T 9f8f16c406474c98a27604be7c0aca64 P ffbb6758e9844e559889a1a5c44bcc22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.456179  5137 tablet_replica.cc:333] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22: stopping tablet replica
I20260709 12:31:44.456657  5137 raft_consensus.cc:2243] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:31:44.457226  5137 raft_consensus.cc:2272] T 6c70cfe6a061403c91a1ccbdf236b4ed P ffbb6758e9844e559889a1a5c44bcc22 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.503515  5137 tablet_server.cc:196] TabletServer@127.5.4.67:0 shutdown complete.
I20260709 12:31:44.543931  5137 master.cc:562] Master@127.5.4.126:34231 shutting down...
I20260709 12:31:44.577744  5137 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:31:44.578583  5137 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:31:44.579159  5137 tablet_replica.cc:333] T 00000000000000000000000000000000 P 711e6333830a4d7f9b0331814a98eb76: stopping tablet replica
I20260709 12:31:44.628505  5137 master.cc:584] Master@127.5.4.126:34231 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (21434 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (43137 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (498918 ms total)
[  PASSED  ] 8 tests.
I20260709 12:31:44.695142  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2165 messages since previous log ~33 seconds ago
I20260709 12:31:44.695410  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2165 messages since previous log ~33 seconds ago
I20260709 12:31:44.695650  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2161 messages since previous log ~33 seconds ago
I20260709 12:31:44.695906  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:31:44.696123  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~51 seconds ago
I20260709 12:31:44.696362  5137 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 2 messages since previous log ~229 seconds ago
I20260709 12:31:44.696529  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 2 messages since previous log ~229 seconds ago
I20260709 12:31:44.696798  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 1 messages since previous log ~158 seconds ago
I20260709 12:31:44.697085  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 12:31:44.697407  5137 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~20 seconds ago
